The auto completion is very convenient but sometimes you mistype so you end up with a list full of irrelevant items making editing less efficient.
Often you can use the down arrow + DEL key in Windows to remove an item from the list but this doesn't work in WMP.
The only way I know is to use regedit.
Go to HKEY_CURRENT_USER\Software\Microsoft\MediaPlayer\AutoComplete and delete the values you don't like.