Basicaly they're a diskless, CD-less, casette-less walkman.
We bought our kids a MP3 player, which plays videos, show pictures and plays mp3 music for about $80.00 with 2Gb RAM from dealsdirect.com. I just had a look and the one we got has sold out, but this is the site
http://www.dealsdirect.com.au/c/mp3-players/
When we bought some tunes from Itunes they only come in MP4 format, which is Applesmac's Ipod format. Itunes lock them somehow so you can't convert them to mp3 and thus they couldn't play them on the mp3 player. I think they do this coz Itunes is also owned by applemac and they want to force you to buy an Ipod.
My wife then bought a mp4 to mp3 converter which cracks the lock for $20.00US and now they can all be played.
Basically the way they work is when you connect them to your PC they show up as a removeable hard drive. You can drag & drop music, picture or movies files onto it. Once disconnected form the PC it has a very small operating system that automatically starts a graphical menu system that lets you select what you want to play. 2Gb will allow you store somewhere around 700 tunes on it.