I want to get a camera for the kid for his Birthday. Cost is not much of an issue (largely because I intend on borrowing it a lot!), but let's target a max of $350. He's going to be 8 years old. So, I like the idea of a 'tough' camera as we are always at the beach and in the water etc. I have also seen one or two cameras saying they are 'fast' because of their f2.0 setting. I really see little to be gained from things like GPS, wifi and other gimmickry on a camera. I just want a camera that takes good shots, easy to use and offers a certain level of robustness and waterproofness (sic!).
If you're looking for something that will basically be indestructible but still take some decent pictures at a decent price, I'd recommend one from the Olympus Tough series. They're not really my first choice for a compact camera, but they really can stand up to a beating (and many of them are fully waterproof).
The Olympus TG-1 has a reasonably fast lens (f/2 at 25mm eq - not sure why all of these fast compacts have these fast lenses but then drop off at any focal length but ultra wide) but the sensor is a little small for my taste. Regardless, it's waterproof up to 40ft and has some cool ruggedized technology... you could probably run it over with a car and still pick it up and start shooting.
If rugged isn't the top priority, you could also go for a M43 camera; the Olympus E-PM1 kit or the Panasonic GF3 kit are both pretty cheap now. Neither would be nearly as durable as one of the tough cameras, but it will surely take much better pictures. The good thing about that is that it would leave some room for expansion if this hobby is something that takes off. I love my old Panasonic GF2 with the 20mm f/1.7 lens - it's pocketable and takes reasonably good images for its size.
If you already have a system and some lenses around, the other option could be to just pick up a cheap used old DSLR body. I know you can typically get a used bodies from a few years ago on B&H for about $250 (USD).