I have repaired my 5th gear problem (it used to keep popping out). This involved replacing the sleeve and the actual 5th gear as well as the components that make up the synchronizer mechanism.
It is now extremely smooth - no rattles or vibrations - I think better than when I first bought the car..
I have also noticed that shifting into other gears especially reverse is now much smoother - without that notchy feeling. This could be down to the new gearbox oil that I put in.
On a scale of 1 - 10 this job is about 5 in complexity, and the only special tools I needed was a puller (I used a harmonic pulley puller) (like this - ancs; http://www.ebay.co.uk/itm/PRO-13PC-HARMONIC-BALANCE-GEAR-PULLEYS-PULLER-SET-FLYWHEEL-CRANK-SHAFT-PULLEYS-/180858460517?pt=UK_Hand_Tools_Equipment&hash=item2a1c011965) - though I had to buy the M8 bolts that attached the puller to the synchro hub. These are standard HT M8s with a standard thread pitch (I obtained these at Halfords).
The pusher (to push the hub back on) was fashioned out of a large socket - plus 2 ring spanners, a M12 bolt (100mm) with a fine thread pitch (1.5mm). The two spanners were used as spacers on the pusher as the M12 bolt's thread length was about 30mm only.
Total time to do this job was about 6 hours all in - could be done quicker I guess but this was my first time.
The information in R4W is valid for my model (RAV 4.2) - the gearbox looks identical.
The only brainy bit that I had to worry about was putting the synchroniser back together - I used the exploded diagrams of the gearbox components that anchorman kindly pointed me at 
to help me here.
When I looked at the replacement parts (Thanks Kingo! ) and compared them to what I pulled out of the gearbox the only visible difference I could see was between the gear teeth patterns on the 5th gear and the sleeve. These were quite different - and could be down to wear.
I dont believe the synchro hub was the problem here - the components (Rings springs etc all looked fine) - however I changed these as well.
I think I saved about £400 in labour. 
I took some pictures (of my pusher) as well as pictures that look like those on the site above..I can post those as well if anyone is interested.
This is the parts list I used:
T08826-00090 C106 SEAL PACKING (gasket maker)
T33336-42030 N/STK FIFTH gear
T90364-28023 N/STK NEEDEL BEARING
T33397-28010 N/STK SYNCHRO-RING
T33396-28010 N/STK SYNCHRO-RING
T33395-42010 N/STK TRANS.HUB&SLEE
T90179-20012 N/STK NUT
T33398-12010 N/STK SYNCHRO-RING
T33393-42010 N/STK TRANS.HUB&SLEE
T33399-12011 N/STK SYNCHRO-RING
T33392-12010 Q1074 SPRING, SHFTING
T33392-12020 Q1074 SPRING, SHFTING
-phil
Later, Chris (Tech01) added this information;
I've had the annoying gear lever buzz in 5th (only). It started about 18 months ago, and slowly became a hammering rattle, which no amount of light finger pressure on the gearstick would prevent. Following various folks' advice, I checked over the gearcables and their rubber bushing, and found no problem. I re-greased the gear lever's ball/socket and joint mechanism, and removed and greased up all the swivels and balljoints at the gearbox end of the shift cables (the bell-crank mechanism). This produced a small degree of muting, but no cure. The gearstick vibration continued, worst at full throttle and at 100-120kph (60-75mph), always and only in 5th.
I concluded that I had a worn Fifth gear (the subject of TSB WS-3002, Dec.2003), and last week replaced the complete 5thgearcluster. The parts list above was right for my 2003 4.2, except for the second item: 33364-42020, instead of 33364-42030. (Also, Nut 90179-20012, I found I didn't need, as it wasn't necessary to remove the layshaft-mounted gear.)
All in, about a four hour job, the only fiddly bit being the assembly of the synchro components. It is essential to make up a disc or plate puller, as suggested, to remove the gear assembly (after first tapping away the thick circlip on the shaft end). Do not try to use a hooked puller on the gear cluster assembly: it’s a very snug fit on the splined shaft, and you’ll break something if you try. Make up a puller, and use the three threaded holes with three suitable bolts. I know you’re going to replace all the bits, but, if they break up and fall out, you’ll never know what goes where. I speak from experience here!
Refilled the box with 75W-80 (has always worked for me, and far better changes than with 75W-90), and it drives and shifts like new. No vibration whatsoever, and a nice positively-located feel to fifth gear.
My local Toyota (main agent) mechanics (Malaga, Spain) say they have replaced countless fifth gear assemblies, on RAVs from late ‘90s onwards. New parts “always” cure the problem, they tell me.
Cost of parts was 658 Euro, plus vat.
It looks to me as if a buzzing gear lever is very likely an early-warning for a fifth gear wear problem of failure waiting to happen. By all means check the cable lubrication (both ends) and bushing, but don’t hold your breath.
Chris
I will add any relevant pdf's asap - ancs.
