Well actually line speed does affect latency, only problem is I can't find the document that explained it properly. Doing a icmp ping in command prompt won't show you much of a difference from 4 to 10mbit, however once you start working with larger packets the difference become noticable.
To explain it, if there are a road with 1 lane, and 4 cars try to get into this 1 lane at the same time, they will have to go one after another causing a delay. Now when you go to a road with 2 lanes this will improve drastically, and once you go to a road with 4 lanes there will be no delay.
Thus it depends on the amount of data you want to send and receive through simultaneously. Now WoW doesn't use enough bandwidth to get to the point where there are more cars than lanes, in fact there will always be more lanes than cars, even for the upstream bandwidth.
Once your line get to 1mbit and up you will almost never notice the delay. What I have noticed is, I can download at 500kbyte/sec and upload at 50kbyte/sec in P2P applications and barely notice a difference in games, because there is still enough bandwidth left for the game, but once I max it out there is a increase in latency because te gaming packets have to wait for the P2P packets.
Conclusion, technically a bigger pine will give better latency, but un-noticable to most people. Technology causes bigger differences in most cases, Fibre connections will give you the better latency over ADSL, but most people won't afford that type of connection.
