Technically it shouldn't, but practically line speed does have a big impact on latency you experience in game.
Remember, when you do ICMP ping it is a tiny packet, so you won't see a large difference on the different speed lines, maybe slight but almost none.
Now when it come to games, their network traffic sometimes carry more data then older games ect, and this is where a faster line will have a lower latency in-game, because there is more lanes to keep the latency down to minimum, as opposed to a slower line, where the packets might get queued up.
Makes sense why my local ping dropped from 45ms to 29ms after upgrading my line from 2meg to 4meg.
Not a big drop but the OCD crowd might say it HUGE