Not a huge difference. 384k gets 40KB/s max off a good download source. 512KB should get about 55KB/s.
So, let's take a 1GB download - assuming that it goes at max speed the whole time. (Has to be a very good source, in other words.) On a 384k line, it will take roughly seven and a half hours to download. On a 512k line, it will take roughly five and a half. (Assuming my maths is correct.)
So unless you're sharing the line with someone else, I'd recommend sticking with 384k.