I have the same problem with my little 2G stick. it starts with a bang and then it slows down to about 400kbps. Can take about 20 minutes to transfer 1.7 gig. I thought its just me having this problem...glad to see I am not alone. It has FAT file system on the stick.
According to the last link, the workaround is to use the cli, i.e. the cp command, to transfer files.
Also, it seems to be a Gnome problem from some comments.
The problem is not the transfer as such, but ending or closing the session, those last few bits. I have ignored this and just carried on working while I waited for the session to close. And I can confirm that it is not just Gnome, I only use KDE and have this issue. Will try CLi when I have a chance to see if it works better.