Here's another possible solution.
The whole idea behind Random Verification Numbers (RVN's) is to provide, in essence, two factor authentication. Two factor authentication works on the idea of needing, well, two things to authenticate. In the classic South African banking story, these two things are your login details, and your cellphone (more accurately, SIM card), which shows you a code that can only be used once. Without both the login details and your SIM card, you can't get in. Which is completely infallible. Not.
There's a wonderful thing called Google Authenticator, which works exactly the same way as those little LCD keyrings that generate the pins that my dad was using to log in to the VPN at work 10 years ago: they have an algorithm which generates codes that change every 30 seconds. Providing the time on your phone and the time on the server are synchronised (easy with NTP), you just enter the code that's on your phone, and yay, you can get into your gmail account (assuming you also know the login details).
Changing over to something like Google Authenticator (which, I might add, is open source) will mean that a SIM swap would be useless as a means to get into one's bank account. Obviously they could just steal your phone, but hunting someone down and stealing their phone is a fair bit more difficult than getting a SIM swap. You'd need to have a phone which can run Google Authenticator (or whatever other app they decide to use), but most of the people having money stolen by means of SIM swaps probably have smartphones anyway. Aaaand there are Java Google Authenticator apps available for the older phones anyway.
Added bonus: never again will you have the issue that your RVN just won't come through when you really need to do something.