How are the batteries connected? ( in parallel, series or a combination of the two).
It would be fun to put a lithium battery in there, just find one with the correct voltage. If you want to get fancy you could also get a charging circuit and build that in and charge it via usb.
Edit: if they are connected in parallel (6v) than you may be even be able to power it with a power bank that outputs 5v.
You should be able to 3d print a new "power pack" with 2 or 4 18650 lithium cells, charging/balancing board and a DC to DC buck converter inside without too much issue.
could i just take a power bank and use a DC buck converter to up voltage to 6v and then connect it inside, that way its easy to charge the lithium battery