The thought has occurred to me. I took the modem with me (it has a battery) and tried it at different areas in the CBD. It worked at other spots. It's working today but speedtest results are baffling. Sometimes I get low downstream speeds and high upstream speeds.
I am now even trying an LTE antenna that I used at home a few years back. So maybe it's a signal issue or congestion. At least for now my connection is usable.