VOX is down

Status
Not open for further replies.
No messages in a while here so i'm just reporting that there is definitely some issues. Our Ironport is still unreachable so i assume some hosted services are still down.
 
VOX should have a deep think about their client communications strategy. We've been with VOX for a very very long time (corporate client), and i used to get tons of updates on SMS (DataPro days even) for critical outages. Which, is actually very good.
I know it costs money (pennies really), but if everything goes down, how are you supposed to communicate with clients? Mail goes down, VOIP goes down, Internet goes down. The only thing you can somewhat trust is not something you are providing to clients.

The previous HUGE outage (storage?) had the same problem. Just no information at all flowing out of VOX (the first notice was from CEO, and that was days later). I would be happy just to be ABLE to register for technical alerts on a secondary email address at gmail.com or outlook.com so i can setup that on my phone, but, guess what, VOX doesn't send alerts AT ALL about anything and there is no portal to register for statuses, or even anything support related.

Outages happen, it is fine, crap hits the fan, Murphy loves the tech sector, even we had a huge outage here 2 weeks ago (also power, whole DC switched off) on a Saturday and the first thing you do is call people to let them know there is an outage so they can activate contingencies. Literally, Step 1 - Contact affected parties.
Preferably one a client does not have to initiate and phone,google,forum trawl and thus to find out nothing specific anyway to indicate level of contingency necessary.

Here is to hoping!

ps. I asked account manager about this "communications hole" after the previous huge outage and it clearly didn't get very far.
 
VertigoOne said:
No messages in a while here so i'm just reporting that there is definitely some issues. Our Ironport is still unreachable so i assume some hosted services are still down.
Click to expand...

For such a thing, please contact the Service Center, they are geared to assist with this type of a problem and have a full check list of what and who is still a problem.

Please escalate through them accordingly.

Thanks!
 
VertigoOne said:
VOX should have a deep think about their client communications strategy. We've been with VOX for a very very long time (corporate client), and i used to get tons of updates on SMS (DataPro days even) for critical outages. Which, is actually very good.
I know it costs money (pennies really), but if everything goes down, how are you supposed to communicate with clients? Mail goes down, VOIP goes down, Internet goes down. The only thing you can somewhat trust is not something you are providing to clients.

The previous HUGE outage (storage?) had the same problem. Just no information at all flowing out of VOX (the first notice was from CEO, and that was days later). I would be happy just to be ABLE to register for technical alerts on a secondary email address at gmail.com or outlook.com so i can setup that on my phone, but, guess what, VOX doesn't send alerts AT ALL about anything and there is no portal to register for statuses, or even anything support related.

Outages happen, it is fine, crap hits the fan, Murphy loves the tech sector, even we had a huge outage here 2 weeks ago (also power, whole DC switched off) on a Saturday and the first thing you do is call people to let them know there is an outage so they can activate contingencies. Literally, Step 1 - Contact affected parties.
Preferably one a client does not have to initiate and phone,google,forum trawl and thus to find out nothing specific anyway to indicate level of contingency necessary.

Here is to hoping!

ps. I asked account manager about this "communications hole" after the previous huge outage and it clearly didn't get very far.
Click to expand...

I couldn't agree with you more.

This topic has been mentioned a few times in the hallways and during a postmortem meeting, this topic will be raised by a few people I'm certain.

While we are still restoring the odd customer who is still down, a full analysis has not yet been concluded to figure out exactly what went wrong downstairs. There are a few different perspectives and different stories that are circulating and the problem with this, is that until an absolute answer can be derived, it's hearsay and that leads to perception issues.

We lost power, be there no mistake, but why the power was lost is the question. We have several UPS systems to carry us over until the generator picks up the load, and a back up generator in case the first one fails.

Everything that should be in place, is in place, unfortunately something went very wrong before the generator could pick up the power.

I'm certain a public announcement of the reality will be published, but only after the issue is clearly defined.

-Chris
 
All our services (70+ clients) are up again.
Ironport also running like it should.

The Communication from Vox side wasn't great at all BUT-
I must commend the engineers in DC3 and 4.
I don't think it's easy resolving the issue and doing client relations at the same time.
The guys we spoke to always picked up their phones even when sitting on the dc floor trying to get their stuff online too.
Eager to help, even when little can be done while waiting for storage to come online.

We've been with Vox for a good couple of years now.
Don't see us moving anytime soon.
Awesome technical support with (in general :) ) good turn around times.
 
Big big thank you to MMorris for all the help and feedback! You are a lifesaver!

Looks like our dedicated unit gave out :cry:, VOX is hard at work moving services over to the shared environment with same settings.

I must agree with chAmp0110, the engineers do provide excellent feedback/info at all times i have had to phone. The service overall is so good i maybe phone once or twice a year. If VOX can just get that initial outage notices done more "push" instead of all "pull", it would be fine! We then know (quicker) what is going on and what is affected, clients have some idea what contingencies to implement based on that information, and we feed this information downstream to our sponsors and users so everybody knows what is going on.

Best of luck!
 
cvanwie said:
Send me your username to my inbox, also are you on Fibrehoods, Telkom FTTH, something else?
Click to expand...

I am with Vumatel in Killarney. The FTTH account was activated on Friday. So best I can do is send my account number. Will that do?
 
VertigoOne said:
If VOX can just get that initial outage notices done more "push" instead of all "pull", it would be fine!
Click to expand...

Feel free to use this forum as your outages source.

I know when I'm experiencing any sort of technical glitch associated with the internet the first thing I do is check mybroadband new posts.

If there's a general issue it will make the top 20 on the new posts list.

Not ideal as hearing the news from the ISP direct, but definitely a close 2nd. In fact a lot of the time the ISP learns they have a problem after picking up feedback from the forum.
 
sand_man said:
Feel free to use this forum as your outages source.

I know when I'm experiencing any sort of technical glitch associated with the internet the first thing I do is check mybroadband new posts.

If there's a general issue it will make the top 20 on the new posts list.

Not ideal as hearing the news from the ISP direct, but definitely a close 2nd. In fact a lot of the time the ISP learns they have a problem after picking up feedback from the forum.
Click to expand...
I noticed that anomaly, and the ISP's say "You're the only one reporting a problem" which is quickly debunked by the forum.

It's also a shame nobody plays online games either so we have to inform them of problems there too.
 
FresJunker said:
I am with Vumatel in Killarney. The FTTH account was activated on Friday. So best I can do is send my account number. Will that do?
Click to expand...

That's not what I wanted to hear... Yes, please send me any info, you might find something that looks like C###### or order number, or something like that. I can then find you on the network.

The problem here is that Vumatel manages the speed rates, we don't. If they aren't, I'd like to escalate to them but I need a user to show them where the problem is.

Send to my inbox when you can!

Thanks.
 
AntiGanda said:
I noticed that anomaly, and the ISP's say "You're the only one reporting a problem" which is quickly debunked by the forum.

It's also a shame nobody plays online games either so we have to inform them of problems there too.
Click to expand...

AntiGanda, there is a problem in Cape Town during peak times that we are trying to sort out with Telkom. You'll notice that not everyone in Cape Town has a problem, which instantly negates that its our network and rather shines a light to the Telkom infrastructure which has been explained a number of times.

Further, your ping of 175-220 to the UK is more than normal and acceptable. If you were pinging 300+ I'd agree with you.
 
The last time I checked [last night], I still could not browse the web. The LED light on the far right of my Dlink 2640U stays red. Under advanced settings, no gateway IP address is detected. Vox is my ISP, so I assumed that this has to do with the electrical outage reported on My Broadband. My line is reported as running at half its usual 2mbps.

I'm in Selborne, East London, and connect to the Chiselhurst exchange.
 
cvanwie said:
You'll notice that not everyone in Cape Town has a problem, which instantly negates that its our network and rather shines a light to the Telkom infrastructure which has been explained a number of times.
Click to expand...
After being with several ISPs, seems everyone always has issues in CT in particular. Its always a thorn in the side thats never pinned down.
Makes a certain sense that it would be "the common" factor at fault.
 
cvanwie said:
AntiGanda, there is a problem in Cape Town during peak times that we are trying to sort out with Telkom. You'll notice that not everyone in Cape Town has a problem, which instantly negates that its our network and rather shines a light to the Telkom infrastructure which has been explained a number of times.

Further, your ping of 175-220 to the UK is more than normal and acceptable. If you were pinging 300+ I'd agree with you.
Click to expand...

So what's happening now with pings?

C:\Users\Ant>tracert www.twitter.com

Tracing route to twitter.com [199.16.156.70]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms 192.168.1.1
2 74 ms 9 ms 9 ms dsl-197-245-124-1.voxdsl.co.za [197.245.124.1]
3 11 ms 10 ms 10 ms 41.193.121.9
4 8 ms 9 ms 8 ms 41.193.121.10
5 15 ms 11 ms 12 ms 209.203.1.41
6 10 ms 10 ms 9 ms 41.193.120.113
7 175 ms 176 ms 180 ms 196.41.24.174
8 177 ms 255 ms 176 ms xe-10-2-2.edge4.London1.Level3.net [195.50.120.1
3]
9 177 ms 176 ms 288 ms ae-125-3511.edge5.london1.Level3.net [4.69.166.4
1]
10 184 ms 176 ms 297 ms ae-125-3511.edge5.london1.Level3.net [4.69.166.4
1]
11 176 ms 176 ms 176 ms TWITTER-INC.edge5.London1.Level3.net [195.50.120
.66]
12 284 ms 282 ms 609 ms ae65.atl1-er2.twttr.com [192.133.76.19]
13 276 ms 277 ms * 199.16.156.70
14 394 ms 276 ms 276 ms 199.16.156.70

Trace complete.

C:\Users\Ant>
 
Status
Not open for further replies.
Back
Top
Sign up to the MyBroadband newsletter
X