WebAfrica - get your **** together!

D

Desteffe

New Member
Joined
Jan 12, 2011
Messages
6
Reaction score
0
Location
Hoedspruit
Since 6:30 AM yesterday, one of WebAfrica's mail servers is down. Which means a whole bunch of businesses that haven't been receiving emails for almost 2 full business days now. And no one can tell us when it'll be sorted out - not the call centre, not their Facebook/Twitter admins, it's not even on their own forum.

All we get is "engineers working on it, apologies for the inconvenience".

Really? Come on WA, you can do better than that.

What's your CEO up to - why aren't we hearing from him?

What's the action plan? How you are going to prevent your customers from ever losing business again because of your stuff-up?

So many questions, and the ones at Webafrica that have the answers aren't talking to their customers.
 
Look at this for a joke!! :

[ZA] - Winmails01.cpt.wa.co.za - Outage 2013/05/22 15:33:49
We are commencing with the emergency maintenance. We expect downtime to be less than 30 minutes. We will provide further updates as soon as services are restored.

We apologise for any inconvenience this may cause.
[ZA] - Winmails01.cpt.wa.co.za - Outage
Published: 2013/05/22 14:30:39
We will be performing emergency maintenance at around 15h30. A notification will be posted when this maintenance begins.

We apologise for any inconvenience this may cause.
[ZA] - Winmails01.cpt.wa.co.za - Outage
Published: 2013/05/22 11:45:11
We are continuing to investigate authentication issues experienced by some users.

We apologise for any inconvenience this may cause.
[ZA] - Winmails01.cpt.wa.co.za - Outage
Published: 2013/05/22 09:14:49
Engineers are still investigating intermittent authentication issues some users continue to experience.
[ZA] - Winmails01.cpt.wa.co.za - Outage
Published: 2013/05/22 07:16:46
We have noticed authentication issues on the server, we are currently investigating this.
[ZA] - Winmails01.cpt.wa.co.za - Outage
Published: 2013/05/21 21:40:18
The filesystem check took much longer than originally anticipated.

All services have been restored. We will however continue to monitor the server closely.
[ZA] - Winmails01.cpt.wa.co.za - Outage
Published: 2013/05/21 14:47:12
The server is currently running a filesystem check. We will update as soon as this operation has been completed.

We apologise for any inconvenience caused.
[ZA] - Winmails01.cpt.wa.co.za - Outage
Published: 2013/05/21 14:07:15
The hardware replacement has been completed. We will update shortly regarding service restoration.

We apologize for any inconvenience caused.
[ZA] - Winmails01.cpt.wa.co.za - Outage
Published: 2013/05/21 12:06:31
The hardware replacement is taking longer than expected. Our engineers are working to resolve the issue as soons as possible.

We apologize for any inconvenience caused.
[ZA] - Winmails01.cpt.wa.co.za - Outage
Published: 2013/05/21 09:51:03
Our engineers are on site replacing hardware, ETR is 90 minutes.
[ZA] - Winmails01.cpt.wa.co.za - Outage
Published: 2013/05/21 08:14:30
We have isolated the issue to hardware failure, we are in the process of replacing the hardware.
[ZA] - Winmails01.cpt.wa.co.za - Outage
Published: 2013/05/21 07:03:41
Date: 21 May 2013
Attending: Server Administrators
Impact: All mail domains hosted on this server are currently inaccessible.

We apologise for any inconvenience caused.

:mad:
 
Thats not very nice!

Surely their hosting is virtualized and could have been moved across to a new hardware node, at least that's what we would have done.
If it is a physical machine, then they should invest in some bare metal backups. From the sounds of it, it seems like a raid card failure and its possibly messed up some data on the discs. If they running raid 5 then its possibly caused a URE (unrecoverable read error).

Goodluck to them, I hope they manage to get it sorted for their customers sake.
 
Last edited:
Murmaider said:
Thats not very nice!

Surely their hosting is virtualized and could have been moved across to a new hardware node, at least that's what we would have done.
If it is a physical machine, then they should invest in some bare metal backups. From the sounds of it, it seems like a raid card failure and its possibly messed up some data on the discs. If they running raid 5 then its possibly caused a URE (unrecoverable read error).

Goodluck to them, I hope they manage to get it sorted for their customers sake.
Click to expand...

(Visualization + replication)*some clever DNS = HA
If you aren't monitoring the nodes you won't even know your own servers are down

Who still hosts on the hardware layer?
 
The really bad part of this fiasco is no feedback from WA between 1:00am and 7:00am this morning indicating that they went to bed at 1:00am leaving us in the lurch.
 
Hi,

We had outages on 2 of our mail servers which unfortunately took much longer to fix as we initially thought. The one server was brought back online yesterday morning but the other was only back up late yesterday evening. This outage has impacted the affected customers in a really bad way and we have to apologise to everyone affected.

At this time our team are monitoring both servers closely and we will be sending out an email to all affected customers which will explain the cause as well as our action plans to avoid outages of this magnitude. You can also find more information around this via our status page.
 
You must log in or register to reply here.
Back
Top
Sign up to the MyBroadband newsletter
X