-
Join the MyBroadband community
South Africa’s biggest forum. Discuss, discover, and connect with thousands of members.
-
Question of the Day: How often do you receive spam calls?
Afrihost cloud down again, and again, and again, and again.
- Thread starter ezras
- Start date
-
- Tags
- afrihost cloud down
It took another phone call after finding that my server was 70% restored from backup to then find out that they couldn't restore from backup, and wanted to put the 'hard drive' from the virtual machine into a new machine to get it working. This does not make any sense to me, but once again I'm left with no other option and at this rate as of tomorrow, no more clients.
Furthermore, the excuse I was given as to why the restore failed was the kernel panic, which I'd believe if Afrihost were incompetent enough to not backup images outside of the VM - and even so, how does an internal restore get to 70% before failing thanks to kernel panic when prior to that I was told the system wouldn't even boot due to that reason?
We're coming up on 40 Hours downtime, and once again, I'm promised that it'll be back up in an hour. This strikes me as convenient, given that hometime for them is just half an hour away. I suspect my night is going to be much longer, and I can only hope it'll be because I'm fixing things on my restored server - somehow though, I don't hold out much hope for that.
Furthermore, the excuse I was given as to why the restore failed was the kernel panic, which I'd believe if Afrihost were incompetent enough to not backup images outside of the VM - and even so, how does an internal restore get to 70% before failing thanks to kernel panic when prior to that I was told the system wouldn't even boot due to that reason?
We're coming up on 40 Hours downtime, and once again, I'm promised that it'll be back up in an hour. This strikes me as convenient, given that hometime for them is just half an hour away. I suspect my night is going to be much longer, and I can only hope it'll be because I'm fixing things on my restored server - somehow though, I don't hold out much hope for that.
Chevron
Serial breaker of phones
A phone call back from them!
It seems that they are in fact provisioning an entirely new server, and (hopefully) making the old hard drive available to that new server. This means that I will in fact need to recreate and re set up EVERYTHING from the old server - including mail, databases, websites...
It's the worst of bad options, and it speaks tellingly of Afrihost's non-existent backup strategy, and most definitely of their lack of failover clustering, no matter what their site (http://www.afrihost.com/site/product/cloud_hosting?src=website_nav) might tell you. At this point, all I can do is be grateful that they have apparently gotten around to doing anything - although there's no guarantee on whether it's actually being done, let alone when it will be ready (other than the ever present 'within an hour' platitude).
It seems that they are in fact provisioning an entirely new server, and (hopefully) making the old hard drive available to that new server. This means that I will in fact need to recreate and re set up EVERYTHING from the old server - including mail, databases, websites...
It's the worst of bad options, and it speaks tellingly of Afrihost's non-existent backup strategy, and most definitely of their lack of failover clustering, no matter what their site (http://www.afrihost.com/site/product/cloud_hosting?src=website_nav) might tell you. At this point, all I can do is be grateful that they have apparently gotten around to doing anything - although there's no guarantee on whether it's actually being done, let alone when it will be ready (other than the ever present 'within an hour' platitude).
Hi all
Sorry I'm only attending to this thread now, not quite sure why it escaped me - we didn't pick it up on our searches.
Onto the issue at hand though,
I can only sincerely apologise for the amount of downtime that everyone has seen here. We definitely do acknowledge and agree that any downtime isn't great and this is no different. The issue seemed more complex than what we had initially anticipated. Our Server team have been doing an absolutely phenomenal job at getting the sessions back up and running. I promise you, we worked as hard, fast and smart as we possibly could.
I do have reports that the majority of the servers are back up and running at the time of writing this post, but I urge you - if you are still seeing prolonged downtime to please email [email protected] and PM me the ticket reference number. I'll escalate this directly over to our Server team.
Please include as much info in this ticket as possible.
Thanks for everyones understanding, I know it's been a trying and stressful time
Sorry I'm only attending to this thread now, not quite sure why it escaped me - we didn't pick it up on our searches.
Onto the issue at hand though,
I can only sincerely apologise for the amount of downtime that everyone has seen here. We definitely do acknowledge and agree that any downtime isn't great and this is no different. The issue seemed more complex than what we had initially anticipated. Our Server team have been doing an absolutely phenomenal job at getting the sessions back up and running. I promise you, we worked as hard, fast and smart as we possibly could.
I do have reports that the majority of the servers are back up and running at the time of writing this post, but I urge you - if you are still seeing prolonged downtime to please email [email protected] and PM me the ticket reference number. I'll escalate this directly over to our Server team.
Please include as much info in this ticket as possible.
Thanks for everyones understanding, I know it's been a trying and stressful time
Fulcrum29
Honorary Master
Azure can be quite a hurdle when PoPI comes knocking. It don’t seem that MS have any plans to come here yet… same goes in line with Google.
PoPI is definitely something I need to start looking at. Any easy resources, or should I be hitting the web rather than asking here?
Fulcrum29
Honorary Master
I was hoping that Michalsons would enlighten the MyBB community. We attend workshops to pick up on the guidelines. The commencement date is still to be announced on which companies will have 12 months to comply with PoPI. How entities will be audited... don't know. The positive side is that this is a good implementation and will move companies abroad faster to SA whom conduct business here so infrastructure deployments will happen at a more rapid pace. The problem however is a company which deploy now with a service which is not PoPI ready, when they must change again when the service will not be within compliance in 12 months when announced...
I've said it before, and I'll say it again - the front line staff (with just two exceptions) are terrible at phoning back, but they were friendly and did try to help as best they could.
My server was finally bought back online at 6pm (41 Hours of downtime). However, I am still copying data from the old, mounted backup drive to the new one, and setting all of my services back up - not helped by a complete loss of connectivity with the server about an hour ago. On calling through to support, the guy on the other end (whose name I didn't catch) 'helpfully' assisted by rebooting the system - a hard reboot too, it seems. There's nothing in my log file to indicate what might have caused the system to stop responding, and this is reminiscent of previous late night server monitoring incidents I've picked up where the server will disappear for a few minutes or an hour before reappearing. Luckily, I was able to remount the old drive and resume restore after the reboot.
In retrospect, I think my biggest mistake was overestimating Afrihost. It was, it seems, foolish to assume that just because they claim on their website that they are running top notch hardware with reliable infrastructure that they have actually set up any kind of real life backup system, let alone actual fail-over servers. While I understand that their staff worked hard on getting the system back up and running, this was entirely at the whim of the elusive sysadmins, and this belies a greater concern - a company so hell bent on marketing seems very unwilling to dig deep for actual server infrastructure, and this does not bode well for the future of the business.
Had I known that Afrihost did not have proper clustered/failover/backup solutions in place I wouldn't have used them, or would at least have set up my own failover DNS - it would have meant registering an extra server, but that's a cost I would have happily paid. Now it's a cost I will be paying - but it won't be with Afrihost.
Solitude
Executive Member
My one VPS is not working again this morning. The other two are up. I wonder when they will have everything sorted out.
Soon they can start advertising:
"We promise at least 70% up time!"
Here's what they currently say:
It feels like it's a monthly thing that they have problems. Every time we got to explain to clients that it's out of our hands and it's ridiculous.
Soon they can start advertising:
"We promise at least 70% up time!"
Here's what they currently say:
It feels like it's a monthly thing that they have problems. Every time we got to explain to clients that it's out of our hands and it's ridiculous.
Last edited:
Having gotten mail and everything except web up and running, I find that I need to copy over (properly) my passwd, shadow and other files from the old drive for the new server. It's a pretty straightforward operation but there is still a chance of failure, and so I phoned this morning to ask nicely if they could watch the server and replace the new files with the ones I've already backed up into a different directory - all they'd need to do, if necessary, is mount the drive, copy the files in one directory into another, and restart the server - but this they will not do - 'you need to email in a ticket and then we'll need to upgrade your hosting'. It means that sites will take a little longer to come up, but they won't be coming up on an Afrihost server.
If any of their senior management read my last post, they may now want to decide not to invest in new hardware to cover the promises they make on their site after all - their current hardware should soon be ample to cover the user base they have left.
If any of their senior management read my last post, they may now want to decide not to invest in new hardware to cover the promises they make on their site after all - their current hardware should soon be ample to cover the user base they have left.
I'm up and running and smiling, except that my main site dropped from google ranking 1 to 3. Did I mention that everyone returned to work this week? Did I mention that we had a record visits yesterday afternoon alone for the time the server was up?
Just image the amount of business we lost in the tree days. Our helpdesk handles about 15 to 20 calls a day, support tickets the same amount (training industry). We got 3 complaints on Monday, another 3 complaints on Tuesday and not ONE business call. 90% of our business comes from google, and not to even talk about the elearning complaints.
Afriman - I see your comments and sure the people worked hard to get it up, but why having all these issues? Its not the first time. It happens about every month! We got notice that 11 domains moving away form our server, owners given notice and said they will let us know when they want to move when they signed new agreements. First they want to make sure the new companies dont host with Afrihost to cause a repeat in problems.
joes3029 - You 100% right. If Afrihost is the best according to their website, then why do they have so many issues.
O, and I was told its a "dos attack" that affected the entire data centre. They had to wait for it to stop before they could get the servers up again. Why again all the different excuses?
Shocking.
Just image the amount of business we lost in the tree days. Our helpdesk handles about 15 to 20 calls a day, support tickets the same amount (training industry). We got 3 complaints on Monday, another 3 complaints on Tuesday and not ONE business call. 90% of our business comes from google, and not to even talk about the elearning complaints.
Afriman - I see your comments and sure the people worked hard to get it up, but why having all these issues? Its not the first time. It happens about every month! We got notice that 11 domains moving away form our server, owners given notice and said they will let us know when they want to move when they signed new agreements. First they want to make sure the new companies dont host with Afrihost to cause a repeat in problems.
joes3029 - You 100% right. If Afrihost is the best according to their website, then why do they have so many issues.
O, and I was told its a "dos attack" that affected the entire data centre. They had to wait for it to stop before they could get the servers up again. Why again all the different excuses?
Shocking.
Good point 70%
If you are having downtime regularly, and you haven't moved yet... I start loosing sympathy for you. How are you expecting it to change? If they aren't changing their infrastructure, and all they are doing is bringing their servers up again... how can you NOT expect it find yourself in the same situation again soon?
Ezras, your misfortune is so much worse than mine. It is our busiest time of year - I reckon our losses are in the tens of k. And I'm STILL down: "Test successful. PHP Successful too! 
" is all the link to my site delivers. But life will go on. Joes3029 is a saint - not only working ridiculous hours but posting to keep us all in the picture. I have only been running sites for ten years and this is FAR the worst outage I've ever seen. But something good will come of it - it always does. I find I only learn anything when the thing stops working. As for Afriman - well, I wouldn't be in your shoes right now. But maybe the directors will learn something. I also have an eLearning offering - fortunately still hosted by Webafrica. My motto is "keep learning."
" is all the link to my site delivers. But life will go on. Joes3029 is a saint - not only working ridiculous hours but posting to keep us all in the picture. I have only been running sites for ten years and this is FAR the worst outage I've ever seen. But something good will come of it - it always does. I find I only learn anything when the thing stops working. As for Afriman - well, I wouldn't be in your shoes right now. But maybe the directors will learn something. I also have an eLearning offering - fortunately still hosted by Webafrica. My motto is "keep learning."
HelterSkelter
Expert Member
LOL, maybe if it is 30% up, it passes.