Hi all,
So apologies for the lack of responses in the thread since my last update. As previously mentioned there have been very sensitive legal and commercial discussions taking place on our end, which as anyone in business will know, is not something that can be openly discussed. I can however disclose the following now:
1) nearly all network performance issues have been fully resolved. Even Gauteng which had remnants of issues after the upgrades has had a stark performance increase over the last few days.
2) Http download traffic remains an issue that the upstream provider are busy resolving. It's a pretty complex problem they are having to deal with, but in our testing multi-threading the download if possible resolves most of the problems with http downloads.
3) other traffic is working very well in our testing: streaming, torrents, nntp and the like, although we are still monitoring this and sending requests for changes when we spot problems. Let us know what's not working optimally and we'll try to get them fixed up now.
4) the big announcement will be sent tomorrow to customers. We have done something pretty unprecedented in the history of ISPs to a) fix the performance issues; and b) mitigate the possibility of them ever returning. This is what has prevented us from commenting on this publicly - needless to say an email is being drafted to be sent to customers during the course of tomorrow to explain exactly what we've done to resolve any possible future performance issues on the network. I'm quite sure once the mail is read you will understand why this was kept so quiet and why it involved such a lengthy legal and commercial avenue to get right.
5) the issues with some customers not receiving updates from our support systems has been fixed.
6) the automated systems that involved customers moving to a blocked IP range has been fixed (read: won) by us.
In terms of the lengths we have gone to, to protect customers; to ensure performance issues don't crop up again; and to resolve actual issues will become abundantly clear in tomorrow's direct mail to all customers. It's been a rough patch the last month or so and there's no denying it. But unlike other ISPs we actually tried to contact every customer who may have been affected who tried to sign up to alert them to the fact, and even went as far as to ask customers to delay their activations with us - not only to preserve the network for existing customers but because it's the right thing to do. We've had to undergo some drastic, expensive, and serious changes to ensure that we mitigate as many risks of quality and performance issues cropping back up again. Naturally one cannot guarantee that no network issues will ever crop up again, but there are measures one can take to prevent them in future, and this is precisely what we've done. And it's a rather massive change that is now on its way, which ensures that we not only take direct control over the network, but that we employ redundancy even on the core network and backhaul network should a future issue ever crop up again. Details will become clear tomorrow when we will be able to disclose more information, and when we'll be able to explain what happened, and what's been done about it.
The bottom line is that the corner has been turned on network performance in a massive way; we've essentially built a completely redundant network around the existing infrastructure at a huge expense; and we will be ensuring that all customers don't have to walk into the new year with concerns about their ISP - as long as they're a Crystal Web customer. We do need your support to make this a reality, and while I know that it's been a rough ride the last month and a bit, we are incredibly grateful to everyone who has stuck with us through it. With the changes that are about to be made, it is about to become well worth it, so thank you.
I will be able to get back to the frequency of updates and openness you've come to expect of us from tomorrow onwards now, which is such a relief for all of us in management here. And I am now back in the office after all commercial and legal negotiations are done, which means I can help out on the forums again and we'll now be back to looking after you guys the way we have always committed to on the MyBroadband forums.
