Official Crystal Web ADSL performance feedback thread Part 2...

[aktor]

Crystalweb.

At what point do you intervene on a matter between you and telkom? Your account has packetloss and ping spikes on my exchange. You blame the exchange that is fine as far as I know you are probably correct.

However afrihost and webafrica capped accounts function normally on my line including a telkom uncapped account. However ont he second adsl line it seems to have the same ping spikes and packet loss?

I got exactly the same problem. I followed up with Telkom though, they said they're going to plug my line into another card on the DSLAM and see if that clears it up. But also, only happens with the CW account, not on AH or MWeb - so the network engineers at Telkom are also baffled as to why it's happening. They're going to try do my port change tomorrow, hopefully that helps. I'm at my wits' end.
(Wellington exchange)

 

[xrapidx]

I will refrain from posting, till tomorrow.
My current mood might just influence my judgement and what I will say.

I'm pretty much feeling the same, I signed up for what I thought was the end of ISP troubles based on the initial feedback... No more troubleshooting, something straight forward, you know what you're getting, and it just works

At least Crystal Web try and keep us in the loop, although at this rate - I'm not sure I'm prepared to pay double what I was and sit with similiar issues with no set end in sight... Thought paying double would not have these issues

I'm actually sitting at 28kb on a 10mb account right now, nntp. going to bed.

 

[mugen]

Please don't think that I'm trying to be difficult or derogatory, I honestly just want to understand what going on. But..

Except for the vast majority this is not the case - they don't find it weird, and don't think it's too sweet.

But you aren't dealing with a majority, you're dealing with an individual. A individual who gets bad results from an ISP on the same line where he gets good results from another.

According to that individual, perhaps. Others reckon Burger King is horrid and prefer their McDonalds.

The taste of burger is subjective, good versus bad quality internet is not. But that's not the point I failed at making, the mutual use of an ingredient(Heinz/Telkom) is what I wanted to point out.

Not possible. To make this truly analogous, adding anything to the sauce is not physically possible.

But you can add to the burger. What I'm getting at here is maybe other ISPs have found workarounds for the various possible bad conditions.

Or as PimpMyISP would say, "I heard you like networks, so I added networking to your networking to make your network network."

 

[aktor]

But you can add to the burger. What I'm getting at here is maybe other ISPs have found workarounds for the various possible bad conditions.

One thing I always said is that ISP's mostly have more direct access to Telkom's network engineers than the public does. So while the ISP absolutely does not own the infrastructure in the exchange, nor have direct control over it, they do however have a better way of communicating to Telkom what to look for and what to investigate. The public often gets stuck with call centre agents who either don't understand or don't want to.

This could be something as simple as the ISP asking their technical contact at Telkom something like "btw, client X on line Y is having some weird issues at the exchange, mind checking A, B and C for me on that DSLAM?"

I know the ISP doesn't have daily conversations with a contact at Telkom, but they surely have a better audience to technical queries than we do. While we can't demand the ISP to help us out with exchange issues, as the equipment is not their responsibility - it does help to know that one of the big guys out there is looking out for us and making sure our issues are investigated by Telkom.

 

[Crystal Web]

Please don't think that I'm trying to be difficult or derogatory, I honestly just want to understand what going on. But..



But you aren't dealing with a majority, you're dealing with an individual. A individual who gets bad results from an ISP on the same line where he gets good results from another.



The taste of burger is subjective, good versus bad quality internet is not. But that's not the point I failed at making, the mutual use of an ingredient(Heinz/Telkom) is what I wanted to point out.



But you can add to the burger. What I'm getting at here is maybe other ISPs have found workarounds for the various possible bad conditions.

Or as PimpMyISP would say, "I heard you like networks, so I added networking to your networking to make your network network."

I'm open to hearing how anyone can compensate for dropped packets off their network. I'm also not trying to be difficult here either, but the packets drop before they reach us - you're suggesting other ISPs have magic packet creation technology nobody has ever seen and keep it to themselves here in SA, and we don't.

And as previously mentioned, this issue is not unique to us whatsoever. You see it in every single ISP thread.

We however are open and engaging on here about all topics, and try to be detailed in our explanations to help the community understand things better. So the focus of the conversation tends to be around us - it's not unique though, and works the other way too.

 

[Crystal Web]

One thing I always said is that ISP's mostly have more direct access to Telkom's network engineers than the public does. So while the ISP absolutely does not own the infrastructure in the exchange, nor have direct control over it, they do however have a better way of communicating to Telkom what to look for and what to investigate. The public often gets stuck with call centre agents who either don't understand or don't want to.

Unless your line is with the ISP, the responsibility for line related issues lies between the end user and Telkom. We cannot report anything on your behalf to Telkom Wholesale without the line having been migrated to us. Not a thing. After we get through the call centre (yes) we'd be told to please call back once the circuit transfer is complete so that we can report on behalf of the customer. There is no 'general fault for one customer at an exchange where others don't have the same problem and the line is not with the ISP' department in Telkom Wholesale.

So yes, we are in a better position to explain the problems. But no, we're not in the best position to report it on your behalf in 90% of cases. Why? Not for a lack of will, that's for sure. I too don't want to be on the forums having to defend my business because of Telkom's infrastructure problems. I too don't want to have to employ extra staff because of Telkom's infrastructure problems. I too don't want to have to bear the brunt of angry customers because of faulty Telkom network infrastructure. But we do, because we have to. Because Telkom in numerous areas and in many cases has faulty network infrastructure, and people's first instinct is to blame the ISP and get angry at the ISP because the thought of having to deal with Telkom makes their blood boil.

 

[mugen]

Ok, but what does it mean when pathping shows packet loss on Crystalweb then no packet loss when using another ISP?

To my understanding networking is a form of black magic. Adding more black magic should make blacker magic.
...or kill us all.

 

[mugen]

I just want to say that I have high hopes for CrystalWeb. That's why I'm posting in this forum instead of looking up cancellation procedure.

 

[Crystal Web]

Ok, but what does it mean when pathping shows packet loss on Crystalweb then no packet loss when using another ISP?

To my understanding networking is a form of black magic. Adding more black magic should make blacker magic.
...or kill us all.

The below post will help answer your first question. The second part is that it's not black magic at all - it's science and maths and engineering and human effort coming together to create a beautiful, interconnected world where anything is possible. On condition that the information you are sending reaches its destination. We are a destination. Your router is a destination. Everything on the path to the final server you are talking to (and back) is a destination in this scenario. If the information you are sending cannot reach any of those points, it expires, by the very nature of the technology - it's built into the packet header. It's an 8-bit field in the packet header that determines how long that packet will survive before being dropped - the technology controls that - not your ISP. It is needed, otherwise you'd operate a completely congested network at all times. This is where it gets tricky - if your wireless signal has interference, you may not really notice it, but the information being sent wirelessly will, and they may expire on the second hop and appear to be exchange congestion. Another spanner for the works here, which is why we always ask customers to test with an ethernet cable. Is that fault-proof? Certainly not. If the cable has a problem, or the drivers are configured incorrectly on the device sending and receiving information, or if the network card is faulty, you'll also have troubles. And when the packets drop right from the get-go, you have a big problem on your hands as it is exacerbated the further the information has to travel (i.e. the more hops it has to go through to its final destination). 1% packet loss for example on the router can result in 1000ms pings on the second hop. Customer A looks at that and says "but 1% is not bad, you're lying". Those in the know say "oh dear, I'd better fix or replace my modem or router".

So while it's not black magic, it is complex enough to appear as such to those who rightfully just want to access the internet and not have to worry about these things. But you're talking about a technology with thousands of "moving parts" and each one of those is crucial to proper service delivery. And there are numerous providers involved along every step of the way. We, for example, do not have any equipment in Alaska. Nor does any local ISP that I know of. But you can talk to websites hosted there. How? Because of peering and inter-operable relationships among global players who make it possible. Your local ISP is a cog in a finely tuned, and very big wheel.

To understand the exchange issue a little better, here is how it works in simplistic terms:

Your line______________Exchange____________IPC_____________ISP Network____________Internet

The last three go hand in hand but all are needed to gain access to the net.

An ISP must have a registered IP range issued by AfriNIC. This range is then populated across the Telkom exchanges around South Africa so that Telkom know how and where to route the data for which ISPs. It moves over their network from there, over to the closest point an ISP has purchased IPC capacity at. It then moves over on to the ISP's network. So really what exists at the exchanges is all of Telkom's routing infrastructure - there's no ISP routing infrastructure there.

I understand the frustration when exchange issues crop up - I really do. About 50% of our support queries are actually us diagnosing a Telkom fault on their behalf. We then bear the brunt of customers getting very angry with us when we state as such. It's exacerbated when a customer switches ISPs and the packet loss disappears. But this doesn't mean the ISP has done anything there - the ISP simply cannot. We don't have facilities leasing to install infrastructure inside Telkom's exchanges - ISPs install their hardware much further down the route at a peering point for IPC. Also remember that an exchange isn't the little box up the road from you - it's a rather large building housing massive amounts of equipment. Each piece of equipment does something different, and each piece of equipment must be fully operational for the network to run smoothly across all ISPs. If one piece of hardware fails or is not working 100%, it can result in packet loss on one ISP's account and not the other, perhaps. Again, we can't test this, nor do we get any data about this from Telkom, so this remains an educated guess.

Here is a pic of a Telkom exchange, for some context:

Now imagine a single loose cable there. Or a single faulty port. Or router. Or configuration. etc. And you start to imagine the complexities of the situation. I hope this helps to explain why we can't fix, diagnose, nor answer exchange congestion for you, irrespective of the results showing no packet loss on one ISP or another.

BUT, you MUST report it to Telkom still. You absolutely have to. if you don't, the problems will 1) persist, and 2) may get worse and end up spreading to all of your ISP accounts. ISPs (including Telkom) rely on customers reporting faults. We can do a lot of stuff behind the scenes to monitor, and Telkom have bettered their reporting and management of this over the years, but they still rely on your calls and fault reports to them, in order to fix the problems. If they don't know an issue exists, how are they to fix it? You cannot assume that they are monitoring your specific connection at all times.

 

[Anthro]

SO, today I won a 10GB capped account from Afrihost.
I also have an uncapped BUSINESS account from them (but I paused it in favour of Crystal Web)

Compared streaming performance, between a CAPPED AH and Uncapped CW - and you guys beat their shoddy account 1000 times over.

Now, just to migrate my line across as well.
Looking forward to a fruitful relationship with the CW team. - Shot guys !

 

[Crystal Web]

SO, today I won a 10GB capped account from Afrihost.
I also have an uncapped BUSINESS account from them (but I paused it in favour of Crystal Web)

Compared streaming performance, between a CAPPED AH and Uncapped CW - and you guys beat their shoddy account 1000 times over.

Now, just to migrate my line across as well.
Looking forward to a fruitful relationship with the CW team. - Shot guys !

Ah, I needed this today. Thank you. It's been a rough day and great feedback is always good to hear. We also look forward to a great relationship in the future, with you, and all of our customers.

We did (I believe) show today that we are agile when an issue is identified, and that we're able to resolve it. Customers who experienced the speed-bump this afternoon will be able to realise the difference between this afternoon and tonight in overall performance. Even though downloads aren't optimal at the moment, the network is still working for them and not nearly what it was this afternoon. In addition, previously when we had an issue we had to manually identify everyone, manually create new logins, test everything, send it out etc. We've learned, and we've grown, and we've invested in the technology. Now you simply had to reboot your modem, and we "fixed" it in under 3 hours, and provided updates along the way, as well as proper ETAs which we stuck to. We even proactively stated there was an issue thereafter before customers had to, because that's how we roll. It may appear as if we have issues and are constantly fixing stuff, but it's actually just us being open and honest and discussing them with you on the forums. Other ISPs have exactly the same problems - some stay quiet for months on end waiting for their unprofitable, large downloaders to leave first and hope it helps network performance. Others remove themselves from the forum entirely and never deal with the accusations leveled against them. Others only post when things are good. We're mybroadband members through and through, and respect this community and our customers enough to own up to mistakes, and work all hours of the day and night to fix them if we have to, and spend all hours engaging with the community about it too.

 

[CataclysmZA]

And as previously mentioned, this issue is not unique to us whatsoever. You see it in every single ISP thread.

I have to agree here. Even with a solid line, I see some very minor packet loss on Afrihost from time to time, as well as other ISPs. Its usually no more than 2% at worst, but it is there and it isn't something they can safeguard against.

Ok, but what does it mean when pathping shows packet loss on Crystalweb then no packet loss when using another ISP?

Consider that some networks have had more time to sort out routing issues than others. Packets don't go straight from the Telkom exchange to an ISP, there's still a long route it must travel through different hardware setups before the ISP takes control of the packet in its network. If the TTL (Time-to-Live) is less than the amount of time the packet spends wandering aimlessly through the network, it gets dropped.

The problem with properly managing a network with manually configured static routes like Telkom's is that it is a lot of admin, a lot of documentation to sort through and a lot of paperwork - all ripe points of failure for human error to creep in. And that's before you have to send people to sort out packet loss on a network - they have to have a very good idea of where faulty hardware or cables may be before they fix it. There's probably at least three thousand errors on their network every day and keeping track of how each one affects their clients is a massive procedure that they don't have enough manpower or computing power for.

If packet loss shows for one ISP but not another, the ISP that does have it probably doesn't know there's an issue with your particular situation and can't be expected to know where it actually is, while the other ISP probably sorted it out a long time ago for someone else who had a similar issue.

 

[aktor]

Well, my CW story so far:

I've been with your big red cousin for almost 2 years. On the 1st of Nov I switched over to CW, hearing many good things about it. The fact that CW consists of like-minded people (streamers, gamers, etc) had me convinced it was apt to be a good move. After going through this thread, I signed up for a 10mbps uncapped package. A bigger package than I had with my previous ISP, and at a higher price. But I went for it. And here's why.

I'll be honest with you - I've had connectivity, speed and stability issues on CW for the last few days. Most of these could be caused by the strange packet loss issue on my exchange (that I only experience when connecting via CW, not when I connect via AH or MW). But through it all, a clear line of communication remained open with CW support. I had countless conversations with live support, gave feedback after conversations with Telkom, etc. And every time the support staff did everything they could to help, and even offered opinions on how exactly to word requests to Telkom's engineers so that they can understand what to investigate on their side.

(I should add here that while I was beyond livid with Telkom, I at least didn't take it out on the CW support guys )

While all of this was going on, CW was also communicating on the forums. Letting people know when there are problems, what they are, what the resolution is, and when it should be resolved. You literally don't see that transparency from any other ISP. And after today, while not sure exactly what went wrong (I wasn't affected, or at least I don't think so - I only came home late), I could still see the massive amount of work and effort going into it from CW's side - and still they kept on communicating with their client base throughout.

While I'm not a fanboy by any means (I'm still somewhat put out by the HTTP issues, but I do understand that a fix is coming soon, and at least I know that they know it's broken and needs fixing). I'm also having a pretty crappy experience with the exchange issues, and while this is not CW's fault, I still resent dealing with Telkom when it comes to advanced network engineering - because you simply don't get to talk to the right people. Again though, I understand that there isn't an official channel for CW to use to communicate with Telkom regarding clients' exchange problems - but how I wish there was...

So yes. I'm very impressed by CW's service, but I do hope the issues (HTTP, exchange, etc) get resolved pretty soon.

 

[Crystal Web]

I have to agree here. Even with a solid line, I see some very minor packet loss on Afrihost from time to time, as well as other ISPs. Its usually no more than 2% at worst, but it is there and it isn't something they can safeguard against.



Consider that some networks have had more time to sort out routing issues than others. Packets don't go straight from the Telkom exchange to an ISP, there's still a long route it must travel through different hardware setups before the ISP takes control of the packet in its network. If the TTL (Time-to-Live) is less than the amount of time the packet spends wandering aimlessly through the network, it gets dropped.

The problem with properly managing a network with manually configured static routes like Telkom's is that it is a lot of admin, a lot of documentation to sort through and a lot of paperwork - all ripe points of failure for human error to creep in. And that's before you have to send people to sort out packet loss on a network - they have to have a very good idea of where faulty hardware or cables may be before they fix it. There's probably at least three thousand errors on their network every day and keeping track of how each one affects their clients is a massive procedure that they don't have enough manpower or computing power for.

If packet loss shows for one ISP but not another, the ISP that does have it probably doesn't know there's an issue with your particular situation and can't be expected to know where it actually is, while the other ISP probably sorted it out a long time ago for someone else who had a similar issue.

Thanks for this. I believe mugen was specifically referring to the exchanges here, in which case it's not even remotely possible to do anything about it, as you point out.

You'd be surprised the tools we have at our disposal to manage accounts. Including live quality and fault management access on the network. We know what's going through where, when, how, and why, and the quality of each account at any given point in time, with automated alerts for problems - you are right in that this is ever evolving and changing though, in terms of what you're monitoring, but the tools and access is there, and constantly being used to look for possible problems even before they happen.

 

[Phinix]

@Crystal Web, you have a PM.

 

[Crystal Web]

@Crystal Web, you have a PM.

And you already have a reply.

 

[mugen]

Wes is spot on with what I am thinking. I've dealt with Telkom enough to consider them a obstacle that needs to worked around.

An example of this is your latest fixes. I've had packet loss since the first day I signed up, but I just now ran a pathping and it reported no packet loss.

Unfortunately latency to my favorite game is still crap and I need switch to my Afrihost account to play.

 

[aktor]

Btw, whoever else reported packet loss at the exchange when connecting via CW - I'm experiencing the same. But I'm on Telkom's case about it so hard that they're probably regretting the invention of callerID right now.
Will give feedback when I have it.
CW, i PM'ed you the latest conversation.

 

[Crystal Web]

Btw, whoever else reported packet loss at the exchange when connecting via CW - I'm experiencing the same. But I'm on Telkom's case about it so hard that they're probably regretting the invention of callerID right now.
Will give feedback when I have it.
CW, i PM'ed you the latest conversation.

I have responded.

 

[xrapidx]

Sitting at full speed ATM.