A new type of distributed denial of service (DDoS) attack that took Spamhaus off the web for some hours (and “almost broke the Internet”) has made its way to South Africa, causing major congestion on the ADSL connections of infected users.
Fialkov said that they first noticed something strange around 15 May 2013 when their “speed complaint” and “can’t browse” problem categories started trending even though they had plenty of capacity everywhere.
“Initially we suspected that it was isolated to a few users,” Fialkov said. “It was only when we saw overall DNS load go over 200Mbps that we realised it was some sort of DDoS attack.”
This load was created by only 327 customers on the Cybersmart network that were roped into the attack.
What was particularly frustrating, Fialkov said, was that the exploit does not start immediately, “so if you switch [Internet service providers], things run fine for a couple of hours until the bot finds the new nameserver and starts again.”
Fialkov explained that the attack generally uses “ANY” DNS lookups against nameservers because it is the request that generates the biggest response.
If an ADSL user generates enough of these “ANY” queries to DNS servers the response could actually saturate their connection, Fialkov added.
“So although the attack is aimed at bringing down a remote target like the root nameservers, or spamhaus.org, or some other well known target, the unfortunate side-effect is that it was affecting the speed of some of the ADSL users.”
Not necessarily a South African botnet or virus
“We have seen various levels of DNS DDoS attacks originating from infected customers on networks we are involved with,” Diedericks said. “The activity has certainly increased over the past few weeks,” he added.
According to Diedericks, attackers find a foothold largely due to open resolvers, poorly configured DSL modems, and buggy firmware.
Fialkov further explained that malware is actually the least common method they saw for perpetrating the DNS Amplification attacks that affected Cybersmartr users.
In essence there are three forms of this attack, Fialkov said:
- Modify the DNS packet and change the source address to a random IP on another network (such as Cybersmart’s);
- “Brute force” the username and passwords on routers and change a user’s DNS settings to point to the corrupt DNS servers;
- Malware (the least common).
Web Africa chief technology officer, Rupert Bryant, said that they haven’t seen such attacks originating from their own users, but that international IPs have tried to use their DNS servers for such DoS attacks.
How do you stop a DNS Amplification attack?
As a result of international IPs trying to perform DoS attacks by DNS amplification with their servers, Bryant said they have now locked down all their DNS servers to WebAfrica IP ranges.
“We use dynamic blackhole routing in our network to isolate the affected devices until fixed,” Diedericks said when asked how they mitigate the effects of the attack.
While Cybersmart had some mitigating measures in place, such as caching of DNS queries and throttling of certain types of traffic, Fialkov said they also restricted the DNS servers their customers could use and limited the number of “ANY” queries users were allowed per second.
“We restricted our ADSL customers to use only our caching nameservers, as well as nameservers that we deem to be trusted,” Fialkov said.
A “trusted nameserver” is any caching nameserver that could show it was rate-limiting “ANY” DNS requests.
“Rate limiting the ‘ANY’ query can be done pretty safely without breaking anything,” Fialkov said.
Modified DNS amplification attack “frightening indeed”
According to Fialkov, rate-limiting the “ANY” DNS query gave immediate interim relief for exploited users, even though they continued to generate the same amount of queries as they were no longer getting the response.
This means that their line did not saturate and their speed returned mostly to normal, Fialkov said.
“The reason I say ‘mostly to normal’ is that sometimes the number of DNS requests could increase the load on the ADSL router, which would continue to cause speed issues,” Fialkov added.
Fialkov expressed concern that a modified version of this DNS amplification attack would be much harder to mitigate.
“The reason for using the ‘ANY’ query is that it returns the biggest response, so it maximises the attack on the target,” Fialkov explained.
For all intents and purposes, “ANY” is a useless query as it is just summary of the all the other DNS queries, such as A, MX, SOA, AAAA, TXT, and CNAME.
“A program can get the same information just by doing the queries individually,” Fialkov said.
These queries have a much smaller response than “ANY”, Fialkov said, which means that attackers would need more machines to have the same impact on the target.
What is a concern is that the attack could be modified to use one of these smaller, more widely used queries, or perhaps even a random combination of queries.
This would be almost impossible to stop from the ISP side, Fialkov said, as rate-limiting “A” record queries will have an impact on browsing experience.
“A modified DNS amplification attack as described would be frightening indeed,” Fialkov said.
Attack from a South African source?
In explaining the forms that the DNS amplification can take, Fialkov said that Cybersmart throttles UDP packets on their overseas routers that originate with a source address that can only occur within its network, but not on its peering links.
This should mitigate DNS amplification attacks of the first form Fialkov described (where the source address is spoofed) that originate internationally.
“UDP packets are easily modified compared to TCP packets; so it is possible to send packets from outside our network, and to change the source to be from an IP in our network,” Fialkov said.
“This will result in the answers coming back to random customers on our network without the customer being exploited at all,” Fialkov explained.
Because of the fact that Cybersmart throttles UDP packets on its international routers, Fialkov said he believes that traffic from this type of attack would have originated within South Africa.
The target of the attack? The website of the Internet Systems Consortium: isc.org.
“We were not contributing at all to bringing down the target because the majority of the ‘ANY’ responses were cached,” Fialkov said.