GitHub and the memcached DDoS amplification attack of 1.35 Tb/s

March 02, 2018
GitHub survived a DDoS attack of 1.35 TB/s. How? They paid someone else to figure it out, as they should.

If you leave your memcached servers listening on UDP and open to the Internet, they will certainly be used to amplify DDoS attacks. During the event described in the article, which happened on February 28th, GitHub had to move all their incoming traffic to Akamai so they would handle the attack – 1.35 Tb/s.

Don’t be an amplifier

Remember: unless you know what you’re doing, do not leave UDP services open to the internet. Most UDP services out there (including NTP, DNS and even memcached) can be used in amplification attacks.

Amplification attacks are simple. The attacker forges a packet pretending that it’s coming from the victim, and sends it to a UDP server. When the server responds, it ends up sending a potentially large response to the victim. The “amplification” happens because the attacker sends X bytes to the server but the server responds with Y (where Y > X, usually multiple times larger), meaning the attacker can use 1 byte of traffic to generate X bytes of untraceable packets.

For example, imagine the attacker asks “hey memcached, give me the object with this hash key”, where this is a field with 8 to 200 bytes. If the attacker requests an object that exists, memcached will respond with a much larger object of, potentially, several MBs.

Since people that put their memcached on the internet usually don’t care about authentication, the attacker can cheat. They can set up the attack by inserting arbitrarily large keys into the cache, and then ask for the object again – except they would forge the packet to use the victim’s address.

By repeating that process many times per second, the attacker makes a random memcached server send hundreds of MB/s of traffic to the victim. And that’s from one node. Now assume the attacker has a list of thousands of open memcached UDP servers…

Very nasty.

CDNs are a must

CDNs with DDoS absorbing capabilities are a must for all businesses connected to the internet.

Sadly, we really can’t make your own CDN, not one that’s useful for absorbing these kinds of attacks.

Given the internet’s architecture, Do-It-Yourself DDoS blockers are just not effective. There’s no way to stop that traffic from flowing into your network unless you ask other peers in the network to absorb your traffic.

But maybe that’s one way? Wouldn’t it be nice if users and companies could help each other absorb DDoS attacks, in exchange for getting similar protection? That’s not easily possible for most users (e.g: residential) because they can’t announce BGP from their networks.

The next idea would be to organize BGP-empowered companies to help each other absorb DDoS attacks.

Sounds good on paper. In practice, it’s better for companies to focus on their business, and pay for Akamai, CloudFlare, etc to worry about DDoS for them.

Thank you capitalism?