PlanetRomeo & the bad guys


Some of you might remember back to February 2013 when Planetromeo was offline twice for several hours. At that time we were facing what are called DDoS attacks, and in this posting I’ll try to explain a bit about the technical background of such attacks and how we’re trying to defend against them.

DDoS means “Distributed Denial of Service” and the attack aim to block or slow down the access to an internet service (just like for regular users.

One approach to achieve this is to simply over-saturate the network connectivity of a website, squeezing out or slow down the transmission for regular users. Considering today’s standard bandwidth in data centers, this can’t be done from a single internet connection. Such attacks require many participating computer systems, which usually have been hacked or are mis-utilized in other ways by the attacker.

There is, however, a more effective way to attack a site due to some design flaws in the data transmission protocols of the Internet. These were created in the 1970s, at a time when nobody could ever imagine that the academically oriented Internet might later open to the whole wide world.

In regular operation, your computer sends a small initial data packet (called “syn packet”) to our servers which are then answered back to establish a connection. This is called “handshake”. Once established, the connection can be used to transmit data like web pages or images. After receiving the first syn-packet, the server already occupies some memory for the expected connection; we call this “half-open”. But what if the other communication partner doesn’t complete the handshake? Connections stay half-open for a while before the server notices and frees the memory. Given the fact that a server’s memory is not unlimited, the amount of half-open connection is neither.

By simply flooding the target website with millions of such syn-packets, but never complete the handshake, thus keep the connections half-open, an attacker might force the servers to use up all their memory. Since syn-packets are very small, it requires only a small internet connection for the attacker.

It is worth noting that DDoS attacks do not touch your personal data stored on our servers! They just try to prevent regular users from accessing the website. It’s the difference between breaking into a house and blocking the door. They can only try to block the path to our servers this way.

We’ve been hit by these syn-flood attacks in the past, but usually were able to cover them by the sheer size of our server farm. However, the February 2013 attacks were especially intense, strong enough to block access for many to our site. In the 14 day graph below you see the number of incoming (purple) and outgoing (orange) packets with the purple spikes indicating the flood attacks. A dip in the orange graph shows that the site was offline.

After we were faced with the initial attack (2), we ordered and installed new firewalling devices to strike back; as you see, attacks 4-6 didn’t shut down the site anymore, even despite the attacker trying to strike us with even more missiles than before. We’re getting pretty experienced at beating these attacks back, and hopefully we’ll be able to continue to do so.

Jan – System Owner