DDoS for dummies — What to do when you’re in the middle of one
The D-Day
It is a lovely Friday afternoon. Recent deployment has been successful and the customer support teams have been unusually quiet. Wrapping up a little early sounds like a good idea. But hang on, you’ve started receiving alarms from your system monitors.
The Database instance CPU is close to 85% which normally hovers around the 10% level. Traffic is almost 3 times the volume normally seen at this hour. Many of the requests are originating from a given set of IPs at a rate that doesn’t look humanly possible.
So it dawns — you are in the middle of a DDoS attack!
A Distributed Denial of Service attack is a malicious attack on the servers of an organisation that involves overwhelming the infrastructure with a flood of requests in order to:
- Disrupt or degrade the ability of the organisation to cater to genuine requests
- Punish the organisation by inflicting high infrastructure costs incurred as a result of processing non genuine traffic
These attacks can broadly be classified into various categories — volume based attacks (UDP and ICMP floods), protocol based attacks (SYN flood, ping of death) and application layer attacks (HTTP flood, Slowloris). For the rest of the article, assumptions have been made around an application layer HTTP flood.
Botnets
The bad guys have made the first move. Trust me, you will want to punch them back on their nose. Alas, you cannot. The attackers rarely employ dedicated compute capacity to orchestrate such an attack and hence tracing them will be a challenge. Their modus operandi is usually to infect unsuspecting computers on the internet by malware. Once the malware is planted, these computers (also called zombies or bots) can be controlled remotely by the attacker. A network of such infected computers put together is called a Botnet. Each bot on this botnet is now at the attackers disposal to participate in this zombie party (read DDoS).
Foes or too many friends?
It’s vital to ascertain whether the surge in traffic is due to a genuine reason or results from a malicious intent. In many cases, there could be a valid increase in traffic when a new marketing campaign, sale or feature is launched (and the infrastructure team is, yet again, out of the loop). There could also be authentic users trying to access the systems from a partner VPN, and you’ll observe a lot of requests being received from a single IP or a set of IP’s. How, then, do we identify the bots from the real users. There are a few indicators:
- It’s important to validate the origin of the requests. If the need to access your system from a VPN or anonymised IP’s doesn’t sound right, it probably isn’t right.
- If the requests are fixated on a particular page such as a login page, and most of the login attempts are not successful (your logging tools would hold the answers)- it is a tell tale sign of an attack.
- The actions that these requests are performing align to a pattern which can possibly be scripted and run in a loop also alludes to an attack.
In case the traffic does not appear to be malicious and the increase in traffic is legitimate, it is very important to scale up or scale out the required servers and resources to ensure that genuine customers/clients do not suffer as a result of the spike in volume. This should also give some food for thought to the infrastructure and management teams to figure out a strategy to anticipate the volumes of traffic better in future and be ready for it.
All hands on board
If you’re sure it is a DDoS attack, there are a set of actions that are at your disposal assuming no protections have currently been put in place. Lets run through some tactical and strategic steps can be taken.
TACTICAL OPTIONS
- Infrastructure cost versus customer experience — It is a vital question whether to incur some extra cost by over provisioning infrastructure during a DDoS to prevent genuine users from being denied their service or to just focus on blocking the attackers. Keep in mind that blocking the attackers can take some time. If your infrastructure is not fully ‘swamped’ or you can afford to live with some ‘downtime’, you could skip provisioning additional resources during the DDoS. Your service needs to be up and running.
- Web Application Firewall — Most organisations usually have one, and if not, it still isn’t too late to get one and turn it on. Infrastructure providers today have the firewall as a cloud service. For instance, AWS offers the WAF service which protects your applications from common exploits. It also enables you to add both managed rules (such anonymised IP blocks, source IP reputation based blocks) as well as custom rules (rate based rules, explicit allow and block lists). With some analysis on the incoming traffic, the right combination of rules can be put in place to secure the systems from the attackers. These rules will evolve over time.
- DDoS protection as a managed service — Sometimes it does make sense to let the experts take the protection of the infrastructure up. Lack of know-how in the organisation around security, failure of the above steps to curtail the attack and the overall stakes involved in keeping the infrastructure up are all valid factors to steer your organisation towards this decision. Note that this may not be the ’quick fix’ that you hoped for. While being extremely effective, DDoS protection services usually do come with a significant cost and in some cases add some latency as well. The incoming traffic is routed to the infrastructure of the service provider who scrubs the malicious requests based on their algorithms and routes the clean traffic to your servers. AWS Shield Advanced, Akamai DDoS protection and Cloudflare DDoS protection are some popular managed services in this space.
STRATEGIC OPTIONS
- Application Design Considerations — Systems can be designed to show resilience to a great extent in the wake of a DDoS attack. A clear demarcation of public and private networks for your applications can reduce the surface area of attack — and protect your infrastructure. Implementing load balancing to protect the underlying compute resources from exposure from the wider internet is a must.
- Testing for Vulnerabilities — There is no substitute for a regular VAPT (Vulnerability Assessment and Penetration Testing) from third-party security vendors or using open-source tools to identify potential vulnerabilities or weaknesses in the application. The findings must be taken up on priority, evaluated for impact and remediated.
- Preparation for the next attack — A very good practice is documenting the sequence of steps taken during the attack and reviewing those until you arrive at a validated DDoS Standard Operating Procedure (SOP). A SOP will go a long way in speeding up your response in the event of another attack — leading to lesser downtime and reduction is the cost incurred as a result of the resources drained by the attack. SOP must be reviewed post each attack and evolved based on new learning and actions.
In conclusion
A DDoS attack is a nightmare both for the provider of service as well as the genuine recipient of the service. The nature of these attacks is also evolving with each passing day, and it is important that as a service provider one must think ahead and put in place the measures that protect the infrastructure from such attacks. In addition, its vital to educate not only the technology teams on whose shoulders the responsibility to deal with the attack lies, but also the wider organisation about their role in securing the resources.
There are multiple courses and programs available online in this field — the Cyber Security Expert Master Program and the Post Graduate Program in Cyber Security are a some of industry recognised programs offered by Simplilearn. Feel free to check these and some of our other course and content out at www.simplilearn.com.
