Q&A: Distil Networks’ Founder Unpacks the Bad Bot Landscape
With the onslaught of unrelenting cyber attacks continuing to make headlines around the world, we spoke with cybersecurity expert and ffVC portfolio company founder, Distil Networks’ Rami Essaid, to get a sense of the landscape of what Rami refers to as the “centerpiece” of a hacker’s toolkit — the “bad bot.”
First off, can you tell us what Distil Networks does and how your technology mitigates threats from bots?
Distil Networks provides some of the most accurate measures to protect web applications from bad bots, API abuse, and fraud. We automatically block 99.9% of malicious traffic without impacting legitimate users.
We offer two core products: Distil Web Security and Distil API Security. Distil Web Security defends websites against web scraping, competitive data mining, account takeovers, transaction fraud, unauthorized vulnerability scans, spam, man-in-the-middle attacks, digital ad fraud, and downtime.
Our Distil API Security protects public and partner-facing APIs against developer errors, integration bugs, automated scraping, and web and mobile hijacking.
What are the security risks related to bots that most people don’t immediately recognize?
In the past, bots were primarily used for web scraping and content theft, but the landscape has evolved in recent years. Now, people are starting to recognize that bots are the centerpiece of a hacker’s toolkit, and responsible for a wide range of sophisticated attacks like transaction fraud, account takeovers, and man-in-the-middle attacks.
One particularly interesting use case deals with website credentials. It seems we can’t go a week without hearing about massive password dumps — where hackers break into systems and steal user credential information. Although companies can alert users to change passwords quickly, once the information is out there, it’s too late. Password credentials for major websites floating around creates a huge bot traffic problem.
People reuse username and password combinations all the time. Following a password dump, other online services are hit with automated attacks, where hackers use bots to cycle through enormous lists and test passwords for various websites en masse. This is known as credential stuffing.
Your hallmark piece of research is the yearly Bad Bot Landscape Report. What are the top three takeaways from this year’s findings?
This year’s Bad Bot Landscape Report uncovered that, although overall numbers of bad bots are decreasing, there is a rise in the sophistication of bots. Our report found that 88% of all bad bot traffic had one or more characteristics of, as we have termed, an advanced persistence bot (APB). APBs have the ability to mimic human behavior, load JavaScript and external assets, tamper with cookies, perform browser automation, and even spoof IP addresses and user agents.
Our report also found that medium-sized websites (i.e. those with an Alexa ranking between 10,001 and 50,000) are at the greatest risk of bot attacks, as bad bot traffic made up 26% of all web traffic from this group.
And as for specific vertical markets, real estate websites saw a 300% increase in bad bot activity, but digital publishers were hit the hardest with bad bots, which made up over 31% of all traffic.
With all this said, the bad bot landscape continues to evolve rapidly and, with cheap or free cloud computing recourses readily available, almost anyone with basic computer skills can get into the bot game. IT infrastructure teams, security teams, and marketing teams are all under increased pressure to maintain speed and availability, prevent nefarious actors from harvesting data, and access accurate conversion metrics. The problem of bots ultimately persists and haunts almost every aspect of online businesses, but we are working tirelessly to mitigate such risks for current and future customers.
The legal issues around malicious bot activity has been making headlines lately, such as a LinkedIn lawsuit against anonymous data scrapers and the BOTS Act calling for a crackdown on online ticket scalpers. What are your thoughts on the legality of bots?
In all aspects of digital life, hackers have a habit of staying one step ahead of the good guys and the law. The bot activity in both of these situations is unsavory, but falls within a grey area that unscrupulous organizations are willing to exploit. Whether it’s illegal or not is still up for debate.
If the stolen data was made public by the business, how can the legal system intervene? Where is the line between protection and preferential treatment? What legal claims do companies have to data that belongs to its members? Modern society is still grappling with these questions with few, if any, answers forthcoming.
Another problem is that many businesses don’t have the time or inclination to pursue legal recourse. Even those who do will have the same uphill battle that many law enforcement agencies have when fighting cyber threat actors across state and national borders. Because online ticket scalping and data scraping are not bound by these borders, it can be problematic for laws to have teeth.
Instead of more laws, we need to demonstrate a call to action to solve this problem once and for all, with a multi-pronged effort that relies on bot-blocking technology and stronger human intervention.
Early in 2016 Distil acquired Sentor’s ScrapeSentry, which complements Distils’ technology with human analysis. Despite the massive shift to automation, can you talk about the importance of the human element in your work?
Prior to our acquisition, Distil did an excellent job of identifying and blocking bots. However, we found that customers were actually requesting this human element at all stages of engagement.
Our enterprise customers were looking for someone to walk them through their data and explain why certain bots were blocked, to better understand their traffic patterns and how Distil decisions are made. As we grew, we were unable to dedicate resources from our engineering team to work with our large customers on the analysis piece, and an acquisition made sense for our business.
It became clear to us that while machines can do the heavy lifting and provide initial explanations or patterns, there is nothing quite like the human touch to get down to the nitty gritty.
