Botnets are Go …

If you run a Web server, you will see that botnets are a particular problem. Within the Web logs, you will often see continual scanning from botnets, and where it can be expensive to employ anti-bot technology. They are can also, sometimes, lead to false positives, such as where multiple valid requests from a number of sites at the same time can look like botnet activity. These valid requests could then be blocked for their IP address (or at least, blocked for a hold-down period).

Along with this, there are good bots, such as the Google bot, so you often want to be selective in the bots you allow. But, overall, they typically consume a good deal of bandwidth and CPU time, and are either there to “steal” data, or harvest credentials (and that could be used at some time in the future).

A popular one on my site is the PHP and WordPress bots:

2022-06-19 01:17:39 10.0.0.106 GET /wp-includes/ID3/license.txt - 443 - 172.71.102.42 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/95.0.4638.69+Safari/537.36 - 404 0 0 89
2022-06-19 01:17:39 10.0.0.106 GET /feed/ - 443 - 172.71.102.42 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/95.0.4638.69+Safari/537.36 - 404 0 0 109
2022-06-19 01:17:39 10.0.0.106 GET /xmlrpc.php rsd 443 - 172.71.102.42…