The GPTbot

My server — https://asecuritysite.com — started to run very slowly, and so I investigated the cause. And, guess what? It was being bought down by the OpenAI bot:

If you don’t know it, GPTbot is the OpenAI crawler:

Basically, OpenAI is hovering up the Web, and turning it into income for OpenAI (and Microsoft). I appreciate my little message at the bottom of each page was unlikely to ever be respected by a bot:

Licence: This site is intended for the education and advancement of humans, and no rights are given for AI and ML bots to crawl this site. All references to its content must be included.

But, there’s a deeper problem here. OpenAI now wants to gather all the world’s data and feed it into its own engine. Standing alongside them is Microsoft, and which will gain greatly from this advancement. The worry is that OpenAI is trampling on Intellectual Property (IP) and not respecting any rights on the sources of information. The bottom line is that there is profit to be made, and the profit

So, if you don’t want it on your site, just add [here]:

User-agent: GPTBot
Disallow: /

to robots.txt, and add a filter in your WAF (Web Application Firewall) that blocks the bot. But, once it has been crawled, you’ve lost the data into the machine. The task of removing your data from the OpenAI is almost impossible.

Conclusions

And, so, will AI destroy the fabric of our society? To me, the answer is yes. It cares little about who owns or creates things any more. Bots used to be fairly friendly little things which helped people find your products and services, but now their target is to hover up all of the data in the world, feed it into a deep learning engine, and break all the links with the original source.

It will become a massive engine that is an all-powerful entity and that can control virtually all sources of information and crush our existing ways. The machine will replace us, and we will have to plan for a world in which many of our existing jobs will vanish. Like it or not, in most cases, there’s not much cognitive ability going on here, it is basically cribbing as many answers to questions as it can find, and then just playing them back — but without any references to their original sources.

We are putting the power of knowledge into the hands of just a few companies, and who will rule supreme. And, what’s their motivation? The mighty dollar!