Last Updated on: 22nd November 2023, 01:00 pm
A recent in-depth investigation carried out by ContentDetector.AI has shed light on the usage of OpenAI’s GPTBot on the top 1000 websites in the United Kingdom. The study unveils that a mere 5.7% of these prominent sites have implemented blocks against GPTBot, marking a notable 34% decrease from the global average of 8.6%.
The research, conducted by ContentDetector.AI, delves into the interaction between GPTBot and over 1000 leading UK websites. These websites were chosen based on their prominence and search traffic, as determined by Semrush. The team meticulously scrutinised the websites’ robots.txt files to ascertain whether GPTBot was actively blocked from crawling their content. The outcomes of this study yield valuable insights into the prevalence of GPTBot restrictions across the UK’s digital landscape.
Key Findings:
- Top 100 UK websites: Within this subset, the study uncovers that approximately 11% of the sites have taken measures to block GPTBot from accessing and crawling their web content.
- Top 1000 UK websites: A total of 5.7% of the most influential 1000 UK websites have chosen to block GPTBot.
Comparative Analysis: The study further juxtaposes the practices of the top 100 UK websites with those of the top 1000 UK websites, revealing a noteworthy contrast. It is revealed that the top 100 UK websites exhibit a 1.93-fold greater inclination to block GPTBot in comparison to their top 1000 counterparts.
In a global context, the number of websites worldwide that have enforced blocks against GPTBot surpasses those in the UK. The figure stands at 1.51 times more global websites employing such restrictions when compared to the top 1000 UK websites, as highlighted in another study conducted by ContentDetector.AI. This global perspective highlights the varying levels of acceptance and utilisation of GPTBot in different regions.
Prominent Websites’ Responses: Notably, even well-established websites such as BBC, Amazon UK, and NewsNow have opted to restrict GPTBot’s access within a week’s time, demonstrating the swift nature of their response to this AI-powered bot.
For a comprehensive breakdown and detailed analysis of this study’s results, View the complete analysis of the study!
ContentDetector.AI’s investigation underscores the intricate relationship between GPTBot and the top UK websites, providing valuable insights into the extent of its acceptance and utilisation across the digital landscape.
