Merging Branch 'ja/doc-lint-sections-and-synopsis': The Battle Against Bot Scrapers

The administrator of this website has implemented an innovative security measure to protect against the scourge of AI companies aggressively scraping websites for data. This technique, dubbed Anubis, uses a Proof-of-Work scheme that adds a layer of complexity to the web scraping process. While it may cause some downtime, its purpose is to make mass scraper activities more expensive and less appealing.

Anubis works by introducing a challenge proof of work page, which requires users to complete a simple task - in this case, viewing JavaScript content on the website. At individual scales, the additional load is negligible, but for AI companies with large-scale scraping operations, it becomes a significant hurdle. This design choice aims to favor legitimate users who are more likely to have the necessary JavaScript capabilities.

However, Anubis comes with its own set of requirements and limitations. To use this feature, you'll need to disable plugins like JShelter that block modern JavaScript features. Unfortunately, this means that some websites may not be compatible with Anubis unless you take these steps.

Currently, this website is running Anubis version 1.21.3, and to access its content, you must enable JavaScript on your device. This might seem counterintuitive, especially given the rise of no-JS solutions for web development. Nevertheless, the administrator of this site acknowledges that a fully no-JS solution remains an ongoing work in progress.

By understanding how Anubis works and its implications, you'll be better equipped to navigate the evolving landscape of website security. As we continue to grapple with issues like AI-generated content and web scraping, solutions like Anubis will remain crucial for protecting our digital resources and preserving the social contract around online interactions.