Merge branch 'tc/diff-tree-max-depth' - The Battle Against AI Scraping

As you read this, you may be wondering why this particular website has suddenly become inaccessible to you. This is not an error on our part, but rather a deliberate security measure put in place by the administrator of this site.

The culprit behind this blockade is a system called Anubis, designed to protect our server from AI companies that have been aggressively scraping websites for malicious purposes. While it may seem counterintuitive, Anubis serves as a necessary compromise between security and accessibility.

Anubis uses a Proof-of-Work scheme similar to Hashcash, a proposed solution to reduce email spam. On an individual scale, the additional load is negligible; however, when mass scraper bots come into play, it becomes prohibitively expensive for them to continue their nefarious activities.

The true intention behind Anubis lies not in providing a foolproof solution but rather as a temporary placeholder measure to buy more time. This allows researchers to focus on identifying and fingerprinting headless browsers – those sophisticated AI-powered tools that are notorious for disguising themselves as legitimate browsers – making it essential to present users with a challenge-proof-of-work page without overwhelming them.

However, Anubis comes with its own set of requirements. It necessitates the use of modern JavaScript features, which plugins like JShelter can disable. As such, we request that you either disable JShelter or other similar plugins for this domain to ensure a smooth experience.

This website is currently running on Anubis version 1.21.3. Despite its inconvenience, enabling JavaScript is still necessary, as AI companies have redefined the social contract around website hosting. A no-JS solution remains an ongoing work in progress.