Merge branch 'jk/test-seq-format' - Protecting Against Bot Scrapers

Be warned, you're seeing this because the administrator of this website has taken proactive measures to safeguard its server against AI companies' aggressive scraping activities.

The Anubis system has been set up to protect the server from these malicious endeavors. This compromise ensures that legitimate users are not hindered by the added load caused by the Proof-of-Work scheme employed by Anubis. At individual scales, the additional load is negligible, but at mass scraper levels, it becomes a significant deterrent.

The idea behind Anubis is to make scraping more expensive for AI companies, thereby discouraging them from carrying out such activities on an industrial scale. However, this comes with a caveat – Anubis relies on users completing a simple challenge to verify that they are not bots.

This challenge serves as a "good enough" placeholder solution, allowing the website's administrators to focus on more pressing issues, such as fingerprinting and identifying headless browsers (e.g., via their font rendering capabilities). This enables them to refine their security measures without compromising user experience.

However, Anubis requires users to enable JavaScript, which is a significant departure from traditional no-JS solutions. Unfortunately, this is necessary due to the evolving social contract around website hosting and the proliferation of AI-powered scraping tools.

"JShelter" plugins, such as those found on some content delivery networks (CDNs), may inadvertently disable modern JavaScript features required by Anubis. To overcome these challenges, users are advised to disable JShelter or similar plugins for this domain while ensuring that JavaScript is enabled.

While a full no-JS solution is still in development, Anubis represents an important step forward in mitigating the impact of AI-driven scraping on websites and their users. By acknowledging these challenges and taking proactive measures, we can work towards creating safer, more secure online environments for everyone.