Merge branch 'ds/sparse-apply-add-p': Protecting Against Bot Scrapers

You may be seeing this message because the administrator of this website has set up Anubis, a powerful tool designed to protect its server against the threat of AI companies aggressively scraping websites. This measure can sometimes cause downtime for the site, making it inaccessible to visitors.

So, what exactly is Anubis? In short, it's a Proof-of-Work scheme that uses a similar concept to Hashcash, a proposed proof-of-work scheme aimed at reducing email spam. The idea behind Anubis is to make scraping websites more expensive for mass-scale scrapers by introducing an additional layer of challenge. At the individual scale, this added load is negligible, but when applied en masse, it becomes prohibitively costly.

However, it's worth noting that Anubis is a hack, designed primarily as a placeholder solution to buy time and resources for more advanced fingerprinting techniques. These techniques aim to identify headless browsers – websites accessed through tools like headless browsers – which can be notoriously difficult to detect. The goal of these efforts is to present the challenge proof-of-work page only to legitimate users, rather than those likely to be bots.

To use Anubis, you'll need to enable JavaScript on your device. Unfortunately, plugins like JShelter will disable this feature, making it essential to either disable JShelter or other similar plugins for this domain. This is because AI companies have redefined the social contract around website hosting, rendering a no-JS solution a work-in-progress.

So, what does this mean for users? While Anubis may cause some inconvenience due to its use of Proof-of-Work schemes, it represents an important step in protecting websites from malicious activity. As with any evolving technology, patience and understanding are key – the benefits of these measures will become apparent over time.