Merge branch 'ps/misc-build-fixes': The Battle Against AI Scrapers

As you're reading this, you're likely wondering why you've been redirected to this page. The reason lies in the efforts of website administrators to protect their servers from the scourge of AI companies that have taken it upon themselves to aggressively scrape websites.

The administrator has set up a security system called Anubis, which uses a Proof-of-Work scheme similar to Hashcash, a proposed method for reducing email spam. The idea behind Anubis is that while the additional load may seem insignificant at an individual scale, it becomes prohibitively expensive when attempted by mass scrapers.

So, how does it work? In essence, Anubis requires users to complete a simple challenge, which involves solving a proof-of-work puzzle. This process uses modern JavaScript features that plugins like JShelter are known to disable. To access the page, you'll need to enable JavaScript.

This may seem like an inconvenience, but it's a necessary evil in today's digital landscape. AI companies have changed the social contract around how website hosting works, making it increasingly difficult for legitimate users to access websites without being forced to engage with these malicious actors.

Furthermore, Anubis is not a foolproof solution and requires ongoing improvement. The current implementation serves as a "good enough" placeholder until more robust solutions can be developed. One such solution involves fingerprinting and identifying headless browsers, which are increasingly popular among AI scrapers.

For now, users must take steps to accommodate Anubis' requirements, including disabling plugins like JShelter that may interfere with the challenge page. We apologize for any inconvenience this may cause and appreciate your patience as we work towards a more secure online environment.