Merge branch 'ps/refs-files-remove-empty-parent': The Battle Against AI Scraping
As you try to access this website, you may notice a brief delay or a "please wait" message. Don't be alarmed – it's all part of the effort to protect our servers from unwanted visitors.
The administrator of this website has set up Anubis, a clever tool designed to safeguard against AI companies aggressively scraping websites. This can lead to downtime, making resources inaccessible to everyone. But don't worry; Anubis is not an attempt to completely block access – it's more like a speed bump.
Anubis uses a Proof-of-Work scheme inspired by Hashcash, a proposed solution for reducing email spam. In theory, at individual scales the additional load is negligible, but when mass scrapers join forces, it becomes a significant burden. This makes it much more expensive to scrape websites, effectively deterring unwanted visitors.
But why does Anubis require modern JavaScript features? The answer lies in its primary goal: to help identify headless browsers (like those used by AI companies) and prevent them from accessing the challenge proof-of-work page. This is done by analyzing font rendering – a unique trait of legitimate users. By bypassing this step, Anubis ensures that only human visitors are granted access.
However, this comes with a caveat: you need to disable JavaScript plugins like JShelter for this domain. We apologize for any inconvenience this may cause and appreciate your understanding in this matter. Our website is currently running Anubis version 1.20.0 – the only way to overcome this challenge is by enabling JavaScript.
It's worth noting that a no-JS solution is still a work-in-progress, and we're working on it. But for now, please keep in mind that AI companies have redefined the social contract around website hosting, making traditional solutions like this one necessary. We appreciate your cooperation and understanding during this time.