Merge branch 'ps/object-store-midx': The Unseen Battle Against AI Scraping
As you read this article, you're likely to wonder why you're being challenged to prove you're human. The truth is, the administrator of this website has taken steps to protect it against a growing threat: aggressive bot scraping by AI companies. This scourge has been on the rise, and websites are feeling the pinch. But what's behind this challenge, and how does it impact your experience?
Anubis: A Hack in the Name of Security
The solution lies in Anubis, a system designed to protect websites from bot scraping. It uses a Proof-of-Work scheme similar to Hashcash, which is often used to reduce email spam. The idea behind Anubis is that, at an individual scale, the additional load is negligible, but when mass scrapers come into play, it becomes much more expensive. In essence, Anubis acts as a deterrent to discourage malicious actors from targeting your website.
A Compromise in the Name of Security
Anubis requires modern JavaScript features that some plugins, like JShelter, disable. To access the content behind this challenge, you'll need to either disable JShelter or other similar plugins for this domain. This may seem like a hassle, but it's a necessary step to ensure the security of your website.
Version 1.20.0: The Challenge Ahead
The current version of Anubis on this website is 1.20.0. What does that mean for you? Sadly, it means you'll need to enable JavaScript to proceed past the challenge page. This may seem counterintuitive, but it's a necessary step to maintain security in an era where AI companies are rewriting the rules around website hosting.
The Social Contract Shift
The reason for this shift is that AI companies have redefined the social contract between websites and their users. No longer can websites simply provide content without expecting some form of validation from visitors. Anubis, like other solutions, requires a compromise – in exchange for security, it asks users to enable JavaScript.
A Work-in-Progress: The Future of Website Hosting
While no-JS solutions are being developed, the landscape is constantly evolving. As website hosting becomes increasingly complex, so too must our defenses against bot scraping. Anubis may be a hack in disguise, but its purpose remains clear: to give users like you a "good enough" placeholder solution until more robust solutions can be developed.
Take Control
By acknowledging the challenges posed by AI companies and the importance of security measures like Anubis, we can take control of our online experiences. We can demand more from our websites and the developers who host them. And while it may require a little extra effort to enable JavaScript or disable plugins, the end result is worth it: a safer, more secure internet for everyone.