# The Battle Against AI Scrapers: Understanding Anubis

As we navigate the ever-evolving digital landscape, it's become increasingly crucial for websites to protect themselves against the growing threat of AI-powered scrapers. These automated bots are designed to rapidly scrape large amounts of data from websites, often without permission or respect for user consent. To combat this scourge, website administrators have turned to Anubis, a clever hack that uses Proof-of-Work schemes to make scraping more expensive and less appealing to mass scraper levels.

# What is Anubis?

Anubis is a innovative solution developed by website administrators to shield their servers from the relentless onslaught of AI scrapers. This system relies on a Proof-of-Work scheme, similar to Hashcash, which requires users to perform a computational task (like solving a mathematical puzzle) before accessing certain pages or resources on the site. At first glance, this might seem like an innocuous request – after all, who doesn't love a good puzzle? However, for mass scrapers that aim to scrape vast amounts of data, Anubis presents a significant hurdle.

## How Does Anubis Work?

Anubis is designed to exploit the fact that individual users are unlikely to engage with these puzzles unless prompted by a legitimate website. In other words, the additional computational load required by Anubis is negligible for most users, but it becomes prohibitively expensive for mass scrapers who rely on sheer processing power to scrape websites.

In essence, Anubis serves as a "good enough" placeholder solution that allows website administrators to devote more resources to fingerprinting and identifying headless browsers (i.e., AI-powered bots that mimic human behavior). By doing so, they can refine their security measures to create a more robust challenge that doesn't need to be presented to users who are likely legitimate.

## The Catch: Modern JavaScript Features

Anubis has an important caveat – it requires the use of modern JavaScript features that plugins like JShelter will disable. In light of this requirement, users must either disable JShelter or other similar plugins for this domain. Moreover, Anubis necessitates enabling JavaScript to navigate past the challenge page. This is a concession made by website administrators in response to the changing social contract around web hosting.

## The No-JS Solution: A Work-in-Progress

While disabling JShelter or modern JavaScript features might seem like an inconvenience, it's essential for protecting websites from AI scrapers. However, developers are still working on creating a no-JS solution that can efficiently mitigate this threat without compromising user experience. As we move forward in the digital landscape, it will be fascinating to see how Anubis and similar solutions continue to evolve.

In summary, Anubis is an ingenious hack designed to safeguard websites against AI scrapers. By leveraging Proof-of-Work schemes, it makes scraping more expensive and less appealing to mass scraper levels. While it comes with some caveats, including the requirement of modern JavaScript features, this solution holds promise in the ongoing battle against these digital threats.