Merge Tag '6.17-rc1-smb3-client-fixes' of Git://Git.Samba.org/Sfrench/Cifs-2.6

As a visitor to this website, you're likely wondering why you've been presented with a page that looks and feels like a standard article – but for one crucial difference: the unmistakable warning sign that says "You are seeing this because..." You see this because the administrator of this website has taken proactive steps to protect its server against the scourge of AI companies aggressively scraping websites. This may seem like a trivial concern, but trust us, it's a serious issue.

The Problem with Scraping

Scraping – the process of extracting data from websites without their explicit consent – is a growing problem. AI companies have developed sophisticated techniques to scrape websites with ease, often using automated scripts that can navigate even the most complex web pages. This not only infringes on website owners' rights but also puts an enormous strain on server resources, making them inaccessible to legitimate users.

Enter Anubis

To combat this issue, the administrator of this website has implemented a solution called Anubis. Anubis is a Proof-of-Work scheme that uses a similar mechanism to Hashcash, a proposed proof-of-work scheme for reducing email spam. In simple terms, Anubis adds an extra layer of challenge to websites, making it more expensive for AI companies to scrape them.

How It Works

Here's how it works: at individual scales, the additional load is negligible, but when mass scrapers come into play, the added cost becomes significant. This makes scraping much more expensive and less feasible. The ultimate goal of Anubis is not to completely eliminate website scraping but to make it a costly proposition that websites can no longer afford.

Requirements and Caveats

There are some caveats to consider when using Anubis:

* Anubis requires the use of modern JavaScript features, which plugins like JShelter will disable. Therefore, you must either disable JShelter or other similar plugins for this domain. * This website is running Anubis version 1.20.0, and to get past the challenge page, you need to enable JavaScript. * A no-JS solution is still a work-in-progress.

The Verdict

While Anubis may seem like a hacky solution, its real purpose goes beyond mere website protection. It's a temporary measure that allows website administrators to spend more time fingerprinting and identifying headless browsers (e.g., via how they do font rendering) so that the challenge page doesn't need to be presented to users who are much more likely to be legitimate.

By acknowledging the limitations of Anubis, we hope to have provided you with a better understanding of this complex issue and its potential solutions.