AI Agents Need Data Integrity

Think of the Web as a digital territory with its own social contract. In 2014, Tim Berners-Lee called for a “Magna Carta for the Web” to restore the balance of power between individuals and institutions. This mirrors the original charter’s purpose: ensuring that those who occupy a territory have a meaningful stake in its governance. Web 3.0—the distributed, decentralized Web of tomorrow—is finally poised to change the Internet’s dynamic by returning ownership to data creators.

This will change many things about what’s often described as the “CIA triad” of digital security: confidentiality, integrity, and availability. Of those three features, data integrity will become of paramount importance. When we have agency in digital spaces, we naturally maintain their integrity—protecting them from deterioration and shaping them with intention.

But in territories controlled by distant platforms, where we’re merely temporary visitors, that connection frays. A disconnect emerges between those who benefit from data and those who bear the consequences of compromised integrity. Like homeowners who care deeply about maintaining the property they own, users in the Web 3.0 paradigm will become stewards of their personal digital spaces.

This will be critical in a world where AI agents don’t just answer our questions but act on our behalf. These agents may execute financial transactions, coordinate complex workflows, and autonomously operate critical infrastructure, making decisions that ripple through entire industries. As digital agents become more autonomous and interconnected, the question is no longer whether we will trust AI but what that trust is built upon.

In the new age we’re entering, the foundation isn’t intelligence or efficiency—it’s integrity. In information systems, integrity is the guarantee that data will not be modified without authorization, and that all transformations are verifiable throughout the data’s life cycle.

While availability ensures that systems are running and confidentiality prevents unauthorized access, integrity focuses on whether information is accurate, unaltered, and consistent across systems and over time. It’s a new idea. The undo button, which prevents accidental data loss, is an integrity feature. So is the reboot process, which returns a computer to a known good state.

Checksums are an integrity feature; so are verifications of network transmission. Without integrity, security measures can backfire. Encrypting corrupted data just locks in errors. Systems that score high marks for availability but spread misinformation just become amplifiers of risk.

All IT systems require some form of data integrity, but the need for it is especially pronounced in two areas today. First: Internet of Things devices interact directly with the physical world, so corrupted input or output can result in real-world harm.

Second: AI systems are only as good as the integrity of the data they’re trained on, and the integrity of their decision-making processes. If that foundation is shaky, the results will be too.

Integrity manifests in four key areas. The first, input integrity, concerns the quality and authenticity of data entering a system. When this fails, consequences can be severe.

The second issue is processing integrity, which ensures that systems transform inputs into outputs correctly. In 2003, the U.S.-Canada blackout affected 55 million people when a control-room process failed to refresh properly, resulting in damages exceeding US $6 billion.

Safeguarding processing integrity means formally verifying algorithms, cryptographically protecting models, and monitoring systems for anomalous behavior.

Storage integrity covers the correctness of information as it’s stored and communicated. In 2023, the Federal Aviation Administration was forced to halt all U.S. departing flights because of a corrupted database file.

Addressing this risk requires cryptographic approaches that make any modification computationally infeasible without detection, distributed storage systems to prevent single points of failure, and rigorous backup procedures.

Finally, contextual integrity addresses the appropriate flow of information according to the norms of its larger context. It’s not enough for data to be accurate; it must also be used in ways that respect expectations and boundaries.

Preserve contextual integrity requires clear data-governance policies, principles that limit the use of data to its intended purposes, and mechanisms for enforcing information-flow constraints.

The Need for Integrity in Web 3.0

The digital landscape has shifted from Web 1.0 to Web 2.0 to Web 3.0. Each new generation introduces new challenges and opportunities for data integrity.

Web 1.0 was characterized by a centralized, hierarchical approach. Web 2.0 introduced social media and the web of trust, where users could control their own data and interactions. Web 3.0 promises a decentralized, blockchain-based internet, where data is encrypted and transparent.

This shift requires new approaches to data integrity, including cryptographic techniques, secure storage solutions, and transparent governance structures.

Ensuring Integrity in AI

AI agents are only as good as the data they receive. If that data is compromised, so too are their decisions.

Integrity controls can include encryption, secure storage, and access controls. They can also involve formal verification of algorithms and models, as well as monitoring for anomalies.

But integrity control measures can slow systems down, particularly in real-time applications.

Emerging technologies like quantum computing threaten current cryptographic protections.

The distributed nature of modern AI presents a large attack surface.

The challenges to data integrity in AI are social as well as technological. Companies may prioritize speed to market over robust integrity controls, while development teams may lack specialized knowledge for implementing these controls.

Real-World Examples

Ariane 5 Rocket (1996) Processing integrity failure: A 64-bit velocity calculation was converted to a 16-bit output, causing an error called overflow. The corrupted data triggered catastrophic course corrections that forced the US $370 million rocket to self-destruct.

Mars Climate Orbiter (1999) Processing integrity failure: Lockheed Martin’s software calculated thrust in pound-seconds, while NASA’s navigation software expected newton-seconds. The failure caused the $328 million spacecraft to burn up in the Mars atmosphere.

Tay Chatbot (2016) Processing integrity failure: Released on Twitter, Microsoft‘s AI chatbot was vulnerable to a “repeat after me” command, which meant it would echo any offensive content fed to it.

Boeing 737 MAX (2018) Input integrity failure: Faulty sensor data caused an automated flight-control system to repeatedly push the airplane’s nose down, leading to a fatal crash.

SolarWinds Supply-Chain Attack (2020) Storage integrity failure: Russian hackers compromised the process that SolarWinds used to package its software, injecting malicious code that was distributed to 18,000 customers, including nine federal agencies.

ChatGPT Data Leak (2023) Storage integrity failure: A bug in OpenAI’s ChatGPT mixed different users’ conversation histories. Users suddenly had other people’s chats appear in their interfaces with no way to prove the conversations weren’t theirs.

Midjourney Bias (2023) Contextual integrity failure: Users discovered that the AI image generator often produced biased images of people, such as showing white men as CEOs regardless of the prompt. The AI tool didn’t accurately reflect the context requested by the users.

Prompt Injection Attacks (2023–) Input integrity failure: Attackers embedded hidden prompts in emails, documents, and websites that hijacked AI assistants, causing them to treat malicious instructions as legitimate commands.

CrowdStrike Outage (2024) Processing integrity failure: A faulty software update from CrowdStrike caused 8.5 million Windows computers worldwide to crash—grounding flights, shutting down hospitals, and disrupting banks.

Voice-Clone Scams (2024) Input and processing integrity failure: Scammers used AI-powered voice-cloning tools to mimic the voices of victims’ family members, tricking people into sending money.