Part 4: The Machine Cannot Govern Data Nobody Understands

“Know your data” sounds simple until somebody asks where it is

Let’s start this part with a joke. Shall we?

I felt a bit like Sheldon Cooper in The Big Bang Theory trying to use a literal flowchart algorithm just to make a new friend.

I ran the draft of this article through Substack’s new Pangram scanner — mostly out of plain curiosity after they announced their AI transparency feature on July 21st. The tool took a look at my draft, thought for about 4 seconds, declared it AI-generated, and left me staring at the screen. Here I am, four articles deep into a long series explaining why artificial intelligence behaves the way it does, only for an automated scanner to decide I’m just a chatbot pretending to have a state government IT career.

Pangram itself is and AI agent. It flags text primarily based on two things - predictable word choices, uniform sentence lengths and paragraph structures. It uses statistical probability without understanding. Isn’t this ironic?

Full disclosure - the only thing I ask AI bot to check is grammar and spelling. English is not my first language and often times I miss verbs, mix-up pronouns, and drop articles before roles.

It’s funny, sure. But it just confirms the main point of this whole article better than any diagram could. The scanner saw structured arguments, precise technical terms, and consistent formatting, so it mapped those patterns to a machine. It has no idea I spent last thirty some years watching legacy systems evolve. It was just calculating word probabilities.

And that is the exact trap we fall into when trying to make AI work in the real world.

Back in Part 3, I tried to explain the AI language in simple terms. Agents, RAG, embeddings, vector databases, orchestration, guardrails. All those terms matter, but at the end of the day, three practical questions rule everything: what can the machine see, what can it do, and who gets blamed when it screws up?

Even if you manage to answer those three, another question pops up right away. And this one is harder to ignore:

Do we understand the data the system is using?

Ask most IT leaders and they’ll say yes without even thinking. Government agencies run on process. Systems of record, assigned data owners, security frameworks, privacy officers, retention schedules, backup platforms, and auditors. There are architecture diagrams, classification schemes, and formal procedures for almost everything.

Then somebody asks a simple question: “Where does every copy of this citizen’s record live?”

Crickets.

They aren’t asking about the main database. Everyone knows where that sits. The real headache is tracing what happened after real people started doing their daily jobs. Which version was dumped into an Excel sheet three years ago? Which copy was emailed over to an insurance company? Which contractor got a “quick subset” for a pilot project? Which staging database was cloned during a cloud migration and then left behind because someone forgot to delete it?

Which version is current? Which one holds sensitive information? Which one should have been deleted five years ago?

“Know your data” is no longer a strategic principle. It’s more like a scene from CSI.

---

The Illusion of the Architecture Diagram

Government technology relies on the “systems of record.” Tax engines hold tax data. Benefits portals handle eligibility. Public safety manages criminal records. HR has employee files. Finance tracks the money.

On paper, it’s comforting. Everything sits neatly in its assigned box.

Real life, however, doesn’t care about an architecture diagram.

Data moves because people need to get work done today. Employees run data exports. Analysts build custom models. Developers clone production databases to troubleshoot ‘undocumented features’. Agencies send files back and forth. An intern attaches a spreadsheet to an email because the “approved” request process takes three weeks and the citizen standing right at the counter needs help right now.

Over time, that single source of truth gets surrounded by a massive cluster of copies, drafts, old revisions, extracts, caches, backups, and forgotten project folders.

Official Database ---> Export to Excel ---> Emailed to Team ---> Saved to Desktop --->
---> Forgotten in "Archive_v2"

We need some of those copies. Many are harmless. But some are completely outdated, sitting around holding sensitive details that we forgot even existed.

We like to call this dark data. The name sounds like a sci-fi villain, but the reality is much more simple. Dark data usually lives inside a completely ordinary folder named something like [Archive_Final_UseThisOne] or [Old_DoNotDelete] — file names that have quietly caused more damage over the years than any zero-day attack.

Dark data isn’t hidden by complex encryption; it’s hidden by neglect. Nobody is paying attention to it anymore.

And that’s why it’s so dangerous. An old folder from 2018 might still hold Social Security numbers, tax filings, health records, or plain-text API keys. While your production systems are patched, monitored, and protected, these forgotten folders sit in the dark with open permissions and no oversight.

They are still there simply because cloud storage got cheap enough that keeping everything forever felt safer than making a hard decision about what to delete.

---

The ROT Problem and the AI Game of Telephone

Have you heard AI folks talking about ROT?

All it stands for is Redundant, Obsolete, and Trivial data.

It sounds dramatic, but the actual risk is clear. Every duplicate is another place where hackers can attack. Every obsolete document increases the odds that a human — or an AI — grabs the wrong facts.

People usually frame ROT as a storage cost issue. But I think, it’s a trust issue.

Imagine an AI bot tasked with pulling the current policy for a state benefits program. It scans the repository and finds four separate documents:

• The currently approved policy.

• An early draft from two years ago.

• A policy that was officially replaced last month.

• A copy modified by an employee, complete with personal notes, saved with a file name that looks more official than the actual master file.

All four documents match the search criteria. Only one is actually correct.

[ Draft Policy (2022) ]
       [ Replaced Policy (2025) ]   --->  AI Engine  --->  Confidently Summarizes
       [ Annotated Copy ]                                  the WRONG Rules
     ★ [ Official Policy (2026) ]

So now, imagine the AI agent pulls a wrong file and runs with it.

That is one of the most frustrating things about modern AI: it doesn’t need to hallucinate to give you a terrible answer. It can follow instructions perfectly, retrieve the exact document it was pointed to, and still produce total nonsense because the source material was obsolete or misleading. And when the response comes back in polished, grammatically correct form, people are not going to question it.

---

Mainframes, Institutional Memory, and the Context Gap

I hope it doesn’t surprise you that some of state government legacy applications still run on mainframes. These legacy engines handle taxes, licensing, pensions, and public safety — systems that cannot go offline.

The standard complaint is that mainframes are old, but age isn’t the main issue. The real problem is context. The data inside a mainframe is very hard to translate, link to modern apps, or move around without making yet another copy.

Building a modern AI workflow usually means putting together data from a mainframe, a SaaS tool, a cloud repository, and three separate databases. Unifying that mess requires custom translation layers, plus that one OG staff member who remembers that the field STAT-7 actually means something completely different than what the 1994 manual claims. And that person is usually about three months away from retirement.

I used to have a neighbor who worked for IBM for 37 years, mainly because he was the only support guy left who knew a specific mainframe application. When the last customer using it didn’t renew their maintenance contract, he was laid off.

Migrating data to the cloud doesn’t magically fix this either. Moving a database over to AWS doesn’t tell you whether a status code of 3 means Active, Suspended, Deceased, or Pending Review.

Modernization isn’t the same as understanding. Often, it’s just moving a mess from an old basement to a newer, much more expensive basement. And yes, the cloud gets expensive fast — all those egress charges add up... Just ask any government agency CFO.

People survive in these environments using their experience and instinct. They know which database is the true authority, which folder holds junk drafts, and which status codes are legacy quirks. AI doesn’t have instinct — it only has what you feed it.

---

Where Control Breaks: Shadow Environments and “Fast” AI

Most compliance models look great on paper. Federal Tax Information (FTI) follows IRS Pub 1075. Health data follows HIPAA. Criminal justice details follow CJIS.

The rules are clear enough as long as data stays inside the production system. Everything breaks down the moment data starts migrating.

A tax record doesn’t lose its legal protections because an analyst exported it to a .csv file. Criminal history doesn’t become public domain because it was copied into a dev environment for testing.

This makes non-production environments — dev, test, staging, and sandboxes — much more dangerous than production systems.

Production (Strict Security) -->[ Copy for Testing ]---> Dev Sandbox (Weak Access, Forgotten)

Production systems get the budget, the active monitoring, and the security patches. Sandboxes, on the other hand, are treated like temporary playgrounds. But developers need realistic inputs to build tools. So what happens? Someone clones production data into a dev environment. The project finishes, the team moves on, but that cloned dataset sits there indefinitely with loose access controls and stale credentials.

---

I wrote about this back in October of last year: A SAP Business Objects reporting server running in the development environment that someone had pointed at the production database back in 2012. Granted, that specific case had nothing to do with AI — the year was 2016, I think — but the underlying habit is exactly the same.

---

Generative AI projects make this habit much worse because the priority now is speed. Teams launch vector databases, create embeddings, spin up sandboxes, and index file shares just to prove a concept works. Security and privacy reviews are treated like a final checkbox after the demo works.

By the time everyone finishes celebrating a working demo, sensitive data has quietly moved across five new, unmonitored systems.

---

You Can’t Just Hit “Delete”

There is a dangerous assumption AI project teams make: “If there’s an issue, we’ll just delete the source file later.”

If you were using a basic search index, sure — you might be able to purge a cached file. But if that data was used to fine-tune a model or build embeddings, cleanup isn’t that simple. Models don’t store files like a database folder; they absorb statistical relationships. You can’t just run an UPDATE command to wipe a single person’s history out of a model’s weights.

That’s why production data must be classified, masked, or converted to synthetic data before it ever touches an AI pipeline.

Synthetic data isn’t just junk filler. It mirrors the statistical patterns and structural logic of real data — letting you train and test workflows — without risking actual people’s identities.

An AI workflow needs to know how a benefits application flows. It doesn’t need my real Social Security number to figure that out.

---

The Real Cost: Public Records and Broken Trust

Let us look at state public records requests to see what poor data management actually costs in practice.

A person files a simple request for documents regarding a municipal project. Finding those files requires digging through agency inboxes, legacy network shares, backup tapes, and folders nobody had touched in a while.

Finding the files is only half the battle, though. People then have to manually review every single page:

• Is this correct?

• Is this a duplicate?

• Is this a draft or the final version?

• Does page 47 contain a plain text SSN?

In states like Massachusetts, tight mandatory deadlines mean agencies have days — not months — to respond. When delays happen, the public assumes the agency is covering something up. In reality, a team of tired people is usually just struggling to figure out if a 2018 PDF was ever officially signed.

---

From Static Inventories to Knowledge Graphs

Last week I completed technical training on the Everpure Data Intelligence platform. The technology focuses on continuous discovery and contextual mapping across structured and unstructured data — cloud apps, mainframes, and local shares alike.

The tool itself is slick, but the real takeaway was a shift in mindset: data classification can never be a one-time event.

Traditional data inventory projects are static. Agencies pay a lot of money to hire consultants, run scans, generate a report, pat themselves on the back, and archive the findings. That report becomes obsolete the moment an employee creates a new spreadsheet, copies a folder, or spins up a new AI test sandbox. Static tracking fails because our data is dynamic.

We need a continuous, living understanding of our environment. This is where semantic metadata and knowledge graphs enter the picture.

[ Citizen Entity ]
       │
       ├── Linked to ---> [ Federal Tax Record (IRS Pub 1075) ]
       │                        │
       │                  Copied to...
       │                        │
       └── Linked to ---> [ Dev Sandbox Folder ] ---> (!*!) ALERT: Flagged for Masking

Instead of treating every file or database as an isolated island, a knowledge graph maps real-world relationships. It understands that File X contains federal tax info, connects to User Y, was cloned into Staging Area Z, and falls under specific privacy mandates.

With that context layered over your storage, you can automate real governance:

• A production database cloned into a test environment gets automatically masked.

• A vector database trying to index restricted CJIS files gets blocked instantly.

• An obsolete, duplicate policy doc gets flagged for removal before the AI can retrieve it.

---

Self-Aware Storage

For decades, enterprise storage vendors focused on performance, uptime, and cost. Can it hold the data? Can it read it fast? Can it survive a drive failure?

AI introduces a fundamentally new requirement: does the storage infrastructure actually understand what it is holding?

This isn’t about storage becoming sentient (You may have seen in my article predicting the future of data - The Future of Data Experience). It’s about infrastructure having enough context to recognize that two identical 10GB files require completely different access rules, retention schedules, and security protocols.

Instead of passively storing the data, intelligent storage should actively participate in data governance.

┌──────────────────────────────────────────────────────────┐
│                   SELF-AWARE STORAGE                     │
├────────────────────────────┬─────────────────────────────┤
│   Public PR Documents      │   Federal Tax Records       │
│   • Open Access            │   • Restricted Access       │
│   • Indexed for AI         │   • Masked in Test Apps     │
│   • Standard Retention     │   • Blocked from AI Training│
└────────────────────────────┴─────────────────────────────┘

I want to be clear here: tools don’t replace human oversight. Vendors love to make platform architectures sound like magic spells. A discovery engine can highlight risk, but it can’t write your agency’s policies. A knowledge graph reveals connections, but it can’t replace your privacy officers, legal counsel, or security teams.

Technology simply gives human policy a chance to scale.

---

The Checklist Before You Launch

While researching and pulling together information for these four posts, I feel like I’ve really grasped where the core bottlenecks lie. At least for now!

AI connects infrastructure, quality, security, and privacy together whether we like it or not.

An AI strategy centered purely on model selection (choosing between OpenAI, Anthropic, or open-source alternatives) misses the point entirely. The model matters far less than the operating framework surrounding your data.

I don’t pretend to know everything, but before launching any major AI initiative, please skip the shiny vendor pitch and ask these operational questions first:

1. Can we dynamically identify where regulated data lives across all systems?

2. Do we know with 100% certainty which copy of a document contains the truth?

3. Are we able to automatically distinguish production data from test data?

4. Can we trace exactly where a dataset originated and everywhere it has moved?

5. Is our AI strictly prevented from retrieving documents outside approved permissions?

6. Can we cite the exact, verified source behind an AI-generated answer?

7. Can we purge obsolete records from the AI’s search path without breaking downstream workflows?

8. Can we reconstruct every step an AI bot took when executing a task?

9. Does the system stop and flag an error when two core source systems contradict each other?

10. Can we clearly explain the output to a person affected by the decision?

You won’t find the answers to those questions on a benchmark leaderboard. And a slick chat interface won’t save you when things go sideways.

---

Final Thoughts: Trust is the Real Runtime

“Know your data” has become a corporate tagline because you cannot really argue with it. But for too long, many organizations treated it as a vague, long-term goal while managing only their top three production databases.

We simply cannot do things that way any longer.

When a person searches a system, they're after something specific, and they bring their own judgment and instinct. An AI agent doesn't work that way. It crosses system boundaries, puts together data from sources that were never meant to sit next to each other, and then acts on all of it at scale. Which leaves us somewhere uncomfortable: the better we get at teaching machines to find their way around our enterprise environments, the more dangerous it becomes that we're still lost inside them ourselves.

This is where the whole series has been heading:

• Part 1 was about AI-generated information becoming part of the data that future AI systems consume.

• Part 2 looked at the machine behind the language and why statistical prediction is not the same as understanding. Take that, Pangram!

• Part 3 followed the path from an assistant that writes to an agent that acts.

• Part 4 ends with the foundation underneath all of them.

The machine cannot govern data nobody understands. It cannot separate facts from junk.

AI doesn’t just run on compute. It runs on trust. And that trust has to be engineered directly into how you discover, map, and control your data.

That is the line between playing with AI and actually being ready for it.

Appreciate you reading.

Dmitry Gorbatov

© 2025 Dmitry Gorbatov | #dmitrywashere