Part 3: The Government AI Dictionary Nobody Asked For

Before an AI agent can transform government, somebody should probably explain what one is

Dear readers,

Every article here takes a fair amount of research, trial and error, rewriting, editing, and the occasional argument with English grammar. I am not monetizing this. I am just learning in public and sharing what I figure out along the way.

If you find it useful, interesting, or even mildly entertaining, please subscribe and share it with someone else. It costs nothing, and it helps more than you may think.

The least you can do is click the button.

---

In Part 1, I wrote about the hall of mirrors created when AI-generated information becomes part of the data that other AI systems consume.

In Part 2, I looked at the machine standing inside that hall. Large language models are remarkable statistical systems, but they do not begin with truth. They generate plausible language based on patterns, context, and the information made available to them.

That led me to a fairly simple conclusion:

AI does not run on data. It runs on trust in that data.

I think the problem is that this conclusion arrives just as government technology leaders are being buried under an entirely new vocabulary.

Generative AI. Retrieval-augmented generation. Vector databases. Embeddings. Knowledge graphs. Semantic search. Large language models. Small language models. Fine-tuning. Grounding. Inference. Orchestration. Tool calling. MCP. Agentic workflows. Multi-agent systems. Human-in-the-loop. Autonomous agents.

Somewhere in a state office building, an employee who spent twenty years learning procurement rules, public-records requirements, agency policy, and the difference between seven systems that all claim to contain the official citizen address is now being told to develop an “agentic AI strategy.”

The terminology is not useless. Each term describes something real. But the volume of it creates the impression that government leaders must understand an entirely new branch of computer science before they can make a responsible decision.

The reason I am writing this is to help people understand what authority is being given to the machine, what information the machine can see, and what happens when it is wrong.

Everything else begins there.

Start With the Difference Between Generating and Doing

Most people first encountered modern AI through a chatbot.

You ask it to draft an email, summarize a policy, explain a regulation, or turn meeting notes into something readable. The system produces content, and a human decides what to do with it.

This is generative AI.

The model generates something. It does not necessarily take action in the outside world. It may produce a very convincing letter denying a benefits application, but unless someone or some other system sends that letter, updates the applicant’s status, and closes the case, the model has only created words.

That distinction is important because much of the public conversation now moves casually between generative AI and agentic AI as though they are different subscription levels of the same product.

They are not.

An agent does not merely produce an answer. It is given a goal, access to tools, and some degree of authority to take steps toward completing that goal.

A generative AI system might draft an explanation of why an application is incomplete.

An agentic system might open the case, identify the missing document, search another agency system to determine whether that document already exists, contact the applicant, update the workflow, schedule a reminder, and escalate the case if no response arrives within ten days.

The language model may still sit in the middle of that process, interpreting instructions and deciding what to do next. But the real change is not that the model became smarter.

The real change is that somebody gave it keys.

What Is an AI Agent?

The word “agent” is being used so broadly that it occasionally means little more than “chatbot with ambition.”

A practical definition is simpler.

An AI agent is a system that can examine a situation, decide on a next action, use an available tool, observe the result, and continue until it reaches a goal or encounters a condition requiring human intervention.

The tools may include databases, search systems, email, document repositories, case-management applications, APIs, calendars, or software used to update official records.

---

Consider a state employee onboarding process.

A normal chatbot can answer questions about benefits, office policy, or required forms.

An agent could verify that the employee’s paperwork is complete, create the appropriate accounts, assign training, request equipment, notify payroll, schedule orientation, and track which steps remain unfinished.

That sounds efficient because it probably is.

It also means the system may need access to identity records, salary information, tax forms, network credentials, background-check status, and internal administrative tools. It must know which actions it is allowed to perform, which actions may require approval, and what to do.

This is why the move from a chatbot to an agent is not merely a feature upgrade.

It is a transfer of authority.

The Five Levels of AI Ambition

There is no single maturity model that everyone follows, which is probably for the best because the industry already has enough diagrams. Still, most government AI projects move through a fairly recognizable progression.

They begin with a system that helps a person write or search. Then they give it access to agency information. After that, they let it complete a small task. Eventually, someone asks whether it can manage an entire workflow. The final step is when the system starts acting before anyone asks it to.

That progression matters because each step adds more than capability. It adds access, authority, and consequences.

Level 1: The Assistant

At the first level, AI helps a person produce something.

It drafts emails and follow-up notes, summarizes meetings, explains technical material, produces boilerplate code, or helps an employee find something buried in a policy manual.

The system has very little authority. A person reviews the output and remains responsible for the decision.

This is where many agencies should begin because the risk is still relatively contained. The AI can be wrong, but the mistake has to pass through a human before it becomes official.

That protection only works when the human actually reads what the system produced. Clicking “Looks good” on the way into another meeting technically counts as review, but not by much.

Level 2: The Context-Aware Assistant

The next step is to connect the model to agency information.

Instead of asking it to answer from whatever it learned during training, the system retrieves relevant policies, manuals, case documents, or program guidelines and gives them to the model before it responds.

This is retrieval-augmented generation, usually shortened to RAG.

The name makes it sound more complicated than it is. The system searches for something relevant, passes that information to the model, and asks the model to answer using what it found.

A public benefits chatbot, for example, might retrieve the current eligibility guidelines for a specific program instead of relying on general information that may have been learned months or years earlier.

This is often described as grounding the model.

Grounding does not mean the model has suddenly developed a relationship with truth. It means the answer is tied to selected source material.

That helps, but only if the system found the right source in the first place. Is the document current? Is it authoritative? Is there another version sitting in a different repository? Is the person asking the question even allowed to see it?

RAG can make AI much more useful. It cannot make disorganized information trustworthy.

Level 3: The Task Automator

At the third level, the system is allowed to do something.

The action is usually limited and happens inside a controlled workflow. The system may route a service ticket, populate fields in an application, classify an incoming document, schedule an inspection, or send a standard notification.

The environment is structured. The choices are restricted. The agent is not wandering through the state network looking for ways to improve government before lunch.

A human may approve the action before it becomes final, or the system may act automatically when predefined conditions are met.

This is where the phrase human-in-the-loop begins appearing.

The phrase sounds reassuring, but it does not tell us how meaningful that human involvement actually is.

A person who reviews the underlying records and understands the decision is genuinely in the loop. A person who receives 600 AI-generated approvals every morning and clicks through them because the queue must be cleared by Friday is also technically in the loop.

The question is not whether a human appears somewhere in the workflow. It is whether that person has enough time, information, and authority to catch a mistake.

Level 4: The Workflow Agent

At the fourth level, the system manages a process that crosses several steps and often several systems.

It remembers what has already happened. It knows that a document was requested, that the citizen responded, that a review is still pending, and that the case must be escalated if nobody acts within a required period.

This is sometimes called a stateful workflow. “Stateful” simply means the system keeps track of where the work stands.

More than one agent may participate. One searches documents. Another checks eligibility. Another drafts communications. Another updates the case-management system.

That is a multi-agent system.

The name may bring to mind a small digital workforce holding status meetings inside the server. In reality, it usually means several software components have been assigned different jobs and pass information between one another.

This can be useful when the responsibilities are clearly separated.

It can also create a very modern form of bureaucratic confusion in which nobody knows which agent made the original mistake, but four other agents have already accepted it as fact.

At this level, context becomes critical. The system must understand not just what a record says, but how that record relates to a person, a case, a policy, an agency, and the action being considered.

A date in a database means almost nothing on its own. It could be an application date, a birth date, a review deadline, or the date an old policy expired.

A name is not enough either. Is John Smith the applicant, the caseworker, the parent, the attorney, or an unrelated resident who happens to share a very common name?

Computers have always struggled with these relationships. Agents make the consequences more immediate because now they can act on their interpretation.

Level 5: The Proactive Agent

At the final level, the system does not wait for a person to begin the workflow.

It watches for a condition, identifies something that requires attention, and starts an action on its own.

A system might detect that a regulatory deadline is approaching, that an eligibility record has changed, that an unusual backlog is developing, or that two administrative rules appear to conflict.

Virginia offered an early example in July 2025 when the state launched an agentic AI regulatory-review pilot. The initiative was designed to help agencies examine regulations and guidance for conflicts with state law, outdated language, unnecessary requirements, and duplication. Delaware launched an AI sandbox initiative that same month to create a controlled environment for testing more advanced AI systems and the governance around them.

These projects matter because they show how quickly the conversation is moving beyond chatbots.

A proactive agent may identify problems that employees did not know to search for. It can monitor more information than any one person and respond much faster.

It can also launch the wrong workflow much faster.

Once a system is allowed to act without being prompted, logging, identity, policy controls, exception handling, and the ability to stop or reverse an action are no longer background features. They are the architecture.

A Few More Terms Before the Acronyms Escape

Once RAG and agents enter the conversation, several other terms usually follow. Most of them sound more mysterious than they really are.

Embeddings

An embedding is a way of turning meaning into numbers.

The system converts a piece of text, an image, or another kind of information into a mathematical representation. Content with similar meaning tends to end up closer together in that mathematical space.

That allows a search system to find related ideas even when the exact words do not match.

A keyword search for “vehicle” may miss a document that only uses “car” or “automobile.” A semantic search may understand that the ideas are related.

This is useful in government, where two agencies can describe the same thing using language created by different committees in different decades.

The limitation is that similarity is not authority. A draft policy may look very similar to the approved one. An expired regulation may resemble its replacement. A record for Robert may be connected to a document containing Bob.

The search may be intelligent and still retrieve the wrong thing.

Vector Databases

A vector database stores those mathematical representations and makes them searchable.

When someone asks a question, the system converts the question into an embedding, looks for nearby information, and passes what it finds to the language model.

It is not necessarily a replacement for the agency’s existing database or document system. It is usually another layer added to help AI find semantically related material.

That additional layer can be useful. It can also create another copy of the data, another access path, another security boundary, and another place where old information quietly survives.

The database may return an answer in milliseconds. The harder question is whether anyone knows what was copied into it six months ago.

Knowledge Graphs

A knowledge graph focuses on relationships.

A normal database may tell us that two records contain the same identification number.

A knowledge graph tries to show what the relationship means. This person submitted this application, belongs to this household, received this benefit, interacted with this agency, and is connected to this case.

The word “graph” does not mean a chart. It refers to a structure made up of entities and the connections between them.

This becomes valuable when an agent needs to understand how information in one system relates to information somewhere else.

It also creates risk when those relationships are wrong, incomplete, or exposed to people who should not see them.

Fine-Tuning

Fine-tuning is additional training applied to an existing model.

An agency might fine-tune a model to recognize a certain document format, classify records, or produce responses in a more consistent style.

That can improve how the model behaves. It does not automatically give the model current agency information.

A model fine-tuned on last year’s policy manual does not know that the policy changed yesterday. For information that changes regularly, retrieval is usually more practical than retraining the model every time something is updated.

Fine-tuning teaches the model how to behave.

Retrieval gives it something current to read.

Inference

Training is the process of building or modifying a model.

Inference is what happens when that model is used.

Every time the model generates an answer, makes a prediction, or classifies something, it is performing inference.

When people say inference should happen near the data, they usually mean the model should process information close to where it is stored instead of forcing the agency to copy everything into a distant external service.

That decision affects data gravity, sovereignty, latency, security, and cost.

Orchestration

Orchestration is the part that coordinates everything.

It decides which model to call, which document repository to search, which tool an agent is allowed to use, when approval is required, how one step passes information to the next, and what happens when something fails.

The model gets most of the attention because it is the part people can see and talk to.

The orchestration layer is often where the real application lives.

It is also where a harmless chatbot begins turning into something capable of changing an official record.

Guardrails

Guardrails are controls intended to limit what the system can do or say.

They may include access restrictions, content filters, approval steps, tool permissions, policy checks, logging, output validation, and rules that prevent certain types of information from leaving an environment.

The word sounds reassuring.

A guardrail on a mountain road works because engineers know where the road ends and the cliff begins. AI systems operate in language, context, policy, and user intent, where the boundary is often much less obvious.

Guardrails are necessary. The existence of the word does not mean the problem has been solved.

The Three Questions Behind All the Jargon

When a vendor, consultant, or internal team presents an AI proposal, government leaders do not need to answer with an equally impressive collection of acronyms.

They can ask three fairly ordinary questions.

What can the system see?

What can the system do?

Who is accountable when it is wrong?

The first question gets to the data architecture. Is the system working with public information, approved agency records, employee email, copied production data, tax information, criminal justice records, health information, or some combination that nobody has fully inventoried?

The second gets to authority. Is the system generating a recommendation, or can it update a record, contact a citizen, approve a transaction, or trigger another workflow?

The third question tells you whether the project was designed for government or merely demonstrated to government.

Accountability cannot be assigned to “the AI.” A model cannot testify at a court hearing, explain why it retrieved an outdated document, accept disciplinary action, or restore a benefit that was denied incorrectly.

A person or an organization still owns the system. Someone approves the data it can use. Someone defines the permissions. Someone decides where a human must remain involved.

And when the system does something nobody expected, someone must be able to reconstruct what happened.

The Agent Is Only as Ready as the Data Beneath It

The current AI conversation often starts with the model.

Which one performs best? Which has the largest context window? Which platform offers the strongest reasoning? Which vendor has the most convincing demonstration?

Those are legitimate questions. They are just not the first ones I would ask.

Before an agency gives an AI model access to a workflow, it needs to know what information exists, where that information lives, whether it is current, what it means, who is allowed to use it, and how records across different systems relate to one another.

Back in March, I wrote about why the public sector reveals the truth about data. State governments operate some of the most complicated data environments in the world. Modern cloud services sit next to decades-old applications, mainframes, departmental databases, email archives, file shares, contractor platforms, backups, test environments, and copies created for projects everyone has since forgotten.

An agent does not clean up that complexity simply by arriving.

It inherits it.

The better an agent becomes at searching across systems and taking action, the more dangerous unknown, duplicated, stale, or exposed information becomes.

We have spent the last several years asking whether AI can understand our questions.

The next question is whether government understands the information being placed in front of it.

---

Coming in Part 4

Part 4 is where the fun ends and the inventory begins.

I will probably get to it in the next 2 weeks, or tomorrow if the flight gets delayed.

The plan is to close where this whole series has been heading with a fairly blunt question: whether “know your data” can be something a system actually enforces, or whether it stays what it usually is: a sentence a CIO says with great confidence from a conference stage.

Appreciate you reading.

Dmitry Gorbatov

© 2025 Dmitry Gorbatov | #dmitrywashere