Most AI agent projects spend weeks choosing a model and an afternoon on the documents. Then the agent confidently quotes last year's price list, an offboarding checklist from a department that no longer exists, or two different refund rules, depending on which file it found first. The model did not fail. The company's knowledge was never ready to be read by anything that cannot ask a colleague, "Is this still true?"
This article is for the manager who owns the process the agent will support. It follows describing the process and comes before data handling and retention. Here the question is narrower: which documents may the agent rely on, and who keeps them true?
1. Why documents matter more for an agent than for people
A new employee who finds two conflicting price lists will ask. An agent that retrieves documents does not. It takes what looks most relevant and answers in a calm, confident tone. If you want the technical background, our guide to retrieval-augmented generation explains how the agent finds text, and why the quality of the answer is limited by the quality of what it finds.
The same applies to the "unofficial" knowledge: the spreadsheet one person keeps, the email thread where an exception was agreed, the rule everybody knows but nobody wrote down. If the agent needs it, it has to be written down. If it should not be used, it has to be out of reach.
2. Start with a short list, not the whole file server
Do not feed the agent everything. Take the process you chose and list the questions the agent will be asked. For a customer service agent that might be delivery times, returns, invoicing, warranty and a few product facts. For an internal agent it might be leave rules, expense limits and IT access requests.
For each question write down which document answers it. Typical categories are:
- Procedures: how a task is done step by step, including exceptions.
- Prices and conditions: price lists, discount rules, delivery and payment terms.
- FAQ and product information: the short answers customers or colleagues ask for repeatedly.
- Policies: returns, privacy, security, travel and similar rules.
- Templates: standard replies, contract wording, forms.
A list of twenty documents that are right is worth more than two thousand that might be.
3. One source of truth per answer
For every kind of answer pick exactly one place where the correct version lives. Not "the wiki and also the PDF on the shared drive". If copies must exist, for example a PDF sent to customers, they are generated from the master and carry its version date.
When you find two documents that disagree, do not let the agent choose. Decide which one is correct, fix or delete the other, and note the decision. This is usually the slowest part of the preparation, and also the most valuable: the agent project simply exposes inconsistencies the company has lived with for years.
4. A named owner for each document
Every document in the agent's scope needs a person, not a department, who is responsible for its accuracy. The owner is not the one who stored the file, but the one who would know when it changes: the sales manager for the price list, the head of customer service for the returns policy, the HR lead for leave rules.
The owner answers three things: Is this still correct? When will it next change? Who do we tell when it does? Put the name next to the document, in the file itself or in a simple register. A document without an owner should not go into the agent's knowledge until it has one. Ownership of data and accountability for AI systems is also a core theme of the NIST AI Risk Management Framework, which asks organisations to assign clear responsibility for how AI systems and their inputs are managed.
5. Validity dates and version marks
Add two simple fields to every document: the date it was last confirmed as correct, and the date by which it must be reviewed again. Quarterly is a reasonable default for conditions and policies, monthly for prices and promotions, yearly for slow-changing background.
Dates help in three ways. People see at a glance how old the text is. The agent can be told to mention the date or to hand over to a person when a document is past its review date. And your monthly check has something concrete to filter on. Where information has a hard end, such as a campaign or a temporary rule, write the end date into the text itself.
6. Retire what is outdated
Before launch, go through the documents in scope and sort them:
- Current: keep, with owner and dates.
- Superseded: move to an archive the agent cannot reach, and keep it for your own records.
- Unclear: ask the owner, and keep it out until answered.
- Not needed: leave out. A narrow knowledge base gives better answers than a broad one.
Pay attention to old versions with similar names, such as "Price list 2024 final" and "Price list 2025 final v2". They are the most common source of wrong answers. If you must keep history, store it where the agent has no access, not in the same folder under another name.
7. Check personal and sensitive content
Documents often contain more than their title suggests: customer names in an example, salaries in an attachment, passwords in a manual. Remove or redact such content before it reaches the agent. The GDPR principles require that personal data is accurate, kept up to date and not stored longer than necessary (GDPR Art. 5). An outdated document with personal data in it fails both tests at once. For the rules on retention and access, see data handling and retention.
8. Test with real questions
Once the documents are ready, take thirty real questions from your inbox or support history and ask the agent. For each wrong or weak answer find out the cause: a missing document, an outdated one, two conflicting ones, or unclear wording. Fix the document, not the question. Keep the set of questions, and run it again after every larger change. This links directly to evaluating reliability before production.
9. Keep it current after launch
Documents change, and the agent will not notice by itself. Agree a routine:
- Owners get a reminder when the review date approaches.
- Any change to prices, conditions or policies includes the step "update the agent's source".
- The agent's wrong answers are traced back to a document and reported to its owner.
- Once a month someone checks the register for documents past their date.
This fits into the regular operational review of the agent. Cloud providers describing retrieval solutions make the same point: the quality and freshness of the source content, and how it is split and indexed, directly shape the answers (Microsoft Azure Architecture Center, RAG solution design guide).
A short checklist
- List the questions the agent will answer and the document for each.
- Choose one source of truth per kind of answer, and settle any conflict.
- Name a person as owner of every document in scope.
- Add a last-confirmed date and a review date.
- Archive superseded versions out of the agent's reach.
- Remove personal and secret content.
- Test with thirty real questions and fix the documents behind wrong answers.
- Set a routine so that every change of price or policy updates the agent.
The agent will repeat whatever it is given, with great confidence. Preparing the documents is not glamorous work, but it is where most of the answer quality is decided.
