You have chosen a vendor and a draft contract has arrived. At this point many managers hand the document to legal and wait. That is sensible for the contract itself, but it leaves a gap: a large part of the legal risk of an AI agent is not in the vendor's paper at all. It sits in how your company uses the agent, which data it sees, who it talks to and who is accountable when it acts.
This article is a preparation guide for the manager who owns the launch. It is not legal advice, and the rules differ by country and sector, so treat it as a list of questions to bring to your lawyer or data protection officer. The commercial side, meaning service levels, liability caps and the way out, is covered in vendor contracts, SLA, liability and exit. Here we look at what you settle inside the company.
1. Be clear about your role: you are the deployer
Most vendor contracts describe the vendor as a provider of a service. The law often looks at it from the other side. Under the EU AI Act, a company that uses an AI system under its own authority is a deployer, and the Act sets out duties for deployers of high-risk systems, such as following the instructions for use, assigning human oversight to people with the right competence, monitoring operation and keeping logs (Article 26). Whether your agent counts as high risk depends on the use case, for example in recruitment or credit decisions, so the first question for your lawyer is simply: which category are we in?
Even outside the high-risk categories, the Act expects organisations to make sure staff who operate AI systems have a sufficient level of AI literacy (Article 4). In practice that is a short training and a written description of what the agent may and may not do, which you need anyway for the runbook and handover.
If you do not operate in the EU, the details change, but the principle holds: the vendor's contract allocates money after something goes wrong, it does not remove your duty of care towards customers, employees and regulators.
2. Map the personal data and name the roles
List every kind of personal data the agent can read, write or send onward: customer records, email content, employee data, support tickets. For each, decide who determines why and how it is processed. Under the GDPR the organisation that decides is the controller, and the vendor that processes data on its behalf is usually the processor. A processor must be bound by a written contract that covers the subject matter, duration, type of data, the instructions you give, confidentiality, security, sub-processors, assistance and deletion or return at the end (Article 28).
Check the following in the vendor's data processing terms:
- Instructions. The vendor processes your data only on your documented instructions. Is the use of your data to improve their models excluded, or does it need your consent?
- Sub-processors. Which model providers and hosting companies sit behind the product, in which countries, and how are you told about changes?
- Transfers. If data leaves your region, what mechanism covers it? In the EU this is often the European Commission's standard contractual clauses, but your lawyer should confirm what applies to the vendor's setup.
- Deletion and logs. How long are prompts, outputs and logs kept, and can you delete them? This links to data handling and retention.
Often the practical conclusion is to reduce the data first. An agent that only sees the fields it needs creates fewer legal questions than one with access to the whole mailbox.
3. Ask whether an impact assessment is needed
Where processing is likely to result in a high risk to people, the GDPR requires a data protection impact assessment before the processing starts (Article 35). New technology, large scale or systematic evaluation of people are typical triggers. Your data protection officer or adviser decides, but you supply the facts: the purpose, the data, the flow, the risks and the safeguards. The one-page requirement and the risk notes from your vendor selection are a good start, and doing this before the pilot is far cheaper than after it.
4. Does the agent decide about people, or talk to them?
Two separate rules are easy to miss.
Decisions about people. The GDPR gives individuals a right not to be subject to a decision based solely on automated processing that has legal or similarly significant effects on them, with limited exceptions and safeguards such as human intervention (Article 22). If your agent approves refunds, ranks applicants or blocks accounts, ask whether a human genuinely reviews the result or merely clicks through. The design of human approval and audit trail is therefore also a legal design.
Talking to people. The AI Act contains a transparency obligation for systems that interact directly with people: they should be told they are dealing with an AI unless that is obvious (Article 50). Whatever the exact reach for your case, telling customers plainly that they are talking to an assistant, and how to reach a person, is good practice and avoids trouble later.
5. Check what the agent says on your behalf
When an agent writes to customers, its statements are your statements. Before launch, agree with legal:
- Which topics the agent must not answer, for example legal, medical or financial advice, pricing promises or contract changes.
- What it must do when it is unsure: hand over to a person with the context attached.
- Whether outputs may contain claims about your products, and who owns the approved wording.
- How complaints about the agent are recorded and answered, building on your post-mortem and customer communication routine.
Also ask the vendor in writing who owns the outputs and any content you upload, and whether the vendor offers protection if a third party claims the output infringes their rights. Terms vary widely, so do not assume.
6. Notify the people affected
Employees whose work is monitored by logs, and customers whose data enters the agent, may need updated privacy notices or, in some countries, consultation with employee representatives. Add the agent to your record of processing, update the privacy notice and tell staff what is logged and why. It is dull work, and skipping it is a common reason a successful pilot has to pause.
7. Get a written internal sign-off
A short sign-off sheet prevents the most common failure: everyone assumed somebody else had checked. It needs four names:
- The process owner accepts the use case, the limits and the human review rule.
- Legal or the data protection adviser confirms roles, data processing terms, the assessment and the notices.
- Security confirms access, logging and the way to switch the agent off, see incident response and kill switches.
- The budget holder accepts the cost and the exit plan.
Attach the vendor contract, the data map and the decision record. Then start with a staged rollout, so that the legal assumptions are tested on a small scale first.
How to work with your lawyer
Do not hand over a finished plan and a 40-page contract with the question "is this fine?". Send a one-page brief: what the agent does, which data it touches, which countries are involved, who the users are, and the five questions from the checklist below. A lawyer can answer specific questions quickly and will point out what you did not know to ask.
A short checklist
- Which risk category does the use case fall into, and which of our duties follow?
- Which personal data does the agent touch, and who is controller and processor?
- Do the processing terms cover instructions, sub-processors, transfers, deletion and no training on our data?
- Is an impact assessment needed, and who owns it?
- Does the agent decide about people or talk to them, and is the human review real?
- Are notices updated, staff informed and the record of processing amended?
- Is there a written sign-off from the process owner, legal, security and the budget holder?
A signed contract answers who pays when something goes wrong. The work in this article decides how likely it is to go wrong and whether you can show that you were careful.
Sources
- EU AI Act, Article 26: Obligations of deployers of high-risk AI systems
- EU AI Act, Article 4: AI literacy
- EU AI Act, Article 50: Transparency obligations
- GDPR, Article 28: Processor
- GDPR, Article 35: Data protection impact assessment
- GDPR, Article 22: Automated individual decision-making
- European Commission: Standard contractual clauses for international transfers
