If a software supplier has emailed to say AI features are coming to a product you already use, these are the questions to send back before your team starts using them. There are 12, with what a good answer looks like and the replies that should make you pause.
Why this is landing on your desk now
Most businesses aren't going out to buy an AI product. AI is arriving inside software they already pay for: the accounting package, the CRM, Microsoft 365 or Google Workspace, the document system, and in finance firms, practice management and portfolio systems. It usually comes as an email announcing "new AI features", and sometimes the features are switched on before anyone has read it.
An update doesn't change what you're responsible for. If the supplier handles personal data for you, such as customer, client or staff details, UK GDPR says you must only use suppliers that provide "sufficient guarantees" that the data will be handled properly (Article 28). It also says they can't bring in another company to process that data without your written authorisation. An AI feature very often means exactly that: a new company in the chain, usually the business that provides the AI model.
You can still switch the feature on. Ask the questions first and keep the supplier's answers on file.
Before you ask: three things to know about your own position
What goes through this system? Customer names and addresses, staff records, bank details, contracts, unpublished figures? The more sensitive the data, the more the answers matter.
How much do you rely on it? A tool one person uses now and then is different from the system your invoicing or reporting runs on every day.
Who decides? Somebody in the business needs to own the decision to switch the feature on, leave it off, or allow it for some uses only. In a small business that's usually the owner. In a larger one, agree who it is before the questions go out.
The 12 questions
Your data
1. Will our data be used to train or improve any AI model, yours or anyone else's?
Why it matters: once your data has been used to train a model, you can't take it back.
A good answer: "No", in writing, or a clear opt-out that is already set for your account.
A red flag: "only in anonymised form", with no explanation of what that means, or an opt-out you have to request.
2. Where is our data processed and stored when the AI feature is used?
Why it matters: the supplier may be UK-based while the AI processing happens somewhere else.
A good answer: named countries or regions, and a UK option if you need one.
A red flag: "in the cloud", or no answer about where the AI part runs.
3. Which other companies are involved, and has your list of sub-processors changed?
Why it matters: this is the Article 28 point above. If your contract allows them to add sub-processors, they must tell you about changes and give you the chance to object.
A good answer: an updated sub-processor list that names the AI provider, sent before the feature goes live.
A red flag: "we use a leading AI provider" with no name.
4. How long are prompts and outputs kept, and can we delete them?
Why it matters: whatever your team types in, and whatever comes back, is data too.
A good answer: a stated retention period, and a way for you to delete it.
A red flag: no retention period, or "kept to improve the service".
Control
5. Is the feature on by default, and can we switch it off for a person, a team or a client?
Why it matters: you want to decide who uses it, rather than discover who already has.
A good answer: off until you switch it on, with settings by user or group.
A red flag: on for everyone, with no way to switch it off below account level.
6. Which AI model does it use, and what happens when that model changes?
Why it matters: providers update and retire models, and the results can change when they do.
A good answer: the model is named, and you're told in advance before it changes.
A red flag: "our proprietary AI", with nothing more specific.
Accuracy and records
7. How have you tested that it does the job accurately for work like ours?
Why it matters: a tool that is good at summarising emails may be poor at reading invoices.
A good answer: a description of how they tested it, and what it is known to get wrong.
A red flag: marketing claims with no testing behind them.
8. What does it do when it isn't sure?
Why it matters: if it gives a wrong answer without any warning, your team is more likely to rely on it.
A good answer: it highlights low-confidence results and shows where each answer came from.
A red flag: it always gives an answer, with no indication of how sure it is.
9. What is recorded about its use, and can we export those records?
Why it matters: if it drafts an entry, a letter or a summary, you may later need to show what it was given, what it produced and who checked it. Your accountant, auditor or insurer may ask.
A good answer: a log of inputs, outputs and users that you can export.
A red flag: "we can't share that".
Contract and leaving
10. Does this change our contract or our data processing terms?
Why it matters: new features sometimes come with new terms that you accept by using them.
A good answer: a clear statement of what has changed, with the updated documents.
A red flag: a link to updated online terms and nothing else.
11. What does it cost, and does the price change if we switch it off?
Why it matters: AI features are often bundled into a price rise, or charged per user whether or not people use them.
A good answer: the cost is clear, and ideally switching it off brings the price down, or at least doesn't raise it.
A red flag: a price rise whether or not you use it.
12. If we leave, what happens to our data, including anything the AI produced?
Why it matters: summaries, extracted data and settings created by the AI are part of your records.
A good answer: a full export in standard formats, and deletion from the AI provider's systems as well as the supplier's. (For comparison, here are our own exit terms.)
A red flag: an export that leaves out the AI-generated content, or no answer about the AI provider.
Answers worth resolving before you switch it on
You don't need perfect answers to all 12. In our view, these are the ones worth sorting out before the feature goes live:
- your data is used for training by default, with no opt-out
- nobody will name the AI provider or where it processes your data
- you can't get records of what the feature did
- the terms changed and nobody told you
What to do with the answers
Write them down. For most businesses a simple list is enough: the tool, the feature, who owns the decision, what was decided and when to look at it again.
Decide. You can switch it on, leave it off, or allow it for some uses only. "Only for drafting, never with client data" is a perfectly good decision.
Check whether you need a data protection impact assessment (DPIA). The ICO's guidance says that "in the vast majority of cases, the use of AI will involve a type of processing likely to result in a high risk to individuals' rights and freedoms, and will therefore trigger the legal requirement for you to undertake a DPIA." If personal data is involved, assume you need one until you've checked. The ICO notes that this guidance is under review following the Data (Use and Access) Act, so check the current version.
Set a review date. Suppliers keep changing these features, so look at the answers again in six or twelve months, or sooner if the supplier announces a change.
For most businesses, the supplier's written answers and a short note of what you decided are enough.
The checklist
Print this or copy it into an email to your supplier.
| # | Question | A good answer | A red flag |
|---|---|---|---|
| 1 | Will our data be used to train or improve any AI model? | "No", in writing, or an opt-out already set | Vague "anonymised" use, or an opt-out you have to ask for |
| 2 | Where is our data processed and stored for the AI feature? | Named countries, with a UK option if needed | "In the cloud" |
| 3 | Which other companies are involved, and has your sub-processor list changed? | An updated list naming the AI provider | "A leading AI provider", unnamed |
| 4 | How long are prompts and outputs kept, and can we delete them? | A stated period and a way to delete | No period, or "kept to improve the service" |
| 5 | Is it on by default, and can we switch it off by person, team or client? | Off until you switch it on, with per-user settings | On for everyone, account-level only |
| 6 | Which AI model does it use, and what happens when it changes? | Named model, advance notice of changes | "Our proprietary AI" |
| 7 | How have you tested it on work like ours? | A description of testing and known weaknesses | Marketing claims only |
| 8 | What does it do when it isn't sure? | Flags low confidence and shows sources | Always answers, never flags |
| 9 | What is recorded, and can we export it? | Exportable log of inputs, outputs and users | "We can't share that" |
| 10 | Does this change our contract or data processing terms? | A clear summary of changes and updated documents | A link to new online terms only |
| 11 | What does it cost, and does switching it off change the price? | Clear cost; ideally switching off lowers the price or keeps it level | A price rise whether or not you use it |
| 12 | If we leave, what happens to our data and the AI's output? | Full export in standard formats, deletion everywhere | AI output excluded from export |
Want a second opinion on a supplier's answers?
If a supplier has announced AI features and you'd like an independent view, send us their answers. We'll go through them against your obligations and tell you plainly whether we'd switch it on, and with what limits.
This article is general information, not legal, financial or regulatory advice.