← Newsletter · Issue 002
Ten questions that make an AI vendor sweat
Policy does not get decided in the policy meeting. It gets decided in procurement, by whoever asks the awkward question.
Last week’s issue argued that the work is the decision, not the tool. Here is the other half of that. Once you have decided a tool belongs in the work, someone has to sign for it — and that signature is where your actual data policy gets written, no matter what the policy document says.
Almost nobody enjoys this part. The demo was good, the team is keen, and the contract is forty pages of defined terms. So the questions do not get asked, and eighteen months later you find out what you agreed to.
These are ten questions in plain language. None require a lawyer to ask. What matters is less the answer than how fast it arrives — a vendor who has thought about this answers in a sentence, and a vendor who has not will offer to follow up.
What happens to the data
1. Is our data used to train or improve your models — including any model you don’t own? Listen for the second half. “We don’t train on customer data” and “no data leaves our systems” are different promises, and subprocessors live in the gap.
2. Name every subprocessor that will touch this data. A real answer is a list, or a URL to a list that is kept current. A vague answer means nobody has mapped it.
3. Where is it stored, and where is it processed? These are two questions. Storage region is easy to promise; inference often happens somewhere else.
4. How long do you keep it, and what starts the clock? Retention measured from deletion request is very different from retention measured from last touch.
What happens when something goes wrong
5. Who at your company can read our data, under what process, and is that access logged? “Only authorised personnel” is not an answer. Ask what triggers access and whether you can see the log.
6. What is your notification window after you discover a breach, in hours? If the contract says “without undue delay,” ask them to put a number in it.
7. Has this product been through an independent security assessment, and can I see the report or a summary? Not a badge on the marketing site — the report.
8. What happens if the model returns something harmful or wrong about a real person? Every AI vendor has now thought about this. The ones worth buying from can describe the process, not just the disclaimer.
What happens when you leave
9. On termination, what is deleted, when, and how do I get proof? Ask for the artefact: a deletion certificate, a confirmation, something you can file.
10. Can I export everything in a format that is useful somewhere else? A CSV of records is table stakes. Ask whether the derived material — labels, embeddings, configurations you built — comes with you or stays with them.
Why the awkwardness is the point
If you work in education, the stakes are not abstract: the data is about people who did not choose the vendor and often cannot opt out. The same is true of patients, employees, and anyone whose file ends up in a system they never evaluated. The person who asks question six is doing the work of protecting all of them.
And the questions are cheap. Ten of them, sent by email before a renewal, cost you an afternoon and reset the whole relationship — because a vendor who knows you read the terms behaves differently for the rest of the contract.
One thing
Pick a tool you are already mid-conversation about. Send three of these — I would send 1, 6, and 9 — and see how long the reply takes. The latency is the finding.
Working on a procurement that turned interesting? Hit reply. I read everything.