You're evaluating a new line-of-business platform (a CRM, a customer service tool, a document management system) and the vendor proudly demos an AI assistant that reads, summarizes, and answers questions from your internal files. The demo is convincing. But one question should come before price: if this assistant reads your documents, where do they physically go, do they train a model, and who, exactly, holds the subscription to the AI underneath it?
This isn't a side technical detail. It's the question that determines whether your future vendor can contractually guarantee that your contracts, your supplier correspondence, and your general operations data stay exclusively yours. Most procurement scorecards evaluate which AI model a vendor uses (GPT, Claude, Gemini) when the real risk variable is somewhere else entirely: the document's actual route, not the model's name.
This is written for the IT director, the compliance lead, and the exec sponsoring a software purchase during the scoping stage. It lays out a five-rung spectrum of control, a plain reading of what AI vendors' usage terms actually promise, and a requirement clause you can paste straight into your RFP. If you're scoping a broader AI integration project, our AI transformation practice starts from exactly this data-governance question before it ever gets to use cases.
The real question isn't which model: it's where the document goes
A software buyer almost always asks: "which AI model do you use?" That's the wrong first question. Whether the answer is GPT-4, Claude, or a proprietary model tells you nothing about what actually happens once a document leaves your network.
The right sequence of questions is: does the document go to a service shared with the vendor's other customers? Does it pass through a third-party AI subprocessor you've never heard of? Is there a contractual record of what that subprocessor is allowed to do with it? A vendor can genuinely use "the best model on the market" while routing your documents through three layers of subprocessing with no clause forbidding anything. The model is visible in the demo; the subprocessing chain never is, unless you ask for it directly.
The five rungs of control, from consumer service to your own infrastructure
Not all AI integrations are equal. Here is a five-level spectrum, from least to most controlled, with what you gain, what you give up, and a rough order of cost at each rung.
Rung 1: consumer-grade service. The vendor (or worse, one of your own employees) runs professional documents through a free or paid consumer AI chat interface. Zero visible cost, maximum risk: these interfaces are typically governed by consumer terms of service, not an enterprise agreement. Most consumer tiers do not enable a no-training guarantee by default.
Rung 2: API with a contractual no-training clause. The software vendor calls a professional API (OpenAI API, Anthropic API, Azure OpenAI Service) under a business agreement that explicitly excludes your data from model training. This is the first genuinely defensible tier for general-operations or supplier-relationship data. Cost to the vendor: roughly a few cents per request at volume, folded into your software subscription.
Rung 3: dedicated region / data residency. Processing is contractually confined to a specific geography (the EU, for instance), with data-residency guarantees. Useful if you have cross-border transfer obligations to meet. Typical premium: 15-30% over a standard multi-region deployment, based on published cloud pricing tiers.
Rung 4: private environment deployment. A dedicated instance isolated from the vendor's other customers (a dedicated VPC, a private Bedrock or Azure OpenAI endpoint). You get real network isolation. Typical cost: roughly 2,000-8,000 EUR (about $2,200-$8,700) per month depending on volume, on top of the software license itself.
Rung 5: open-weight model on your own infrastructure. An open model (Llama, Mistral) runs on your own servers or private cloud. No data ever leaves your perimeter. This is the maximum-control tier, and also the most expensive to operate, more on that below.
Every rung of control gained costs a rung of flexibility lost: the newest, most capable models ship first to consumer services and APIs, rarely first to a self-hosted release.
What "no training on your data" actually says in the fine print
The major AI vendors have documented, verifiable policies on this specific point. OpenAI's API terms state that data submitted through the API is not used to train its models unless you explicitly opt in, a different policy from the consumer ChatGPT interface, where conversation history can be used for training unless manually disabled. Anthropic applies the same logic to its commercial offering: conversations through the API or enterprise plans are not used to train models absent an explicit separate agreement.
What these policies don't cover: the software vendor that buys API access can still choose to keep a copy of your documents in its own logs, its own vector search index, or its own debugging traces, entirely independent of the underlying AI vendor's policy. OpenAI's or Anthropic's no-training clause protects the relationship between the software vendor and its own AI subprocessor; it says nothing about what the software vendor does with the copy it keeps for itself.
Who holds the subscription, and why it changes everything
Here is the detail most vendor scorecards miss: contractually, who is the customer of the AI API? Two scenarios, with opposite consequences.
Scenario A: you subscribe directly. Your organization opens its own enterprise account with the AI vendor, with its own contract, its own no-training clause, and its own data processing agreement. The software simply calls into your account. You have a direct, enforceable contractual relationship with the AI vendor.
Scenario B: the vendor subscribes on its own behalf. The software vendor holds its own API account, pooled across all of its customers, and you're just an end user of its platform. In that case, the vendor's contractual guarantees with its AI provider don't automatically extend to you; you depend entirely on what the vendor negotiated, and on whatever it chose to pass through into your own contract with it.
Scenario B is by far the more common setup for standard line-of-business software. That isn't necessarily a problem, provided the contract between you and the vendor explicitly carries, via a "flow-down" clause, the same guarantees the vendor negotiated with its own AI provider. Without that flow-down clause, the vendor's commitment to its AI provider creates no direct right for you at all.
What self-hosting actually means, and what it costs in practice
"Self-hosted" is often pitched as the ultimate privacy answer, which is true in principle and misleading on budget. Running a mid-size open model (in the range of 70 billion parameters) in production with reasonable uptime generally requires several current-generation GPUs, meaning either a hardware investment or a managed cloud cost on the order of 3,000-10,000 EUR (roughly $3,300-$11,000) per month depending on request volume and redundancy, plus in-house or outsourced MLOps capacity for maintenance, model updates, and security monitoring.
For an SME or mid-market company processing a few thousand documents a month, that cost is rarely justified compared to a private-environment deployment (rung 4), which already delivers most of the isolation you're after for a fraction of the price. Full self-hosting earns its keep at high, sustained volume, or under a strict regulatory constraint that forbids any transit outside your own infrastructure, still uncommon for general-operations or supplier-relationship data, but relevant as soon as the data category is classified highly sensitive under your own internal policy.
Drafting the AI confidentiality requirement for your RFP
Here is a clause you can adapt directly for the "data security and confidentiality" section of your RFP:
"The bidder shall describe precisely, for each AI-powered feature embedded in its solution: (1) the underlying AI provider(s) and which rung of the following control spectrum applies: consumer service, API with no-training clause, dedicated region, private environment, self-hosted infrastructure; (2) whether the subscription to the underlying AI provider is held by the bidder or by the client, and if the former, the contractual flow-down clause carrying no-training and confidentiality guarantees into this agreement; (3) the retention period for submitted documents, including in technical logs and the bidder's internal search indexes; (4) the geographic location of processing and storage."
Insist on getting this in writing, attached to the contract, not just stated reassuringly in the sales pitch. A verbal answer that doesn't survive being put on paper has no contractual value.
The consistency test: one vague answer disqualifies the vendor
Ask these three questions directly, in this order, and require a precise written answer to each: where are our documents physically processed and stored? Does our data, in any form, train a model (yours or your AI subprocessor's)? Who, contractually, holds the subscription to the underlying AI?
A serious vendor answers all three precisely, usually by pointing to security documentation it has already written. A vendor that deflects on even one of the three (for example "our platform is secure" or "we use industry-leading standards," without ever answering the where, the training, or the who) is sending a clear signal: either it doesn't know the answer itself, or it would rather not say. Either way, one vague answer should be enough to disqualify the vendor, regardless of how polished the product demo was. On this specific question, there is no acceptable middle ground between a precise answer and an evasive one.
If you'd rather have a technical third party stress-test your vendor shortlist before signature, our team supports this scoping work as part of our business applications engagements, and our dedicated enterprise RAG offer details the reference architecture for an assistant that reads your documents without ever letting them leave your perimeter. On the adjacent problem of controlling AI use by your own employees (rather than by software you're buying), our article on enterprise AI governance covers the other half of the picture.
FAQ
The vendor tells me their AI "doesn't store anything," should I take that at face value?
No, ask for the exact written wording and the retention period for technical logs. "Doesn't store anything" in a sales demo can coexist with debugging logs kept for months. Get the precise number, not the reassuring adjective.
Does an AI provider's no-training clause (OpenAI, Anthropic) automatically protect me?
Only if you're the direct subscriber to that AI provider's contract, or if your contract with your software vendor explicitly carries that guarantee through a flow-down clause. Without that clause, the protection exists between the vendor and its AI provider, not between you and the vendor.
Is private-environment deployment (rung 4) enough for most SMEs?
In most cases where the data involved is supplier relationships, general operations, or routine internal documents, yes. Full self-hosting earns its cost mainly at high volume or under a strict regulatory constraint, not as a default choice.
How do I verify a vendor isn't overstating its written answer?
Cross-check it against its public security documentation (often a "trust center" or security page) and ask whether its most recent security audit or certification specifically covers AI-based processing, not just standard data hosting.
Should I disqualify any vendor whose staff uses a consumer AI service internally?
Not necessarily for internal uses unrelated to your documents, but yes for any direct processing of your business documents. Rung 1 (consumer service) should never be the answer to "where does the document I'm handing you actually go."