What Is Private AI? A Plain-English Definition

Private AI is an AI system in which the model does its computing on machines the organization controls, and the organization, not a vendor, holds the keys to those machines and to the data on them. The definition says nothing about which model is used or how good it is. It is a statement about location and control.

Marketing stretches the term, so anyone asking what is private AI can settle most cases with two questions.

The two questions behind the definition

Where does inference run?

Inference is the moment a model reads a prompt and produces an answer, and the prompt has to sit in memory as readable text while it does. Encryption protects it on the wire and on disk, but during inference it is decrypted on whatever machine does the work. Confidential-computing hardware can shield that memory from the host's operators, which narrows the exposure, but the prompt is still processed on someone else's machine.

So the first question is physical. Whose machine is it, and in which building? If the answer is "a server in the office closet" or "a rack the company leases and administers", inference is private. If the answer is "the vendor's fleet, somewhere", it is not, whatever the contract says about what happens afterward.

Who holds the keys?

Keys here means three practical things:

Administrative access. Who can log in to the machine as an administrator, read its memory, or attach a debugger?
Encryption keys. Who can decrypt the disks where prompts, documents and logs are stored?
The off switch for logging. Who decides what is recorded, how long it is kept, and who may read it?

In a private setup the answer to all three is the organization itself. In a cloud service the answer to at least one is the provider, and the customer's protection is a promise in the terms of service.

Three shapes private AI takes

Three arrangements meet the definition. They differ in who owns the hardware and how the system connects to the outside world.

Shape	Who owns the hardware	Network connection	Who can technically read a prompt
On-premise	The organization	Office network, usually with internet access	The organization's own administrators
Private cloud tenancy	A cloud or hosting provider	Private network inside the provider's facility	The organization's administrators, plus the provider at the hardware and hypervisor level
Air-gapped	The organization	None, data moves by hand	The organization's own administrators with physical access

On-premise

The model runs on a server the organization owns, in a room it controls. Staff reach it over the office network the same way they reach a file server. Prompts and documents never cross the building's internet connection because there is no reason for them to. The machine itself may still be online to fetch software updates and new model files.

Private cloud tenancy

Here the organization rents computing power, typically a GPU virtual machine or a dedicated physical server, inside a provider's data center. It installs and runs its own model there, on a private network segment, with disks encrypted under keys it manages. No model vendor is involved. The provider supplies electricity, cooling and hardware, and the organization operates everything above that.

This counts as private with one stated caveat. The provider owns the physical machine, so the provider's position has to be trusted or constrained by contract. Jurisdiction comes with the provider too. Under US law, for example, 18 U.S.C. 2713 says a provider subject to the Stored Communications Act must meet that law's preservation and disclosure obligations for data in its possession, custody or control, regardless of whether the data sits inside or outside the United States (statute text). How that applies to a given arrangement is a legal question, and this article is general information, not legal advice.

A close relative sits just outside the definition, the managed model service offered inside a cloud account. Microsoft's documentation for models sold through Azure states that prompts and completions are not available to OpenAI and are not used to train generative AI foundation models without the customer's permission, and that stored data can be encrypted with a customer-managed key (Microsoft's data privacy page). Those are strong commitments, but inference still runs on the provider's service, operated by the provider's staff, with abuse monitoring that the same page says can include human review of flagged prompts by authorized Microsoft employees unless a customer is approved for modified abuse monitoring. That is a well-governed cloud service, and it is better described that way than as private AI.

Air-gapped

An air-gapped system has no network path to the outside at all. The NIST glossary defines an air gap as an interface between two systems where they "are not connected physically" and any logical connection "is not automated", meaning data crosses only manually, under human control (NIST glossary). In practice, model files and updates arrive on removable media that someone carries into the room.

Air-gapping is on-premise taken to its limit. It removes whole classes of remote attack and makes "the data did not leave" something that can be demonstrated by the absence of a cable. The cost is convenience, since every update becomes a manual task. The air-gapped AI page covers when that trade is justified.

Why cloud AI with privacy settings is a different thing

Cloud AI privacy settings are worth using. As of October 2026, OpenAI's API documentation says data sent to the API is not used to train its models by default, that abuse monitoring logs are kept for up to 30 days unless longer retention is required by law, and that Zero Data Retention is available only with prior approval (OpenAI's data controls guide).

Read those three facts against the two questions. Inference runs on the provider's machines. The provider holds the administrative access, the disk keys and the logging switch, and has agreed to use them in limited ways. A setting changes what the provider does with the data after receiving it. It does not change the fact that the provider received it.

The difference shows up when something goes wrong: a provider can change its terms, be ordered by a court to retain or produce data, suffer a breach, or simply misconfigure a log. In each case a customer who relied on settings finds out afterward. A customer who never sent the data has nothing to find out. How to read a provider's terms line by line is its own subject, covered in the guide to whether cloud AI providers train on business data.

None of this makes cloud AI wrong. For public or low-sensitivity text it is often the sensible choice, and the walk-through on choosing between on-premise and cloud AI weighs the two. The point is only that "private" should describe an architecture, not a preference panel.

Tracing one prompt

A simple way to classify any product is to follow a single request and count the organizations whose systems hold it in readable form.

Take a 30-person firm where someone asks an assistant to summarize a client contract.

Cloud chatbot, default settings. The contract is readable on the firm's laptop, on the provider's inference servers, in the provider's conversation history store, and in its abuse monitoring log. That is 1 firm + 1 provider = 2 organizations at minimum, more if the provider runs on another company's cloud.
Cloud API with zero retention approved. The history store and the log drop out, but the inference servers remain. Still 2 organizations, for a shorter time.
Private cloud tenancy. The contract is readable only inside a virtual machine the firm administers. 1 organization can read it in normal operation, and 1 more has physical custody of the hardware.
On-premise or air-gapped. 1 organization, full stop.

If a vendor's answer to "which machines hold the prompt in plain text, and who administers them" includes any name other than the customer's, the product is a cloud service with privacy features. That may be acceptable, as long as it is a decision made knowingly.

What the label does not promise

Private describes where data goes. It is silent on three things buyers sometimes assume come with it.

Security. A local model server with an open port and no password is private in the definitional sense and exposed in every practical one. Control of the keys means the owner also carries the duty to use them well. The basics are in the guide to securing a local LLM server.

Compliance. Keeping data in the building removes a third party from the picture, which simplifies many obligations. It does not create access controls, audit trails or retention rules. Those still have to be built.

Model quality. A private system usually runs open-weight models sized to the hardware on hand. Whether such a model is good enough for a task is a separate question, answered by testing it on real examples from that task.

A workable rule for any conversation about the term is to drop the adjective and ask for the diagram. Where is the model, who can log in to that machine, and what leaves the room.

Frequently asked questions

Is private AI the same thing as a local LLM?

A local LLM is one common way to build private AI, but the terms are not identical. Private AI also covers image recognition, speech-to-text and data pipelines that involve no language model. And a language model on a rented, self-administered cloud server is private without being local to the office.

Can a private AI system still connect to the internet?

Yes, unless it is air-gapped. What matters is direction and content: updates and public web lookups may come in, while prompts and internal documents are not sent out to a third party for processing.

Does private AI mean the model was trained on company data?

No. Most private deployments run a general-purpose open-weight model exactly as published and supply company documents at question time through retrieval. Training or fine-tuning on internal data is an optional extra step, usually considered only after retrieval has been tried.

Is an enterprise chatbot subscription private AI?

By the definition used here, no. Enterprise plans typically add contractual limits on training and retention, plus administrative controls, and those matter. Inference still happens on the vendor's infrastructure under the vendor's administrative access, so it is a cloud service with stronger terms.

Who is responsible when a private AI system fails or leaks?

The organization that runs it. With no vendor operating the service there is also no vendor to share the blame, which is why private setups need a named owner for patching, backups and access reviews before staff begin to rely on them.