The Hidden Math of Cloud AI

A practical way to compare usage-based cloud AI with fixed-cost on-premise infrastructure for steady, sensitive workloads.

Cloud AI pricing feels cheap because it's small and it's per-use. A fraction of a cent per call. You never approve a purchase; you just get a bill.

For a spiky, unpredictable workload - a few thousand calls one month, near zero the next - that's the right model. Pay for what you use. Don't buy hardware to sit idle.

The trap is what happens when the workload stops being spiky.

Most business AI isn't a research toy. It's the same job, all day: classify every incoming document, read every product photo, summarize every support thread, run every invoice. Steady, predictable, high-volume. And per-token pricing on a steady, high-volume workload is the most expensive way you can possibly run it. You're renting, forever, something you use at full tilt every single day.

Where the line crosses

Two lines. A cloud API costs roughly nothing to start, then a monthly amount that scales with every call and never stops. An on-prem setup costs a real number up front - a GPU box and the work to install it - then almost nothing per month, because there's no meter running.

One line starts at zero and climbs forever. One starts high and goes flat. They cross.

Do the arithmetic yourself

Here's a made-up workload so you can copy the method. Say you classify 400,000 documents a month - about 3,000 tokens in and 500 out per document. At commodity API rates, that's roughly $6,600 a month. Call it $79,000 a year, rising with volume, forever.

The on-prem version: a GPU box and the work to install and integrate it, call it $30,000 once. Power and maintenance, roughly $250 a month.

• Monthly saving: $6,350
• Payback: month five
• Three-year total: $39,000 on-prem vs. $237,600 cloud - about 6× cheaper

Now the part most comparisons miss. That box is not running at capacity at 400,000 documents a month. It's running at about a third. Fill it - same hardware, same $30,000, same $250 a month, three times the throughput - and the cloud line triples while the on-prem line doesn't move. Three years: $39,000 against $712,800. Roughly 18× cheaper.

That's the real lever. The multiple isn't a property of on-prem. It's a property of how full your box is. A half-idle server is a mediocre deal. A saturated one is the cheapest compute you will ever buy.

Your numbers will differ. The method won't. Draw the two lines before you sign up for a meter that never stops.

The bill isn't the only thing you're paying

There's a second cost on the cloud line that never shows up as a number: every one of those calls sent your data offsite.

For a lot of companies that's fine. For a hospital, a law firm, a bank, or a manufacturer sitting on trade secrets, it's the whole problem. And the standard answer - "it's encrypted, we signed a DPA" - solves less than people think.

You can't DPA your way out of physics

TLS encrypts data in transit. A Data Processing Agreement is a contract about what a vendor promises not to do. Neither one changes the fact that your regulated data left your building, crossed a network boundary, ran on hardware you don't control, in a jurisdiction that may not be yours, and passed through a third party's logs.

On-prem removes the problem instead of papering over it. When the model runs on your own server, your data never crosses a boundary. There's no subprocessor to audit, no cross-border transfer to justify, no "what happens to the prompt after it's processed" question - because the prompt never left. You're both controller and processor. That's not a stronger contract. It's the absence of the thing the contract was trying to manage.

What that looks like in practice

We built ClaimLens to make the point concrete: a tool that turns car-accident paperwork into a review-ready demand package. Police reports. Medical records and itemized bills. Insurance declarations. Injury photos.

Read that list again. There isn't a single item on it you'd casually POST to a third-party endpoint. Medical records alone put you inside HIPAA before you've written a line of code.

ClaimLens runs entirely on local models. It reads the damage photos with a vision model, assembles a timeline, flags Florida's 14-day PIP gate, checks the package for missing documents, and links every conclusion back to the source page. No call leaves the machine.

It's a public demo on synthetic data - no real people, no real claims - so you can look at the whole thing. The architecture is the argument: the most sensitive document classes in American civil practice, processed end to end, with nothing crossing a boundary.

Where this argument fails

Four objections worth taking seriously, because they're the ones a good CFO raises.

Your box doesn't improve. Frontier models do. True. The hardware you buy today is the hardware you have in two years, while the hosted models keep getting better. The mitigation is that open weights have improved fast and you can swap them on the same box - but if your job genuinely needs frontier-grade reasoning, rent it. Run the steady 95% locally and route the hard remainder out, provided the sensitive class stays home.

Hardware is a bet that the workload persists. Also true. Buying a box for a pipeline you've run for three months is speculation. Buy for workloads that have already proven they're steady.

Capex needs a signature. Opex doesn't. This is the honest reason cloud spreads inside companies - not that it's cheaper, but that nobody has to approve it. A $30,000 purchase order goes to a committee. A $6,600 monthly bill goes to accounts payable. Recognising that as an accounting artifact rather than an economic fact is most of the battle.

Who fixes it at 3am? Fair, and it's the objection we take most seriously. The answer is no proprietary lock-in: standard hardware, open weights, documented runbooks. If we vanish, someone else picks it up. An on-prem system that only its builder can operate is a worse dependency than the cloud, not a better one.

The short version

For a spiky workload, rent it in the cloud. For a steady, high-volume workload, per-token pricing is the most expensive option there is, and it ships your data offsite on top of the bill. A one-time on-prem install starts higher and then runs near-free - typically paying back within months and running several times cheaper over three years, rising toward 18× as you fill the hardware - and your data never leaves the building.

The question isn't "cloud or on-prem." It's "is my workload steady, and is my data sensitive." Two yeses, and the closet wins.

Why we can say this with numbers

We run on-prem AI in production, not in slides. Across the systems we operate - product-photo classification, news monitoring, document search, pipeline automation - that's roughly 8 live applications, 12-15 data pipelines, and a few million records under management, on hardware the client owns. When something breaks at 3am, the person who built it fixes it. No account manager, no handoff, no per-token meter.

If your AI workload is steady and your data is sensitive, the server in your closet isn't the nostalgic option. It's the cheaper one and the compliant one, at the same time.