Local LLM vs Cloud API Cost - Finding the Break-Even Point

A 20-person office that asks an AI model 300 document questions a day, at about 5,000 tokens per question, uses roughly 33 million tokens a month. At a mid-tier cloud API rate that is about $119 a month. A local LLM server that costs $9,000, written off over three years and drawing an average 300 watts, costs about $282 a month whether it answers one question or ten thousand. On pure local LLM vs cloud API cost, the API wins that case by more than two to one, and the gap gets wider, not narrower, against the cheapest API tiers. This article shows where those numbers come from, so you can run the same comparison with your own.

The two formulas

Both sides reduce to a monthly figure. The cloud side scales with use. The local side is mostly fixed.

API cost per month = (input tokens / 1,000,000) x input price
                     + (output tokens / 1,000,000) x output price

Local cost per month = hardware price / months of amortization
                     + average kW x 730 hours x electricity price per kWh

The break-even volume is the point where the two lines cross:

blended API price per million tokens
    = (input share x input price) + (output share x output price)

break-even tokens per month (millions)
    = local cost per month / blended API price per million tokens

Below that volume the API is cheaper. Above it, the local machine is cheaper, provided it can actually process that many tokens, which is a separate check covered further down.

This is deliberately the comparison only. Admin hours, spares, support contracts and the refresh cycle belong in a full budget, and the total cost of owning an on-premise AI server walks through those line items. Adding them only pushes the break-even point higher.

Step 1 - Measure your tokens, input and output separately

Cloud providers charge different rates for the text you send (input) and the text the model writes back (output). Output is usually several times more expensive per token, so the mix matters.

Document question answering is input-heavy. A request might carry 4,000 tokens of retrieved passages and instructions and get back a 1,000-token answer. That is an 80/20 split. A drafting tool that turns a short brief into a long letter runs the other way.

To get your own numbers:

• If you already use an API, the provider's usage dashboard reports input and output tokens per month. Use the last three months, not the best one.
• If you are planning, count requests per working day, estimate tokens per request (roughly 750 English words per 1,000 tokens is a common rule of thumb, but tokenizers differ), and multiply by working days. This article uses 22.

For the opening example: 300 requests x 22 days = 6,600 requests. At 4,000 input and 1,000 output tokens each, that is 26.4 million input and 6.6 million output tokens, 33 million in total.

Step 2 - Pick the API rate you would actually pay

Use the published rate for the model tier whose quality you would accept, not the flagship. An open model on one workstation is usually compared fairly with a provider's mid-tier or budget models. Two examples from published price pages in October 2026:

Tier	Example	Input per 1M tokens	Output per 1M tokens
Mid-tier	Claude Sonnet 5.5	$2.00	$10.00
Budget	Gemini 2.5 Flash-Lite	$0.10	$0.40

Prices are from the Anthropic pricing page and the Gemini API pricing page. They change often, so check the current figure before you rely on it.

With an 80/20 mix, the blended rate is:

• Mid-tier: (0.8 x $2.00) + (0.2 x $10.00) = $1.60 + $2.00 = $3.60 per million tokens
• Budget: (0.8 x $0.10) + (0.2 x $0.40) = $0.08 + $0.08 = $0.16 per million tokens

Two discounts can cut the API side further. Both providers above list a batch mode at 50% off standard rates for work that can wait for an asynchronous result. Prompt caching also bills repeated input, such as a long fixed instruction block, at a fraction of the normal input price. If an overnight job qualifies for batch pricing, halve its rate before comparing.

Step 3 - Turn the hardware quote into a monthly number

Take the full quote for the machine: GPU, chassis, power supply, memory, storage. The example here uses $9,000 as an illustrative figure, not a price for any specific build.

Divide by the months you expect to use it before replacement. Three years is a common accounting choice for fast-moving hardware:

$9,000 / 36 months = $250 per month

Stretching it to five years gives $9,000 / 60 = $150 a month. That looks attractive, but a five-year-old inference box may not run the models you want in year four, so be honest about the period.

Step 4 - Add electricity

Electricity is the smallest line, and the arithmetic shows why. Use average draw over the whole month, not the peak figure on the power supply label. A server that sits idle most nights and weekends might average 300 watts across 730 hours in a month:

0.3 kW x 730 h = 219 kWh

The US Energy Information Administration put the average commercial electricity price at 14.53 cents per kWh in July 2026:

219 kWh x $0.1453 = $31.82 per month

Even at the highest state rate in the same table, Hawaii's 43.63 cents, that becomes 219 x $0.4363 = $95.55. Electricity moves the result by tens of dollars; amortization moves it by hundreds. The power and cooling article covers measuring real draw and what a closet needs.

Step 5 - Find the crossing point

Local cost per month: $250 + $31.82 = $281.82.

API tier	Blended rate per 1M	Break-even per month	Requests per month at 5,000 tokens	Requests per working day
Mid-tier	$3.60	78.3 million tokens	about 15,657	about 712
Budget	$0.16	1,761 million tokens	about 352,275	about 16,000

The arithmetic: $281.82 / $3.60 = 78.28 million tokens. 78.28 million / 5,000 = 15,657 requests, and 15,657 / 22 days = 712 a day. For the budget tier, $281.82 / $0.16 = 1,761.4 million tokens, or 352,275 requests, or about 16,000 a day.

Now compare the opening example, 33 million tokens a month:

• Mid-tier API: (26.4 x $2.00) + (6.6 x $10.00) = $52.80 + $66.00 = $118.80
• Budget API: (26.4 x $0.10) + (6.6 x $0.40) = $2.64 + $2.64 = $5.28
• Local server: $281.82

This is why low volume favors the API. The local machine charges its $282 whether it answers 300 questions or none, while the API bill shrinks with use. A business needs more than twice the example office's traffic just to match the mid-tier API, and over 50 times its traffic to match the budget tier.

Step 6 - Check the machine can actually reach that volume

A break-even figure means nothing if the hardware cannot produce that many tokens. Generation speed is the usual ceiling, and it must be measured on your own model and prompts. The method for benchmarking a local LLM explains how.

Suppose a measurement shows 40 output tokens per second for one request at a time. Over 8 working hours on 22 days:

40 x 28,800 s x 22 = 25.3 million output tokens a month

The mid-tier break-even needs 78.3 million tokens with 20% as output, which is 15.7 million output tokens. That fits, at about 62% of serial capacity (15.7 / 25.3).

The budget-tier break-even needs 1,761 million tokens, of which 352 million are output. Running 24 hours a day for a 30.4-day month gives 40 x 86,400 x 30.4 = 105 million output tokens. One request at a time cannot get there. Only batching many requests together on the GPU raises total throughput enough, and how far it helps depends on the runtime and the card. The article on how many users one GPU can serve shows how to measure that.

There is also a marginal cost check worth running. At the same 40 tokens per second and a 600-watt draw under load, one million output tokens takes 1,000,000 / 40 = 25,000 seconds, or 6.94 hours. That is 6.94 x 0.6 = 4.17 kWh, or about $0.61 at 14.53 cents. Electricity alone, unbatched, costs more than the $0.40 per million output tokens that the budget API tier charges. Treat 40 tokens per second and 600 watts as placeholders and replace them with your measurements.

What the formula leaves out

The comparison above is narrow on purpose, and a few things can tilt a real decision either way:

• Quality is not equal. If the local model needs a second pass or more human review, that staff time costs more than any token.
• Utilization is lumpy. Office traffic peaks mid-morning. A server sized for the peak sits idle most of the day, which is already priced into the fixed monthly cost.
• API prices move. The Gemini pricing page already lists scheduled increases for some of its newer models. Rerun the numbers at each budget cycle.
• Cost is not the only reason to run locally. Data that may not leave the building, or contract terms that forbid sending it out, can decide the question before cost is considered.

A practical decision rule: if your measured monthly volume is below half the break-even figure for the API tier you would really use, the API is cheaper and will stay cheaper unless volume grows. Between half and the full figure, a price change or a busier quarter can flip the answer, so the factors in the list above decide it. If it is above the break-even and the throughput check passes, local hardware starts paying for itself, and the full ownership budget is the next document to write.

Frequently asked questions

Does a free open-weight model make the local side free?

No. The weights cost nothing to download, but the hardware, electricity and staff time to run them do. An API rate bundles the model, the hardware and the operations into one per-token price, which is why the comparison has to put hardware and power on the local side.

Should the API comparison use the flagship model?

Only if a flagship-quality answer is what the task requires. Most local models that fit on one or two cards are a fairer match for a provider's mid-tier or budget models. Comparing a local model with a flagship price inflates the API side and makes local hardware look cheaper than it is.

How does renting a cloud GPU change the math?

A rented GPU swaps the amortization line for an hourly rate, so the local formula becomes hours used times the hourly price, plus storage. It suits bursty batch work better than an always-on office service, because an always-on rental bills every hour whether or not anyone asks a question.

What if the server also does other work?

Split the fixed cost by share of use. If the same box runs image recognition half the time, charge the language-model comparison with half of the $281.82, and the break-even volume halves with it. Be careful not to count the same hardware in two business cases at full price.