Blog
Architecture, numbers, and lessons from production private-AI systems - image recognition, local LLMs, ETL automation. Written by Alex.
Open translation models can run on an office server, but licenses, short input limits, file layout and company terminology decide whether the output is usable. Here is how the pieces fit.
A Mac can hold a 70B model in memory for the price of one high-end GPU workstation. What it gives up is speed on long prompts and on many users at once, and that decides where it fits.
A chatbot can answer routine handbook questions on a company's own server. The hard parts are keeping personal records out, handing sensitive topics to a person and serving the right policy version.
The graphics card gets the attention, but the case, power, noise, storage, network spot and remote access decide whether an office AI server runs quietly for years or trips a breaker in week one.
A local model can write SQL from a plain question if it sees the schema. The hard parts are keeping it read-only, checking the query before it runs, and catching answers that look right but are not.
The VRAM a language model needs comes down to weights plus KV cache plus overhead. Here is the formula, three worked examples and what happens when a model spills into system RAM.
Once a meeting is transcribed locally, a local model can draft the summary too. The hard parts are fitting the transcript in the window, naming the right owners and deciding what to delete.
A three-step method for choosing a GPU to run a language model in-house, with the arithmetic for memory and speed and a plain comparison of consumer, workstation and datacenter cards.
A local model can sort a shared inbox into support, sales, invoices and junk, and draft replies for staff to check. The work is in the label set, the mailbox permissions and measuring misroutes.
A design guide for overnight batch LLM processing on local hardware, from the job table and retry rules to schema validation and the arithmetic that says whether the run fits before morning.
A local model can split a contract into clauses, compare each one to the firm's playbook and flag deviations. It earns trust only when it quotes the clause it relied on and leaves every edit to a person.
A line-by-line, three-year budget for an on-premise AI server, with a worked example where the hardware invoice is only about 40 percent of the cost to own.