Translating Documents Locally With Open Models

A 40-page supplier contract runs to roughly 20,000 words. At about 1.33 tokens per English word, that is around 26,600 tokens. One of the best-known open translation models, Meta's NLLB-200, was trained on inputs under 512 tokens, so the contract has to be cut into at least 52 pieces before the model sees any of it, and in practice into a few hundred sentence or paragraph segments. To translate documents locally with open models, that segmenting step, the license on the model, and putting the translated text back into the original layout matter as much as the model itself.

Getting text out of scanned pages is a separate step that comes before translation, covered in on-premise OCR and document extraction.

Two kinds of model: dedicated translators and general LLMs

Dedicated translation models take a sentence in one language and return it in another. They are usually small, fast and predictable, and they do not wander off into commentary. Their weakness is context: each segment is translated mostly on its own, so a pronoun or a defined term may come out differently in paragraph 3 and paragraph 30.

General instruction-following LLMs translate because they were trained on multilingual text. They can take a whole section at once, follow instructions such as "keep these terms untranslated", and match a register. Their weakness is the opposite: they sometimes add, drop or "improve" content, and they need a GPU with more memory.

A newer middle group is general models fine-tuned only for translation, such as Google's TranslateGemma, which comes in 4B, 12B and 27B sizes fine-tuned from Gemma 3, with its technical report published in January 2026.

Current options and what their licenses allow

License terms decide more shortlists than quality scores do. The table below reflects the model cards and repositories as read in October 2026; check them again before deploying, because terms change.

Model or tool	Type	Languages	License	Commercial use
NLLB-200 (Meta)	Dedicated	About 200	CC-BY-NC 4.0	No, non-commercial only
MADLAD-400 MT (Google)	Dedicated	400+ (204 evaluated)	Apache 2.0	Yes
Opus-MT (Helsinki-NLP)	Dedicated, one model per pair	Many pairs	Models CC-BY 4.0, code MIT	Yes, with attribution
TranslateGemma (Google)	LLM tuned for translation	55	Gemma Terms of Use	Allowed within the terms and prohibited-use policy
TowerInstruct 7B v0.2 (Unbabel)	LLM tuned for translation	10	CC-BY-NC 4.0	No
LibreTranslate	Self-hosted server on Argos Translate	Depends on installed packages	AGPL-3.0	Yes, with AGPL obligations if you modify and offer it over a network

A few details from those pages are worth knowing before you commit:

• NLLB-200 is popular in tutorials, but its model card says it is a research release, "not released for production deployment", and not intended for whole documents or for medical and legal text. The non-commercial license alone rules it out for most companies.
• MADLAD-400 takes the target language as a prefix token, for example <2pt> in front of the text for Portuguese. Its card says it was trained on general web data and has not been assessed for production use.
• TranslateGemma expects source and target language codes in its chat template and lists a total input context of 2K tokens, so it also needs segmenting, just in larger pieces than NLLB. The Gemma terms say Google claims no rights in the outputs, and they require passing the use restrictions on to anyone you redistribute the model to.
• Opus-MT models are separate per language pair (English to German is one model, German to English another).

The license questions here are the same ones covered in open-source LLM licenses for commercial use; a non-commercial clause is commonly read to cover internal business use as well as resale, so treat it as a stop sign unless counsel says otherwise. This is general information, not legal advice.

Keeping the layout of DOCX and PDF files

DOCX and other office formats

A DOCX file is a zip of XML. Text lives in paragraphs, and each paragraph is split into "runs" that carry formatting: a bold word is its own run. The usual approach is:

1. Walk the paragraphs, tables, headers, footers and footnotes.
2. Send each paragraph's text to the model as one segment.
3. Write the translation back into the paragraph, keeping its style.

The hard part is step 3 when a paragraph has mixed formatting. Word order changes between languages, so a bold word in position two of an English sentence may belong in position seven of the German one. Simple tools put all the translated text into the first run and lose the inline bold or italics. Better tools mark runs with tags before translation (<b1>net 30 days</b1>) and ask the model to keep the tags, which works well with LLMs and less reliably with small dedicated models.

The argos-translate-files library used by LibreTranslate handles .txt, .odt, .odp, .docx, .pptx, .epub, .html, .srt and .pdf, which covers most office paperwork without custom code.

PDF

PDF stores positioned glyphs, not paragraphs. Translating in place means extracting text blocks, translating them, and drawing new text into the same boxes. German and French typically run longer than English, so text overflows boxes, and fonts may lack accented characters. Where a source DOCX exists, translate that instead and export a new PDF. Where only the PDF exists, accept a clean reflowed DOCX as the output rather than a pixel-perfect copy.

Glossaries for company terms

General models know general language. They do not know that your "Service Credit" is a defined contract term, that a product name must stay in English, or that your industry uses one specific word for "bearing housing" in Spanish.

A glossary is a two-column list: source term, required target term (or "do not translate"). How it is applied depends on the model type:

• LLMs: put the glossary entries that appear in the segment into the prompt, with an instruction to use them exactly. Include only matching entries; pasting a 2,000-term glossary into every request wastes context and confuses the model.
• Dedicated models: they do not take instructions. Protect "do not translate" terms by replacing them with placeholders (TERM_017) before translation and restoring them afterward. For required translations, run a check after translation.

The check is the part that makes a glossary enforceable. A simple script looks at each segment: if the source contains a glossary term, does the target contain the required translation? Segments that fail go to a list for the reviewer. With 300 segments and, say, 40 that contain glossary terms, the reviewer checks 40 flagged lines first instead of reading for terminology across the whole document.

Review by a fluent reader

Machine translation output reads fluently even when it is wrong, which is exactly why it needs a person who reads both languages. The failure types worth teaching a reviewer to look for:

• Omissions: a clause, a negation or a number quietly dropped. Both kinds of model do it, and long segments make it more likely.
• Number and date formats: 1,500.00 versus 1.500,00, and 03/04 meaning March or April.
• Inconsistent terms: the same defined term translated two ways.
• Hallucinated additions: LLMs occasionally append an explanation or a polite closing line.

A practical way to plan review effort is to tier documents by risk:

Document type	Suggested handling
Internal reading, gist only (a supplier email, a foreign news item)	Machine output, no review
Internal working documents	Spot check by a fluent colleague
Anything signed, published or sent to a customer	Full review by a fluent reader, glossary check passed
Legal filings, regulated labeling, certified translation	Professional translator; machine output as a draft at most

The NLLB card says plainly that its output cannot serve as a certified translation, and the same holds for every model here. Certification is a property of the human translator, not the software.

For review time, plug in your own numbers: if a reviewer comfortably checks N words an hour against the source, a 20,000-word contract takes 20,000 / N hours. At 1,000 words an hour that is 20 hours; at 2,500 it is 8. That figure, not GPU time, is usually the real cost of a translation workflow.

Language pairs that are weak

Quality is uneven, and the unevenness is predictable:

• High-resource pairs with English on one side (English with Spanish, French, German, Portuguese, Chinese, Japanese) are the strongest across all the models above.
• Pairs without English, such as Polish to Portuguese, are often weaker, because many systems were trained mostly on English-centered data.
• Low-resource languages are where the large language counts in model cards mislead. MADLAD-400 supports over 400 languages but was evaluated on 204 of them; support means the model accepts the language, not that the output is usable.
• Specialized domains in any language (legal, medical, engineering) are weaker than general text, as several of the model cards above state.

Before choosing a model, build a small test: 30 to 50 real segments from your own documents in each language pair you need, translated by each candidate, and read blind by a fluent colleague. The same selection method used in how to pick an open-source LLM applies here, with the language pair as the first filter. A model that wins on English to German may lose on English to Vietnamese, and only your own text shows that.

Frequently asked questions

What hardware does local translation need?

Dedicated models such as Opus-MT or small MADLAD and NLLB variants run acceptably on a CPU for occasional use, and quickly on a modest GPU. Translation-tuned LLMs in the 4B to 12B range want a GPU with enough memory for the quantized weights plus context.

Can one model handle every language the company needs?

Sometimes, but mixing is common and reasonable. A company might use a translation-tuned LLM for its main European pairs, where it follows glossary instructions, and a broad-coverage dedicated model for occasional rare languages.

Is the translated document safe to keep on the same server?

Translations carry the same sensitivity as the source, so they belong under the same access rules and retention schedule. Logs of segments sent to the model are copies of the document too, and are often forgotten. Decide in advance whether the translation server keeps those logs and for how long.

Should translations ever go to a cloud service instead?

That depends on the data class, not on the tool. Public marketing text carries little risk; contracts, personnel files and customer records are the reason to keep translation on your own hardware in the first place. Many companies write that rule into their data sovereignty policy so staff do not decide case by case.