How to Pick an Open-Source LLM for Business Use

There is no single best open source LLM for business, and any list that names one is answering a question you did not ask. A model that tops a public leaderboard on math puzzles can still mangle your invoice summaries or answer Spanish customer emails in English. What works is a short funnel: remove the models whose license you cannot accept, remove the ones that do not fit your hardware, remove the ones that do not handle your languages, then put three survivors through 30 examples taken from your own work. Thirty examples is enough to expose a model that clearly fails your task, and scoring them takes an afternoon.

The model families named below are examples as of October 2026, checked against their model cards and license files on that date. New releases arrive every few months, so treat the names as illustrations of the method, not as a recommendation that will still hold next year.

Step 1: Write the job down before looking at models

Selection goes wrong when it starts from the model list. Start from one sentence that describes the work:

• Input: what goes in (a 3-page contract, a support email, a scanned form already turned into text).
• Output: what must come out (a 5-line summary, a category label, a JSON object with six fields, a draft reply).
• Languages: every language that appears in real inputs, including the minority ones.
• Volume and wait: requests per day and how long a person will wait for an answer.
• Failure cost: what happens when the answer is wrong, and whether a person checks every output.

"Extract supplier name, invoice number, total and due date from English and German invoices, about 400 a day, reviewed by a clerk" leads to a very different shortlist than "draft first replies to customer complaints in five languages". The first rewards strict format-following and can run on a small model. The second needs fluent writing in every language and probably a larger one.

Step 2: Filter by license

Open weights do not mean the same terms. In October 2026 the licenses on popular model families fall into roughly three groups, and the differences matter more for a business than a few benchmark points.

Group	Example (checked October 2026)	What to watch
Standard permissive license	Qwen3 models, OpenAI gpt-oss, Gemma 4, Mistral Small 4 and the Ministral 3 models are published under Apache 2.0	Few strings beyond keeping notices; read the license file in the repository you actually download
Custom community license	Meta's Llama 3.1 license; earlier Gemma versions under the Gemma Terms of Use	Use policies, attribution, naming rules and conditions that pass to anyone you redistribute to
Non-commercial license	Some releases, such as Mistral's Voxtral TTS, carry CC BY-NC 4.0	Usually rules out business use outright

The Llama 3.1 license shows what custom terms look like in practice. Anyone who distributes the model, or makes available a product or service that contains it, must "prominently display 'Built with Llama'". The license also incorporates an acceptable use policy by reference, says that a distributed model trained or fine-tuned from it must "include 'Llama' at the beginning of any such AI model name", and requires a separate license from Meta for a licensee whose products had more than 700 million monthly active users in the month before the release date. The last clause will not touch a 50-person firm; the others might.

Two traps catch people here. First, the license can change between versions of the same family: Google describes Gemma 4 as the first Gemma family released under the OSI-approved Apache 2.0 license, while earlier Gemma models sit under custom terms with a prohibited use policy that downstream recipients inherit. Second, a vendor can mix licenses inside one catalog, so "Mistral is Apache" is true of some models and false of others. Check the exact model and version. For a reading guide to the clauses themselves, see what open-source LLM licenses allow for commercial use. None of this is legal advice; have counsel read the license for anything you ship to customers.

Step 3: Filter by size and memory

Your hardware sets a hard ceiling. A rough rule for weights is parameters times bytes per parameter. A common 4-bit format such as Q4_K_M averages a little under 5 bits per weight, so budget about 0.6 bytes:

• An 8 billion parameter model needs about 8 x 0.6 = 4.8 GB for weights.
• A 30 billion parameter model needs about 30 x 0.6 = 18 GB.
• A 70 billion parameter model needs about 70 x 0.6 = 42 GB.

Then add room for the context (the KV cache) and runtime overhead, which can be several gigabytes for long documents or many simultaneous users. On a single 24 GB card, that arithmetic leaves the 8B class comfortable, the 30B class workable with modest context, and the 70B class out of reach without a second card. The full calculation, with context included, is in how much VRAM an LLM needs, and the quality cost of squeezing to 4 bits is covered in LLM quantization explained.

Mixture-of-experts models change the trade

Several current families ship mixture-of-experts (MoE) models, where only part of the network runs for each token. The Qwen3-30B-A3B model card lists about 30.5 billion total parameters with 3.3 billion active per token. The gpt-oss-20b card lists 21 billion total and 3.6 billion active, and says the model can run within 16 GB of memory. The rule to remember: total parameters set the memory you need, active parameters set roughly how fast it generates. An MoE model is often a good fit for a box with plenty of memory and a modest GPU, because it costs the memory of a large model but generates at closer to the speed of a small one.

Prefer the smallest model that passes

Do not shortlist only the largest model that fits. Put one smaller model on the list on purpose. If an 8B model scores within a point or two of a 30B model on your examples, the smaller one answers faster, serves more people per card and leaves memory for longer documents.

Step 4: Filter by language and input type

Model cards state language coverage, and the claims vary widely. The Qwen3-8B card, for example, claims support for "100+ languages and dialects". A claim on a card is a starting point, not a result. Coverage of a language in training does not guarantee the model writes it at a level you would send to a customer, and smaller languages degrade first.

Check two more input facts at this stage:

• Context length. The same Qwen3-8B card lists 32,768 tokens natively and 131,072 with a scaling method called YaRN. If your inputs are 60-page contracts, a model with a short native window is either out or needs a retrieval setup around it.
• Modality. If inputs are images or scans, a text-only model is out regardless of its other scores.

After these three filters, aim for three candidates: usually one strong mid-size model, one smaller model and one wildcard from a different family.

Step 5: Test on 30 of your own examples

Build the test set

Pull 30 real inputs from recent work and write the expected output for each, or at least what a correct answer must contain. Spread them deliberately: about 20 typical cases, about 7 hard ones (long, messy, ambiguous, the minority language) and about 3 that should produce "cannot answer" or a refusal. Strip or replace personal data before the set leaves the system it came from, even if everything stays on your own server.

Run every candidate the same way

Use the same prompt, the same quantization level, the same context setting and a low or zero temperature for every model, so you compare models and not settings. Save every output to a file with the model name and version.

Score blind

Shuffle the outputs and hide the model names, then have the person who does the work today score each answer pass or fail against the expected output. For extraction tasks, score field by field instead.

The workload is manageable. Three candidates times 30 examples is 3 x 30 = 90 outputs. At about a minute each to read and score, that is 90 minutes for one reviewer.

Read the result honestly

With 30 examples, each one is worth 1 / 30 = about 3.3 percentage points. A model that passes 24 (80 percent) against one that passes 23 (77 percent) is a tie; one example flipping either way erases the gap. A sensible decision rule:

1. Any candidate below your minimum pass rate is out, whatever its reputation.
2. If two candidates are within 3 examples of each other, choose the smaller, faster or more permissively licensed one.
3. If the winner clears the next model by 4 or more examples, the gap is worth taking seriously, though 30 examples cannot prove it. Confirm by adding 20 more examples before committing hardware money to it.

Look at the failures, not only the count. Five failures that are all the same mistake (dates in the wrong format, the German emails) often disappear with a prompt change. Five scattered failures usually mean the model is not up to the task.

Keep the test set after you choose

The 30 examples are worth more than the decision they produced. When a new release appears, run it through the same set before switching; a newer model is not automatically better at your task, as updating on-premise AI models explains. Store the prompt, the settings and the scores next to the examples, and the next selection takes an hour instead of an afternoon.

Frequently asked questions

Should a business just pick the model at the top of a public leaderboard?

A leaderboard is useful for building the shortlist, not for making the final call. Its tasks, prompts and scoring may look nothing like your documents, and results often come from the full-precision model rather than the quantized version you would run. Use it to find candidates, then let your own examples decide.

Is an open-weight model the same thing as an open-source model?

Not always. Many popular models publish the weights but not the training data or training code, and some attach custom terms that a standard open-source license would not allow. For a business the practical question is whether the license on the exact file you download permits your use.

How often should the choice be revisited?

A reasonable rhythm is to rerun the test set when a candidate family releases a new version or when staff report a pattern of bad answers, rather than on a fixed calendar. Because the set already exists, a check costs about an hour per candidate.