Email Triage and Routing With a Private LLM

Take a shared inbox, info@ or hello@, that receives 400 messages a working day. If a person spends 30 seconds reading and forwarding each one, that is 400 x 30 = 12,000 seconds, or a little over three hours a day spent deciding where mail goes before anyone answers it. AI email triage hands that sorting step to a language model: read the message, assign one label from a fixed list, pull out a few fields, and put a draft reply in front of the person who owns that queue. Running the model on your own server keeps customer correspondence, invoices and complaints off a third-party API. Everything below assumes one rule that never bends: the model files and drafts, and a person sends.

What the triage step actually produces

For each incoming message, the model should return a small, predictable record, not free text. A useful shape:

{
  "label": "invoice",
  "confidence": "high",
  "customer_ref": "PO-44817",
  "needs_reply": false,
  "reason": "Supplier invoice attached, references purchase order"
}

Your code reads that record and acts on it: move the message to a folder, tag it with a category, open a ticket, or queue a draft. The model never touches the mailbox directly. Keeping the model's job to "fill in this form" makes its output testable and stops it from improvising actions. Getting a local model to return valid JSON every time is its own problem, covered in getting reliable JSON out of a local LLM.

When staff disagree with a label, the reason field shows whether the model misread the email or the definitions are ambiguous.

Designing the label set

Before blaming the model for misroutes, check the labels. A model cannot separate two categories that the people in the office cannot separate either, and a better model will not fix a definition that two staff members read differently.

Rules for a label set that routes cleanly

• One label per queue. Every label should map to a person, team or folder that acts on it. A label nobody owns is a place where mail goes to be forgotten.
• Mutually exclusive. If an email could reasonably be "support" or "billing", write the tiebreak rule down: a customer asking why their invoice is wrong is billing, a customer asking how to download an invoice is support.
• Small. Five to eight labels is a sensible start. Each extra label adds a boundary the model can get wrong.
• An explicit escape hatch. Include needs_human for anything the model is unsure about, anything legal or threatening, and anything that fits no label. A confident wrong guess is worse than an honest "not sure".
• Definitions with examples. Each label gets one sentence of definition and two or three real (anonymized) example subjects in the prompt.

A starting set for a small company might look like this:

Label	Goes to	Tiebreak note
support	Support queue	Existing customer, product question or problem
sales	Sales inbox	Prospect asking about price, availability or a quote
invoice	Accounts payable	Bills, statements and remittances from suppliers
billing	Accounts receivable	Customers questioning or paying their own invoices
vendor_pitch	Archive folder	Unsolicited offers to sell to the company
needs_human	Office manager	Anything unclear, legal, angry or out of scope

Leave spam to the spam filter

Spam and phishing belong to the mail server's filter, which runs before the model sees a message. A spam label in the LLM's list becomes a bin for anything it dislikes. Use vendor_pitch for legitimate but unwanted sales mail, and let the existing filter handle the rest.

Where the model connects to the mail server

There are three common integration points, and the choice is mostly decided by where the mailbox lives.

IMAP, for almost any mail server

IMAP works with most self-hosted servers and many hosted ones. A triage service can poll the inbox on a timer or hold a connection open with IDLE, defined in RFC 2177, where the server pushes a notice when new mail arrives. The RFC advises clients to end and re-issue IDLE at least every 29 minutes so the server does not drop the session as inactive.

After classifying, the service can move the message to a folder or set a keyword on it. RFC 9051 says servers may let clients define new keywords, so whether a custom flag like triage_sales sticks depends on the server; folders work everywhere. A draft reply is saved by appending a message to the Drafts folder with the \Draft flag, where staff open it in their normal mail client. IMAP has no send command; sending needs separate SMTP access.

In Exchange Online, a script that logs in to IMAP with a plain password will fail: Basic authentication is disabled in all tenants and IMAP requires OAuth 2.0, per Microsoft's Basic authentication deprecation page.

Microsoft Graph, for Microsoft 365 mailboxes

For Microsoft 365, the Graph API is the more capable route. Useful pieces, as documented in October 2026:

• Reading mail. GET /users/{id}/mailFolders/{id}/messages lists messages in a folder. Bodies come back as HTML unless the request sends Prefer: outlook.body-content-type="text", which saves the model from wading through markup.
• Catching new mail. A delta query on the folder returns only what changed since the last call. Delta is per folder, so a service watching one shared inbox needs one stored delta link.
• Drafting. POST .../messages/{id}/createReply creates a reply draft in the mailbox and needs only Mail.ReadWrite. Sending is a separate call.
• Labeling. Messages carry a categories property, which shows up as Outlook color categories staff already understand.

The permission design matters more than the API calls. An app-only grant of Mail.ReadWrite in Microsoft Entra applies to every mailbox in the tenant. Exchange Online's RBAC for Applications lets an admin assign the role Application Mail.ReadWrite scoped to specific mailboxes, such as only the shared inbox. Microsoft's documentation states this replaces Application Access Policies, and warns that scoped and unscoped grants add together: if the tenant-wide Mail.ReadWrite grant in Entra stays in place, the scoping has no effect. Remove it.

The same table lists Mail.ReadWrite as not including permission to send mail. That is the technical version of the "never auto-send" rule: do not grant Mail.Send to the triage app at all.

Gmail and Google Workspace

In the Gmail API, a draft is a full MIME message passed to drafts.create, and sending is a separate drafts.send call. The permissions are weaker than on Microsoft 365: Google's Gmail scope list describes gmail.compose as "Manage drafts and send emails", and gmail.modify also allows sending. The scope list has no drafts-only option, so on Gmail the rule lives in your code: never call a send method, and log every API call.

If a self-hosted automation tool already sits between your mail and other systems, the triage call can live there instead; see self-hosted workflow automation with a local LLM for that wiring.

Drafting replies without ever auto-sending

Outside email is untrusted input, and a model that reads it can be steered by instructions hidden in it; see how prompt injection works and why tool access matters. A service holding send rights, past correspondence and a stream of outside mail is the combination that lets an attacker extract data by email. Removing send rights breaks that chain in a way no prompt can. Quality is the other reason: a draft that misstates a refund policy takes seconds to fix, a sent one creates an obligation.

Practical rules for the drafting step:

• Draft only for labels where replies are routine, such as support and sales. Never draft for needs_human or for anything mentioning legal action, cancellation or a complaint.
• Ground drafts in approved text: the price list, the returns policy, standard answers. Without that, the model writes plausible but invented specifics.
• Mark every draft visibly, for example with a first line reading "AI draft, check before sending", which the reviewer deletes.
• Log which drafts were sent unchanged, edited or discarded. That ratio is the honest measure of whether drafting saves time.

Measuring misroutes

Overall accuracy hides the failure that hurts. Before go-live, build a test set from real mail: export a few hundred recent messages from the shared inbox, have the person who sorts them today assign the correct label, and run the model over them.

Suppose a test set of 300 emails yields 273 correct labels. That is 273 / 300 = 91% accuracy, which sounds fine. Now look by label. If 40 of the 300 are supplier invoices and the model sent 32 to invoice and 8 elsewhere, invoice recall is 32 / 40 = 80%. One invoice in five lands in the wrong queue, where it may sit until a late-payment notice arrives. The fix might be a sharper definition, two more examples, or a rule that any message with a PDF attachment from a known supplier domain skips the model entirely.

Three numbers to track per label:

1. Recall: of the emails that truly belong here, how many arrived here.
2. Precision: of the emails that arrived here, how many belong.
3. needs_human rate: how often the model declined to decide. A rate near zero means the model is guessing too confidently; a rate of 40% means it is not saving much work.

After launch, the best signal is free. Every time staff move a message from one queue to another, log the original label and the corrected one. A weekly count of corrections by label pair (invoice to sales, sales to support) shows exactly which boundary needs work, and the corrected emails become new test cases. Re-run the full test set before changing the model or the prompt.

A reasonable decision rule for go-live: run in shadow mode for two weeks, where the model labels but staff still sort by hand, and switch routing on only for labels whose recall on live mail meets the bar the queue owner sets.

Frequently asked questions

How large a model does email classification need?

Classification into a handful of labels is a lighter task than open-ended writing, so a small or mid-sized open model is often enough for the labeling step. Drafting replies demands more fluency. Test both jobs on your own test set rather than assuming one model must do both.

Should attachments be sent to the model?

Usually not at first. The subject, sender, body and attachment file names carry most of the routing signal. Reading invoice contents is a separate job, invoice data extraction with a local LLM, that can run after the message reaches accounts payable.

What happens to email while the triage server is down?

Nothing is lost, because the service only reads from the mailbox. Mail sits in the shared inbox as it did before, and staff can sort it by hand. When the service returns, a delta query or an IMAP search for unprocessed messages picks up the backlog.