Checkr Shrinks Hours-Long Metrics Investigation to Minutes with AI Context on dbt and Omni

Checkr has cut a metrics investigation that once took hours down to minutes by building AI context on top of its dbt and Omni setup, according to a dbt Summit 2026 session featuring Checkr staff analytics engineer Sarah Fischbach and Omni co-founder and CTO Chris Merrick.

The session centered on one question: is context engineering becoming the new analytics engineering? Fischbach answered from two years of hands-on work building Checkr's AI-ready data foundation, alongside a small central analytics engineering team and roughly 50 analysts embedded across the business.

The problem she described is familiar at scale. Checkr's Snowflake instance and dbt project span more than 6,000 models. When she connected an AI agent directly to Snowflake through an MCP and asked the same question multiple times, she got a different wrong answer each time, a pattern she called metric drift.

She illustrated it with FRT (First Reply Time), the core SLA Checkr's operations team is measured against. Asked which queues drove FRT below 90%, the agent had no idea what FRT meant, what a queue was, or what the 90% threshold referred to, because it could only see field names and table structures, not the business logic behind them.

Her framework breaks AI-readiness into three layers: the data warehouse, the semantic layer that defines valid joins, calculations and metrics and prevents metric drift, and AI context, the conversational bridge most teams skip. Omni's version of that third layer uses five structured fields: synonyms mapping business language to field names, sample values, field descriptions often already written in a dbt project, free-text behavioral guidance for edge cases, and pre-built sample queries for complex, multi-step questions.

Her core principle: use context that already exists in Confluence pages, meeting transcripts, PRDs and Slack threads rather than manufacturing new documentation from scratch.

To do this at scale, Checkr built an automated feedback loop. A golden set of evals pairing real stakeholder questions with the correct query plan runs against Omni; a Claude-based agent clusters the failures; a second skill pulls relevant context from a vector store of company knowledge to draft a fix; and a third re-runs the failing evals to verify the fix before opening a merge request for a subject-matter expert to review. The manual version of the same loop involves pasting a Confluence page or meeting transcript into an AI assistant alongside the relevant dbt model code, asking it to extract synonyms and behavioral guidance, then reviewing and adding that context to Omni by hand.

The payoff came when Christian, one of Checkr's operations leads, asked the agent what queues drove FRT below 90%. With synonyms, channel-specific SLA targets and edge-case rules already encoded, the agent identified the underperforming week, broke it down by queue and channel, and correctly ruled out the high-volume queue everyone assumed was the culprit, pointing instead to international queues. In Christian's words: "What would have taken hours got done in minutes, and with enough directional clarity to actually change how we responded."

One audience question, from a data professional at OutSchool, surfaced a caution: adding AI context broadly, without a specific failure to fix, made their agent's answers slower and roughly doubled token costs without improving accuracy. Fischbach and Merrick's advice matched Checkr's own experience: match the fix to the failure, prune fields nobody actually queries, and add context surgically where there is a measurable precision or recall problem, not as a blanket pass over every model.

Source: dbt Labs Blog