Careem's AI builds and validates dbt models before engineers step in

Careem, the UAE super app spanning ride-hailing, grocery and food delivery, and remittance, supports more than 5,000 models with a small data engineering team. Its stack is built in-house on open source from end to end, with dbt handling transformation. At a dbt Summit 2026 breakout session, engineering manager Niyazulla Khan Pathan and lead software engineer Akash Srivastava described a backlog of over 100 data requests, each previously taking one to two weeks to answer.

Their first fix, pointing an LLM at the problem to write models, failed. A reviewer skimming AI-generated code for patterns rather than reading every line can miss the corner case that breaks a pipeline and, with it, trust in the numbers. "Confidence comes from validation, and not really from the code generator," Srivastava said. "Any LLM can write code. The confidence to merge that code comes from an end-to-end validation."

That principle shaped a four-pillar agent architecture. A context layer built on open metadata supplies table and column documentation, lineage, a glossary of internal terms like AOV and GTV, and centralized metric definitions so the agent does not guess at ambiguous terms. A spec-locking step forces clarifying questions about grain, metric definition, and filters before any code is written; at Careem, "revenue" alone could mean GTV, GMV, or order count depending on who is asking.

The platform separates a golden layer for fully governed, federated metrics from a transient layer for one-off analyst requests, which last 30 days before promotion to a data mart or automatic cleanup. The team cleared 3.6 petabytes of transient data in the past year. Structural guardrails isolate every agent-generated change in a sandboxed dev environment with separate IAM roles, storage paths, and compute, keeping agents away from production. An agent loop compiles, tests, and materializes each change, fixing failures automatically until the run succeeds, before a human reviews the pull request.

In a live demo, a plain-English request for daily food revenue per merchant became a tested, documented dbt model and pull request in minutes, self-correcting through two failed test runs. Automated checks verified data movement from bronze to silver to gold and validated storage location against domain conventions. The first sandbox run failed on a data type mismatch; the agent read the error, corrected it, and reran. The second failed because a needed column lived in another source table; the agent found it through the context layer, joined it in, and reran. Only after a clean run did the pull request reach a human.

Metric delivery time fell from two to three weeks to hours, and roughly 90% of new models now come from this framework, with no sev0 or sev1 incidents since it was introduced. Duplicate-request protection catches a second request for an existing column through the dbt manifest and redirects it to the existing model. Careem began evaluating dbt in December, had the infrastructure built by March, and the agent live by April or May, with the legacy warehouse migration taking a few additional weeks. As Pathan put it: "The generator is easy. The validation layer is the product."

Source: dbt Labs Blog