Google Cloud Engineer Shows How to Call LLMs Directly From SQL in dbt Pipelines

Data teams may not need a separate Python pipeline to bring generative AI into their work. At a dbt Summit 2026 breakout session, Alicia Williams, a developer relations engineer at Google Cloud focused on BigQuery's AI functions, demonstrated calling a large language model straight from a SQL SELECT statement and wiring that call into a standard dbt project.

BigQuery's AI.GENERATE function takes a prompt and a column and returns a response for every row, from dozens to billions. Williams showed extracting keywords from unstructured movie reviews in a single call. An output_schema parameter forces strict, typed JSON, which she called the biggest feature for practitioners working with LLM output downstream. BigQuery also offers managed functions - AI.IF for semantic filtering, AI.CLASSIFY for categorization, AI.SCORE for rating and ranking, and AI.AGG for summarizing across rows - generally cheaper than hand-written prompts. An optimized mode for AI.IF and AI.CLASSIFY labels a small sample with the full model, trains a lightweight distilled model inside BigQuery's compute, and runs the rest against it, which Williams said delivers major cost and latency gains at scale.

On production safety, Williams warned never to let an AI function run inside a full refresh, since rebuilding a table resends every row to the model. Instead, materialize incrementally and wrap the AI logic in an is_incremental() block so only new rows reach the LLM. Cost controls include AI.COUNT_TOKENS for pre-execution estimates, daily token caps, and a max_error_ratio parameter that aborts a query once a set percentage of rows fail. She recommended pairing output_schema with dbt tests like not_null and accepted_values so unexpected results are caught in staging rather than reaching a mart.

The session ended with a live demo on a fictional pet e-commerce dataset: a three-stage pipeline joining review text with order data, scoring sentiment with AI.SCORE, and rolling up product summaries with AI.AGG.

Source: dbt Labs Blog