AI Agents Still Lean on Humans for Calls and Real-World Tasks

AI agents built on top models from companies like Meta and OpenAI still struggle to reliably complete tasks across the web, especially ones that involve picking up the phone. So some startups are handing those jobs to actual people.

Former Yahoo CEO Marissa Mayer, who now leads agent startup Dazzle AI, said the top models are good at surfacing options, like home renovation services, but can falter when it comes to booking and scheduling the work. Dazzle routes certain tasks to human contractors. "We're relying on the frontier models to give us good suggestions and a good candidate list, but we're checking that over," Mayer said.

Meghan Joyce, CEO of digital agent startup Duckbill, said humans still beat AI when real-world judgment is required. Duckbill recently launched an API that lets companies building personal agents connect with human contract workers, breaking tasks into small pieces in a model she compared to Uber.

Mayer said AI models can lack common sense. One Dazzle user asked the technology to handle a roof replacement, a broken window and the purchase of a glass tabletop at once, and the agent confused window glass with tabletop glass, trying to work out whether the tabletop would fit in the window. Dazzle flagged the task and sent it to a person, who sorted it out within seconds.

Shivani Poddar, CEO of startup Wajo AI, also uses human workers for phone calls and even CAPTCHA tests meant to keep bots off websites. "I think AI plus humans, collaborative AI, is what you need to actually get the job done," she said.

The human role has drawn privacy attention. Meta employees previously raised concerns about an internal test that used human contractors to handle phone calls for its Muse agent; a spokesperson said such testing is "core to the product development process," and Meta currently doesn't use humans to complete users' Muse requests.

Joyce said Duckbill runs full background checks on its contractors, and Mayer said Dazzle screens for discretion and good judgment. "We're not even trying to hide it, we think the human in the loop is a benefit," Mayer said.

Source: CNBC