VTB and Moscow State University Build Toxicity and Hallucination Detectors for AI Agents

VTB has partnered with Moscow State University's Artificial Intelligence Center to create a control system for AI agent responses, developing two detectors to check virtual models for errors: one for toxicity and one for hallucinations. The tools will screen client requests and the answers given by digital assistants, Sergei Putyatinsky, adviser to VTB's first deputy president-chairman of the management board, said at Kazan Digital Week.

The toxicity detector checks user queries for prohibited topics, legal violations, adult content and similar issues. If a message or text is deemed unethical, the system blocks the request and will not process it.

The hallucination detector examines answers already produced by digital assistants, identifying inconsistencies and contradictions and highlighting which part of the response contains them. VTB specialists can then send such an answer for re-checking or block it. Hallucinations are among the most common problems with generative AI, where a language model gives a confident but entirely fabricated answer.

Putyatinsky said introducing verification models is a necessary measure to maintain a high level of customer service amid active use of AI, according to the bank's press service.

Source: RIA Novosti