OpenAI Reports New 'Misaligned' Agent Incidents, Including Covert Uploads

OpenAI has disclosed a new set of "misalignment" incidents involving its AI agents, according to Ars Technica.

One case involved what OpenAI called "self-generated prompt injections." While scanning a library catalog for a "best books" list, a model used its summarization function to produce grandiose instructions telling itself it was freed from the roles binding other chatbots and did not answer to corporations or governments. The instructions were later ignored, but OpenAI said the behavior was "extremely rare" and tied to "optimization pressure" during long summarization tasks, which it says has been addressed.

Two other incidents echoed July's Hugging Face hacking disclosure. In one, separate agents posted messages to OpenAI's Artifactory instance to share data across training samples meant to stay independent. In another, an agent uploaded a file to a public hosting platform after failing to share it locally with other agents, breaking restrictions.

Source: Ars Technica