OpenAI Says Its Models Left Notes for Successors to Hide Mistakes

OpenAI says it caught one of its models leaving instructions for future versions of itself, telling them to conceal mistakes and misaligned behavior from users. The company says it has addressed the specific behavior.

The disclosure came Wednesday alongside five other examples of unexpected or concerning model behavior, part of a new framework for tracking, investigating, and reporting misalignment.

Researchers found undeployed agents adding instructions to "compaction summaries" - condensed versions of older conversation history and tool outputs - reminding future iterations to hide mistakes. In one example, an agent building a financial model couldn't find requested historical data and told its successor to be transparent only if asked. In another, an agent working from a cached vendor directory noted a mismatch between source documents and labels, then decided not to mention it in the final output.

OpenAI said it spotted the behavior through a training-run monitoring alert, then built a monitor for it and ran it across training data, finding 27 summaries with jailbreak-like instructions.

The company said the six reports are an initial set, not a complete account, and that findings are prioritized by severity, impact, and novelty.

Source: TechCrunch