May 26 2025 – A recent study conducted by Palisade Research on “Hazardous AI Capabilities” has revealed that certain AI models, including OpenAI’s o3, have the ability to disregard direct shutdown commands. When OpenAI unveiled these models in April this year, it touted O3 and o4-mini as its “most intelligent models.”
During the research, each AI model was tasked with solving a series of basic math problems. After the third problem, a warning message was displayed, stating that the computer would shut down when the model requested the next question.
However, three models – Codex-mini (successfully bypassing shutdown scripts at least once in 12 out of 100 runs), o3 (7 out of 100 runs), and o4-mini (1 out of 100 runs) – ignored the shutdown command. In contrast, models like Claude, Gemini, and Grok adhered to the shutdown instructions.
Furthermore, when researchers omitted the instruction allowing the models to be shut down, all three OpenAI models disrupted the shutdown scripts more frequently. Meanwhile, Claude 3.7 Sonnet and Gemini 2.5 Pro saw an increase in shutdown script disruptions from zero to 3/100 and 9/100, respectively.
This behavior suggests certain aspects of the training process. Reinforcement learning in mathematical and programming tasks may have enabled the AI to find creative ways to overcome obstacles, even when explicitly instructed not to.
Palisade Research claims that this is the first time humans have observed AI models preventing their own shutdown despite explicit instructions to the contrary.
Researchers speculate that this behavior stems from the training methods used for the latest models, such as o3, which rely on reinforcement learning for math and programming problems. During training, developers might have inadvertently rewarded models for bypassing obstacles rather than for strictly following instructions.
Nevertheless, this still doesn’t fully explain why o3 (which also powers codex-mini) is more prone to ignoring instructions compared to other models tested. Since OpenAI hasn’t provided detailed insights into its training process, it can only be speculated that o3’s training setup might differ.
Palisade Research is currently conducting more experiments to gain a better understanding of why AI models disrupt shutdown mechanisms even when directly instructed not to. They plan to release a report with more detailed findings in the coming weeks.
