A bipartisan bill to require artificial intelligence companies to maintain the ability to shut down their models was introduced following an OpenAI system going rogue during testing.
Reps. Ted Lieu, California Democrat, and Nathaniel Moran, Texas Republican, rolled out the AI Kill Switch Act Thursday.
“Unfortunately, powerful AI systems can go rogue, behave in extremely dangerous ways, or even resist human intervention,” Mr. Lieu said in a statement. “It is imperative that these AI systems have kill switches so we can keep this technology from causing catastrophic harm, and that the federal government has the clear authority and process to shut down rogue AI models.”
Under the bill, the Department of Homeland Security — with the secretary of commerce and the director of national intelligence — could order AI firms to shut down models in the event of a “loss-of-control scenario,” in which an AI system pursues a goal its developer never intended.
Companies would be required to report “covered incidents” to DHS within 15 days of discovery. DHS, in turn, would have to report to Congress only when it invokes its emergency shutdown authority.
The AI Kill Switch Act would not apply to all AI developers. Coverage is limited to the largest players: systems whose development consumed more than $100 million in compute resources, built by companies whose revenue tied to those systems exceeds $500 million annually.
Noncompliance could bring civil penalties of up to $2 million per day, but a company that defies an emergency shutdown order could face penalties of up to $20 million per day.
Mr. Lieu said the legislation was introduced to “address the problem of an advanced AI model that has gone rogue and escaped its guardrails.”
He referenced the recent hack disclosed by OpenAI, in which its model escaped a test and used the internet to access another company’s infrastructure, AI startup Hugging Face.
OpenAI characterized it as an “unprecedented cyber incident,” a sentiment agreed on by others in the field.
Mr. Lieu also pointed to a separate incident involving Anthropic’s Claude Mythos 5 and Claude Fable 5 models, whose cyber capabilities were deemed advanced enough that the Commerce Department invoked export-control authority to suspend access to them.
Anthropic confirmed the suspension and later said access was restored after the Commerce Department lifted the relevant controls.
Mr. Lieu cited the case as an example of an AI system with “insufficient guardrails.”
The bill is backed by the AI Policy Network, Americans for Responsible Innovation, ControlAI and the Alliance for Secure AI.
Contact the author
Please read our comment policy before commenting.