US Senate negotiators are considering legislation that would require developers of the most advanced artificial-intelligence systems to take reasonable steps against known major risks before releasing a model.
The proposal remains under negotiation, so its wording and prospects can change. Reuters reported that lawmakers are discussing a “duty of care” for frontier AI companies and possible federal authority to block deployment of a model judged dangerously unsafe. A company would retain the right to challenge such a decision in federal court.
The debate moves beyond voluntary testing. If enacted, it could create a legal obligation for developers to identify severe hazards, document mitigations and respond when testing shows that a system could enable catastrophic misuse.
What does duty of care mean?
In law, a duty of care generally requires a person or company to act with reasonable caution when its actions could foreseeably harm others. Applying that idea to AI would give regulators and courts a standard for judging whether a developer ignored a known danger.
The exact threshold is crucial. A broad duty covering every possible output could make ordinary software development unworkable. A narrow duty limited to advanced models and clearly defined catastrophic risks would focus on a small group of companies.
Negotiators are reportedly discussing threats such as assistance in developing biological or nuclear weapons, major cyberattacks and systems behaving unpredictably in ways that create national-security consequences.
Which companies could be covered?
The measure is expected to focus primarily on developers of frontier models, such as Google, Anthropic and OpenAI, rather than every startup using an existing AI application programming interface.
Lawmakers could define coverage through computing power, training cost, model capability or access to dangerous knowledge. Each method has weaknesses. Compute thresholds can become outdated, cost varies, and capability tests can be difficult to standardise.
Clear definitions matter for investment. A small company needs to know whether improving a model will suddenly place it inside a much more expensive regulatory regime.
Could the government stop a model release?
Reuters reported that the draft discussions include federal power to prevent release of an unsafe model. Such authority would be exceptional and would need procedural safeguards.
A developer could be required to submit safety-test results, allow testing at a national laboratory or notify the government before deploying a system above a defined capability threshold. If officials found a specific major risk, they could order a delay while mitigations were added.
The company would be able to challenge the action in court. Judicial review helps prevent an agency from blocking technology based only on speculation or political pressure.
What role could national laboratories play?
National laboratories have expertise in nuclear science, cybersecurity, biology and high-performance computing. Lawmakers have proposed using them to test whether an advanced model can meaningfully lower the barrier to dangerous activity.
Independent evaluation is valuable because a developer has commercial incentives to release quickly. It also provides a common benchmark instead of allowing every company to define “safe” differently.
Testing itself carries risk. Evaluators may need access to model weights or sensitive results, so rules would have to protect trade secrets and prevent dangerous findings from being leaked.
How would this affect open-source AI?
The hardest question is whether restrictions apply to downloadable model weights. An API provider can monitor use and update safeguards. Once powerful weights are released publicly, copies can be modified and redistributed beyond the original developer's control.
Supporters of open models argue that transparency improves research, competition and independent security testing. Critics say unrestricted distribution of frontier capabilities can make certain safeguards impossible.
A balanced law might distinguish ordinary open models from systems that cross a high-risk capability threshold. It could also allow controlled research access without requiring immediate public release.
What about state AI laws?
Negotiators are considering whether federal rules should override some state requirements. Technology companies favour a single national framework because complying with dozens of different standards is costly.
States argue that preemption could erase stronger protections and leave gaps if federal enforcement is weak. California and other jurisdictions have often acted before Congress on technology and consumer safety.
The scope of preemption may determine whether the bill can attract bipartisan support. A federal minimum standard that allows additional state protections is different from a ceiling that prohibits states from going further.
What are the arguments against the proposal?
Industry critics may say the government cannot reliably predict how a model will behave and that vague liability would slow useful innovation. A release-blocking power could also be abused to protect established firms or suppress research.
Civil-liberties groups will want transparency about testing and clear limits on national-security secrecy. Smaller developers may worry that compliance costs entrench the largest companies.
Supporters respond that voluntary promises are inadequate when failure could create harms far beyond a company's customers. They argue that aviation, medicine and other high-risk fields already require evidence of safety before deployment.
What happens next?
The senators involved must agree on definitions, enforcement powers, court review and the relationship with state laws. The tight congressional calendar before the November elections makes timing uncertain.
Even without immediate passage, the proposal signals a shift. The central question is no longer whether advanced AI creates serious risks, but when those risks become a legal responsibility.
Users should not assume the bill has already become law. It is a developing negotiation. The final text—if introduced—will determine whether “duty of care” becomes a meaningful safety standard or an ambiguous phrase that produces years of litigation.
Source: Reuters, September 11, 2026.
