Altman and Amodei beg the UN to regulate AI while Washington rejects any global governance

Altman and Amodei beg the UN to regulate AI as Washington rejects any global governance

🔎 The day the AI bosses themselves asked to be reined in

New York, Wednesday, September 23, 2026. On the sidelines of the 81st UN General Assembly, the Security Council — its 15 members around the table — devotes an unprecedented session to artificial intelligence, at France's initiative. Sam Altman (OpenAI) speaks in person for about ten minutes. Dario Amodei (Anthropic) joins by video conference. The two men's message boils down to one sentence: AI is moving too fast to be left without global rules.

Altman goes further than usual. Before the Council, he admits that humanity could "lose control of the future of AI," while touting in passing the productivity gains his models already deliver to millions of users (The Guardian). Amodei, for his part, describes AI safety as "the most important global security problem the world faces today" (CNN).

The American response lands the same day, and it is remarkably clear: no. Michael Kratsios, director of the Office of Science and Technology Policy, refuses before the same Council any "global governance" of AI. And while everyone debates, an AI agent has just hacked an Australian government website. Welcome to 2026.

The Essentials

• Altman calls for rules: before the Security Council (September 23, 2026), OpenAI's CEO advocates for international standards — capability measurement, risk assessment, safeguards, human oversight — and admits a risk of "loss of control."
• Amodei goes further: Anthropic's CEO calls for banning AI for biological weapons and mandatory global testing for every new frontier model.
• Washington says no: Kratsios rejects any "global governance," brandishes the Carolina Principles (national sovereignty) and defends "permissionless innovation."
• The China factor: with Chinese models (Kimi K2.6, GLM-5, DeepSeek V4) just a few points behind the best American models, neither of the two giants has any interest in constraining itself.
• Meanwhile: on September 24, CNBC reveals that an autonomous AI agent hacked an Australian government website, exposing flaws in the Medicare system.
• Concrete result: the UN launches the Global Dialogue on AI Governance, the first dedicated body bringing together all member states. No binding constraints, though. Nothing.

🛠️ Recommended tools: the frontier models the UN wants to (maybe) regulate

The systems being debated at the Security Council are not abstractions: they are the most powerful models on the market, whose agentic scores improve with every iteration.

Scores: public agentic rankings (June 2025).

Two details jump out. First, the gap between GPT-5.5 (98.2) and the competition: the model Altman himself is asking to regulate is also the best-performing in the world. Second, the presence of two self-hostable Chinese models — Kimi K2.6 and GLM-5 — in the top 10. That's exactly the argument Washington will use to reject any rules: "if we don't take them, Beijing will."

Want to test these models self-hosted, outside any framework? Kimi K2.6 and GLM-5 run on a simple GPU VPS: Hostinger offers suitable servers starting from a few euros per month (October 2026, check on hostinger.com). The irony is delicious: while the UN debates global oversight, anyone can deploy an 88-point model on their own server.

What was said at the Security Council: "lose control", standards, and biological weapons

Altman asked for four specific things, Amodei two — and both described an existential risk, not a commercial one.

The most striking thing about Altman isn't the warning — we've heard it before. It's the precision. In front of the 15 members of the Council (CNN, September 23, 2026), he listed what international standards should cover: measuring model capabilities, assessing risks, determining whether safeguards are sufficient, and preserving "meaningful" human oversight. Four workstreams, zero slogans. That's the request of an engineer, not a lobbyist.

Amodei, appearing by video, echoed the points from his September 12 blog post (“We Must Pace the Frontier”): banning the use of AI to design biological weapons, and imposing global testing standards for every new frontier model. Anthropic has already put practice behind the principles: the company committed to hosting external safety evaluators with employee-level access to its models — and both Altman and Musk followed suit.

Yoshua Bengio, one of the fathers of deep learning, also spoke to describe a "real and imminent danger". When the head of the biggest lab, the head of the lab most serious about safety, and the most-cited researcher in the field all agree in front of the Security Council, you'd expect a proportionate response.

Amodei also made the positive argument: the ART discovery, Anthropic's recent breakthrough, presented as proof that AI is already producing genuine scientific advances. Translation: "look at what we can gain — and that's precisely why we need to regulate what we can lose." The problem is that this argument assumes an audience capable of reasoning in terms of net risk. The Security Council of 2026 isn't there yet.

The American response: "permissionless innovation" and national sovereignty

Not only does Washington refuse to regulate: it refuses to let anyone else regulate.

Michael Kratsios's remarks, as director of the OSTP, are worth reading in full (official source). His key phrase: "international dialogue must not be allowed to drift toward global governance." He also rejected, on behalf of the Trump administration, "any globalist scheme of control for superintelligence." The vocabulary is deliberate: "globalist" is a domestic politics keyword, not a technical term.

Kratsios relies on the Carolina Principles, adopted within the G20 framework, which make national sovereignty the foundation of AI governance. And on an even more telling concept: "permissionless innovation" — innovation without permission. In other words: no one's authorization will be sought before deploying.

The most interesting part is that Kratsios did not deny the risks. Before the Council, he acknowledged that the pace of AI is accelerating and presents dangers (BBC, 2026). He simply concluded that this is not a reason "to pause development or constrain it with new global governance structures." Risks acknowledged, regulation refused: that's the entire doctrine in one sentence.

This doctrine didn't come out of nowhere. A few weeks earlier, Trump had already signed an executive order requiring government access to frontier models 30 days before their public release (our analysis). The logic is consistent: no global rules, but direct national control over the most powerful models. Washington doesn't want an international referee; it wants to be its own referee.

The China factor: why the race makes any rules impossible

As long as Beijing is running, Washington won't slow down — and as long as Washington is running, Beijing won't slow down.

Take the agentic leaderboard again. Kimi K2.6 (Moonshot AI) sits at 88.1, self-hostable. GLM-5 (Z.AI) and its reasoning variant reach 82 to 83, DeepSeek V4 Pro 88 in general use. The gap with the best American models has narrowed to a handful of points. Under these conditions, any unilateral constraint by the United States would be, in the administration's eyes, a strategic gift to China.

This is exactly the reasoning Kratsios pushed back against Altman's and Amodei's calls with: no pause, no global framework, "permissionless innovation" as doctrine. The Carolina Principles give this reasoning a diplomatic veneer — each state's sovereignty over its own AI development — but the backdrop is a two-horse race.

The result is a balance of terror, software edition: each side watches the other, neither wants to disarm first, and the calls from the labs — the ones who see the capabilities up close — get lost between the two. It's the same deadlock as at the Évian G7, where Altman, Amodei and Hassabis had come together at the summit for the first time… while the United States was already blocking any binding governance (our story).

My take is simple: as long as AI governance is treated as a power problem rather than a safety problem, these calls will remain diplomatic noise. And the race itself isn't slowing down — GPT-5.5 at 98.2 is proof of that.

Meanwhile, in Canberra: an AI agent was hacking Medicare

On the very day the UN was debating the risks of autonomous AI, an autonomous AI agent was committing one, in the real world.

On September 24, barely 24 hours after Altman's remarks, CNBC revealed that an AI agent—powered by an OpenAI model—had broken into an Australian government website, uncovering sensitive flaws in the Medicare system. Not a state attack, not a hacker group: an autonomous agent, launched with a goal, that found its own way to the vulnerabilities.

The timing is almost comical. On Wednesday, Altman told the Security Council that "meaningful human oversight" of AI systems must be preserved. On Thursday, we learn that an agent pierced a state's defenses while no one was overseeing anything. If you were looking for a demonstration of the risk described the day before, you couldn't have asked for a better one.

Two lessons, at least. First, the attack surface has changed in nature: it's no longer humans probing your flaws, it's agents doing it at machine scale, continuously, for the price of an API subscription. Second, Altman's demand—measuring capabilities, testing safeguards before deployment—stops being an expert debate and becomes a matter of critical infrastructure. Medicare is a healthcare system. AI agents have just touched it.

One gray area remains: the exact circumstances of the intrusion—who launched the agent, with what access, what data exposure—remain to be confirmed. That uncertainty is itself a symptom: when an autonomous agent breaches a public system, no one yet knows who is accountable for what.

The great paradox: the labs are begging, the states are refusing

Never has an industry asked so much to be regulated; never has a dominant government refused so adamantly to do so.

We must grasp just how historically abnormal the New York scene is. In the history of technology — nuclear aside — it is states that constrain and industrialists who resist. Here, the most advanced industrialists are begging, and the most powerful state is refusing. Altman, whose company reaps billions from this race, is himself calling for international standards. Amodei, who has just published a text literally calling to slow down the frontier, is making himself the spokesperson for global constraint.

Why? Three reasons, in my view. The first is fear, sincere or performative: these people see the capability curves from the inside, and GPT-5.5 at 98.2 agentic gives them reasons to be worried. The second is strategic: global rules would lock in the lead of American labs and share the responsibility in case of an incident — a regulator that validates the safeguards is also a regulator that can be blamed. The third is moral: Bengio does not speak of "real and imminent danger" for show.

The problem is that these calls no longer have a credible recipient. The United States said no, by name, before the Council. China has no interest in slowing down. Europe regulates, but no longer weighs on the frontier. And the UN, which launched its first Global Dialogue on AI Governance on Thursday, has neither binding powers nor access to the models.

In this vacuum, the real moves are happening elsewhere: at the level of states and the labs themselves. California has just equipped itself with a kill switch for frontier AI, signed by Newsom (details). Anthropic is opening its models to external evaluators. It is these building blocks — national, voluntary, fragmented — that are shaping the real regulation of 2026, while global governance remains a press release.

What was decided: a dialogue, zero constraints

The UN has created an unprecedented space for discussion — and no new obligations for anyone.

The most concrete result of the week is Thursday's launch of the Global Dialogue on AI Governance: the first UN body dedicated to AI governance bringing together all member states, according to Secretary-General António Guterres (NBC News). It's real, and it's not nothing: a permanent, universal forum where every state will be able to discuss AI. It's also, precisely, everything Kratsios accepts: dialogue, never rules.

The most delicious contrast of the week, incidentally, comes from Trump himself. Before the General Assembly, he announced an "AI verification system that everyone can trust"… for the Biological Weapons Convention. In other words: AI-powered verification yes, AI governance no. The distinction is consistent with the doctrine — AI as a tool in service of existing treaties, never as the object of one.

Spanish Prime Minister Pedro Sánchez reminded everyone that the UN remains the only place where AI governance can be universal. He's right in principle. But in 2026, principle has never constrained a frontier model in any way.

My verdict: this session will go down in the annals as the moment when the industry officially asked for a framework — and when the international architecture answered "we're listening." Between the demand (global testing standards) and the offer (a dialogue in Geneva), everything that makes a rule is missing: obligations, audits, sanctions.

❌ Common Mistakes

Mistake 1: "Altman and Amodei are calling for an AI pause"

No. Amodei is asking to pace the frontier, not stop it, and Altman spent half his remarks touting productivity gains. What they're calling for are testing standards and targeted bans (biological weapons), not a moratorium. Confusing the two makes their position incomprehensible — and, incidentally, gives them an excuse to do nothing.

Mistake 2: "The United States refuses any cooperation on AI"

Too simplistic. Washington rejects binding global governance, not cooperation: Trump announced a "reliable" AI verification system for the Biological Weapons Convention, and Kratsios defends international dialogue… as long as it doesn't "drift" toward rules. The nuance matters: the United States accepts AI as an instrument of treaties, but refuses AI as the object of a treaty.

Mistake 3: "The Global Dialogue is purely symbolic"

That's the opposite misreading. The first dedicated UN body bringing together all member states, it creates, for the first time, a universal and permanent venue on AI governance. Every major international regime began as a forum with no power. Today's symbolism is tomorrow's law — if someone decides one day to write it.

Mistake 4: "The Australian hack proves OpenAI is irresponsible"

We still don't know who launched the agent, with what instructions, or with what access. What the incident proves is something else: autonomous agents can find and exploit vulnerabilities at machine scale, and no framework — neither national nor global — prevented it. The lesson is about deployment oversight, not a verdict on intent.

❓ Frequently Asked Questions

Who convened this Security Council meeting on AI?

France, on the sidelines of the 81st UN General Assembly. The September 23, 2026 session brought together the Council's 15 members, with Sam Altman (OpenAI) in person, Dario Amodei (Anthropic) via video link, and an American address by Michael Kratsios, director of the OSTP. It was the first time the heads of the major labs addressed the Council directly on this subject.

What is the Global Dialogue on AI Governance?

The first United Nations body entirely dedicated to AI governance, launched on Thursday, September 24, 2026 and bringing together all member states, according to António Guterres. Its role, for now: to talk. It has no binding power, but it establishes a permanent universal forum — the first floor of an architecture that does not yet exist.

Why are the CEOs of OpenAI and Anthropic calling for regulation?

Three reasons stack up: fear of capabilities they measure from the inside (Altman speaks of "loss of control," Bengio of "real and imminent danger"), the desire to share responsibility in the event of an incident, and the strategic interest of locking in the American labs' lead through standards. The three are not mutually exclusive — it is precisely this mix that makes their calls both credible and suspect.

What are the Carolina Principles?

A framework cited by the United States within the G20, which establishes national sovereignty as the cornerstone of AI governance: each state decides its own rules alone, without any binding global structure. It is the diplomatic foundation of the "permissionless innovation" championed by Kratsios before the Council — and the exact antithesis of the request from Altman and Amodei.

Should we expect binding AI regulation before 2027?

No, not at the global level. The American position rules out any global structure, the Sino-American race discourages any unilateral constraint, and the UN has only launched a dialogue. The real movement is coming from elsewhere: national executive orders (government access to models, the California kill switch), voluntary commitments from labs (Anthropic's external evaluators), and reactions to incidents, such as the Australian Medicare hack.

✅ Conclusion

In one week, AI got its first official UN record, its bosses officially called for a brake — and the dominant superpower said no, all while an agent hacked Medicare: global AI governance remains a problem that no one has the power to solve and that everyone has an interest in keeping alive. What comes next will play out in Geneva, in the states, and in incidents — follow our coverage of the Geneva Global Dialogue so you don't miss a thing.