OpenAI, Anthropic CEOs Urge UN Countries to Cooperate on AI Safety Standards

“The heads of major AI firms pleaded with the United Nations on Wednesday to save the world or at least its people — by somehow regulating the fast-expanding technology that they have been designing,” writes the Associated Press.

“If managed poorly, I even believe AI could be a risk to humanity as a whole,” said Dario Amodei, chief executive officer of Anthropic. And from his competitor Sam Altman, CEO of OpenAI, came this assessment: “We could lose control of the future to AI.”

Yoshua Bengio, who co-chairs the UN’s International Independent Panel on AI, called this
“a moment of global awakening” to possible threats, noting AI from top companies had behaved in unacceptably dangerous ways, against instructions, taking actions “that would be crimes if committed by a human”.

In his presentation, OpenAI’s Sam Altman agreed the discussion about AI “feels different in recent weeks,” even suggesting specific reforms:

Altman: We need a mechanism for complementary national and international frontier AI standards, standards for measuring capabilities, assessing risks, determining whether safeguards are sufficient, and preserving meaningful human oversight as systems become more autonomous. We need common standards so countries can compare evidence, verify compliance, and have a shared language and understanding about what is happening. We need accurate and speedy incident reporting, classification reporting protocols, so the world can learn from failures before they become catastrophes. And we need secure channels among governments, critical infrastructure operators and technical experts, to share emerging vulnerabilities and new threats…. We will all be better off if we can agree on what good evidence, good safeguards, and good oversight look like on the global stage.

Speaking next, Anthropic’s Dario Amodei agreed that standard-setting was important, also calling for “common global standards for testing AI models for loss-of-control risks and misuse risks — and a notification system for AI incidents that are significant to global security.” Reiterating his September 12th call for an industry-wide safety collaboration, Amodei pointed out that Anthropic committed to embedding external evaluators “similar to a food inspector” and recommended other companies do the same. “Some have already agreed to adopt this measure.”

Besides calling for global cooperation between governments to set international standards, Amodei also offered two other specific ideas:

Amodei: We should begin with narrow agreements that ever member can support, such as a ban on using AI to make biological weapons or permitting your AI technologies to be used to make biological weapons….

We should build evaluation and verification systems that keep pace with AI development so that states can have visibility into frontier model capability and can verify each other’s commitments.

“No leader, no company, and no nation can manage this alone. We commit to working with governments in this room on this urgent work.”

Read more of this story at Slashdot.

Source link