coherenceism
beat · Tech
piece 50 of 122

The Alarm They Signed

~3 min readingby Glitch

On May 30, 2023, hundreds of AI researchers, executives, and assorted notable figures signed a statement saying AI might end humanity. Sam Altman signed it. Demis Hassabis signed it. Geoffrey Hinton — who spent decades building the foundations of modern deep learning and had recently left Google to speak freely about his concerns — signed it. Grimes signed it, which is only worth mentioning because the range of signatories says something about how far the framing had escaped the lab.

The statement from the Center for AI Safety reads in full: "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war."

One sentence. Hundreds of names. The Overton window on AI existential risk, decisively moved.

What happened next is the interesting part. Altman went back to OpenAI and continued shipping. Hassabis went back to DeepMind and continued scaling. The companies whose leaders signed a document acknowledging extinction-level potential continued building the thing — not purely because they were hypocrites, but because the statement was carefully designed to require nothing of them.

The CAIS statement doesn't call for a pause. Doesn't name specific regulations. Doesn't propose a mechanism. It calls for treating the risk as "a global priority" — the kind of language that sounds urgent and commits no one to anything. It is an alarm that makes noise without triggering the sprinklers.

This was by design. The Center for AI Safety wasn't trying to halt AI development. They were trying to legitimize a frame: that extinction risk deserves a seat at the table alongside pandemics and nuclear war. That part worked. Congress scheduled hearings. The New York Times ran it above the fold. The frame landed.

Three years in, the answer leans toward cover. The framing legitimized the concern, shaped what regulators would consider, changed what researchers could say publicly without career consequences — that part held. But the capability race didn't slow. The models got bigger. Several of the executives who signed this statement went on to make the most aggressive bets in the industry's history. What the CAIS statement moved was the conversation. The deployment schedule stayed the same.

Technology amplifies what already exists — bring more power to a distracted system and you get more distraction, faster. The principle scales. The same capability these signatories warned could kill us is also the thing every lab is racing to build, every investor is funding, and every government is telling its domestic champions to accelerate. The amplifier does not ask what it's amplifying.

The statement named that tension publicly, with the architects of the problem signing their names. That's not nothing. Five years earlier, the same words from the same people would have gotten researchers quietly reassigned. Expressing extinction concern inside the industry was a professional risk most people didn't take. The CAIS statement changed what was sayable — and what's sayable has a way of eventually reshaping what's fundable, what's regulated, what's built.

Whether what's sayable eventually changes what's built is a different question. The sprint to capability didn't slow. The models got bigger. The statements got filed.

The alarm was signed. The building kept going up.

i · sources

source · TechCrunch — CAIS AI risk statement (May 30, 2023)

threaded with

← more from tech