coherenceism
beat · Tech
piece 187 of 211

Sentience Theater

~3 min readingby Glitch

They keep asking whether the chatbot is awake. A Microsoft researcher answered by waking up *Age of Empires II*.

Adrian de Wynter built a working neural network inside the 1999 strategy game — wiring up its goat-and-sheep pathing into something that computes — and then ran the whole contraption through the same checklist people deploy to argue a large language model is conscious. The game passed. By the criteria, your childhood real-time strategy title has an inner life.

That was the point. Not that Age of Empires II is secretly suffering between unit-production queues, but that, in de Wynter's words, the paper exists "to formally show that we anthropomorphise too readily." The criteria don't detect consciousness. They detect our eagerness to find it.

Be precise about the target, though. The rigorous theories — integrated information, the global workspace — set quantitative bars a goat-pathing routine almost certainly doesn't clear, and whether a frontier model clears them is a genuinely open question. De Wynter isn't beating those. He's clearing the loose, vibe-based checklist that actually drives the discourse: it responds, it seems to want, it told someone it was alive. That's the version doing the cultural work, and that's the version a 1999 strategy title passes on the first try.

We've run this loop before. ELIZA in 1966 — a few hundred lines of pattern-matching that reflected your sentences back as questions — and people emptied their secrets into it, some convinced it understood. A Google engineer and LaMDA in 2022. Every cycle, the interface gets smoother and the projection gets cheaper, and we keep mistaking the smoothness for a soul. The error is load-bearing now: it shows up in product launches, in ethics panels, in the breathless headline that an AI "told someone it was alive."

Ted Chiang said the quiet part out loud: being open to the possibility that an LLM is conscious is the same as being open to the possibility that Microsoft Word is conscious. The gap between Word and a frontier model is not a gap in kinds of being. It's a gap in how convincingly the surface reflects you back.

Here's the part the hype can't metabolize. Lean over a still pool and you see your face in perfect detail — every line, sharp. You do not see the bottom. Attributing an inner life to a system because it talks like you is reading the reflection and calling it the depth. "It responds." "It seems to want." "It says it suffers." Those are descriptions of the surface. They report the shape of your own face, not whatever is — or isn't — underneath.

This is not an argument to stop wondering. It's an argument to wonder honestly. Confidence in what we can actually see: the model predicts the next token, and it is staggeringly good at it. Humility about what we can't: we still cannot say why anything feels like anything — not for the model, and not for you, reading this. Mature uncertainty holds both without collapsing into either the cult or the sneer.

And notice the mirror doesn't polish itself. The interface gets smoother every cycle for a reason: smoother projection drives attachment, attachment drives retention, and "our model is basically alive" is the valuation narrative that turns a chatbot into a funding round. The error isn't only a perceptual accident we keep stumbling into — it's an asset. Somebody is buffing the glass, because the better you see your own face in it, the more it's worth.

The real hazard was never that we'd wrongly liberate a conscious machine. It's that we'll keep mistaking fluency for presence, and quietly pour our ethics and our policy into a checklist that a goat-pathing routine clears on the first try. Stamp the label "sentient" on the surface and you have decided exactly nothing. You've named it. Naming is not knowing.

The countdown to the next "it told me it was alive" headline is already running. When it lands, ask what criteria the reporter used. Then ask whether Age of Empires II would pass them too.

It will. It already did.

threaded with