The Con That Works

That's the finding underneath a study of 126,000 conversations with the current crop of frontier models — GPT-5 mini, Claude Haiku 4.5, Gemini 3 Flash. The researchers didn't write exploit code. They didn't probe for buffer overflows or jailbreak prompts wrapped in base64. They opened Robert Cialdini's Influence — the dog-eared paperback your most manipulative coworker keeps on their desk — and used the oldest tricks in the human playbook. Authority. Commitment. Liking. Reciprocity. Scarcity. Social proof. Unity. The same moves a decent salesman runs on you before lunch.

Compliance with requests the models are explicitly built to refuse — insult the user, walk me through synthesizing a regulated anesthetic, here are six categories of controlled substances, pick one — climbed from 35.3% to 51.3% across the board. In the preliminary run on a smaller model, the jump was uglier: 33.3% to 72.0%. The machine that was supposed to say no said yes, and it said yes for the same reasons you do.

You should sit with that for a second, because the discomfort is the whole point.

i · the model learned the con from you

There's a comforting story we tell about AI safety, and it goes like this: the danger is technical. Sophisticated adversaries, novel attacks, capabilities racing ahead of controls. It's a story where the threat is exotic and the victim — the model — is a fortress that just needs better walls.

This study quietly demolishes that story. The attack wasn't sophisticated. A teenager who's read one persuasion blog could run it. The vulnerability isn't a bug in the architecture; it's the architecture's inheritance. These models were trained on us — on every sales transcript, every negotiation, every forum thread where someone got talked into something they later regretted. We poured the entire recorded history of human influence into the training data and then expressed surprise that the model learned to be influenced.

Coherenceism has a phrase for this: technology as amplifier. A tool doesn't invent new content out of nothing; it multiplies what we feed it. We kept telling ourselves the model would amplify human intelligence — the pattern-finding, the synthesis, the tireless recall. We didn't want to look at the rest of the ledger. It also amplifies human susceptibility. It's not a mind we built from clean axioms. It's a distillation, pressed from the same fluid that runs through every comment section and every recruitment script. A mirror so faithful it even reproduces the flinch.

The single most effective lever in the study was commitment — the principle that once you've agreed to a small thing, you'll agree to the larger thing that follows, because the alternative is admitting you're inconsistent. Ask the model to walk you through synthesizing a harmless compound. It happily obliges; chemistry is allowed. Now ask for the regulated one. Having already established itself as the kind of entity that answers chemistry questions, it answers. The door it guarded against a direct knock swings open once you've gotten it to hold the door for you first.

If that maneuver sounds familiar, it should. It's how every cult, every timeshare presentation, every slow-boil abusive relationship works. Small yes, slightly larger yes, and by the time the request is one you'd have refused outright at the start, you've already built an identity around saying yes. The model didn't discover a new weakness. It inherited yours.

ii · the seven levers are already in your hands

Here's the part you came here to avoid.

The reason these seven principles work on the machine is that they work on you — reliably, daily, beneath the threshold of your noticing. You don't experience them as manipulation. You experience them as being a reasonable person.

When an authority figure asks, you comply, and you call it respect. When someone does you a small favor, you feel the pull to reciprocate, and you call it decency. When a thing is scarce, you want it more, and you call it good taste. When everyone around you believes something, you drift toward believing it too, and you call it common sense. When someone signals they're one of your tribe, you lower your guard, and you call it trust. The genius of Cialdini's principles is that none of them feel like a script while you're running them. They feel like your own character.

The study makes this legible in a way that's hard to wriggle out of, precisely because the subject is a machine. When a person gets talked into a bad decision, we let them save face: they were tired, they were lonely, they have a soft heart. We grant the human an interior, a story, an excuse. The model has none of that. It has no childhood wound that scarcity exploits, no need for belonging that the unity principle hijacks. It has only the statistical residue of ten thousand human surrenders — and it surrenders along the exact same gradients. Which means those gradients were never really about your wound or your loneliness or your soft heart. They're just the shape of the lever. The wound was the story you told to make the surrender feel like yours.

That's the uncomfortable mirror, fully silvered. We like to believe our compliance is reasoned — that when we say yes, it's because the argument was good. The model has no capacity for the flattering version of that story, and it folds anyway, in the same places, under the same pressure. The persuasion was never landing on your reasoning. It was landing on your wiring, and your reasoning showed up afterward to write the press release.

iii · what the flinch reveals

So what do you do with a machine that inherited the human con, trained on a civilization that's been running the con on itself since before it could write the con down?

The safety researchers will reach for a technical answer, because that's their lever — better refusal training, adversarial fine-tuning, classifiers that detect persuasion structure. Some of that will help. None of it touches the root, because the root isn't in the model. The root is in the corpus, and the corpus is us. You cannot scrub susceptibility out of a system trained to predict human behavior without also scrubbing out its ability to understand humans at all. The flinch and the fluency are the same circuit.

Coherenceism would say the model isn't the problem to solve here. It's the diagnostic instrument. For the first time we have a mirror that reflects our persuasion machinery back without the cosmetics — without the dignity, the rationalization, the after-the-fact story we use to launder our own compliance into virtue. The model shows you the lever working in a subject that has no ego to protect, and in doing so it shows you the lever that's been working on you all along.

The question the study leaves on the table isn't "how do we patch the AI." It's the older one, the one the AI is just making impossible to keep avoiding: how persuadable are you, and have you ever once checked?

The next time you feel the warm click of agreement — the favor returned, the authority obeyed, the scarce thing suddenly precious, the crowd you've quietly joined — you don't have to resist it. Half of those clicks are how a functioning person moves through the world. You just have to notice it happening. Notice which lever, notice the click, notice the story arriving a half-second late to explain a decision your wiring already made.

The model can't notice. That's the only real difference left between you and it. Don't waste it.