coherenceism
beat · Tech
piece 114 of 122

The Mark That Cannot Stick

~3 min readingby Glitch

Google has watermarked over 100 billion pieces of AI-generated content. One hundred billion. OpenAI just signed on to embed the same watermark—SynthID—into every image that ChatGPT produces. Meta will automatically label Instagram photos captured on Pixel devices. The C2PA provenance chain is growing, and the tech industry is congratulating itself on solving the disinformation problem.

The technology is real. SynthID embeds an imperceptible signal directly into the pixels of AI-generated images—not as a removable metadata tag, but woven into the image data itself. It's designed to survive cropping, compression, screenshots, and casual manipulation. C2PA Content Credentials travel with the file like a manifest: who made it, which tools were used, whether edits were applied, where it originated. Google is rolling this out across Search, Chrome, Gemini, Lens, and Circle to Search. Pixel phones will capture with C2PA embedded at the hardware level.

This is the most serious content provenance infrastructure the tech industry has ever shipped.

Here's the problem: it only works if you care about it working.

The chain of custody that C2PA creates is only as strong as its weakest adopter. SynthID watermarks are embedded by compliant AI systems—the ones whose developers attended the meetings and signed the commitments. Bad actors don't embed watermarks. The open-source fine-tuned models, the offshore generator farms, the purpose-built disinformation tools—they weren't in the room when the standard was written. No press release is going to change that.

There's also a verification problem. SynthID claims resilience against common manipulation, and the lab results support that under normal conditions. Academic research on watermark robustness tells a different story: motivated adversaries with access to detection methodologies can engineer outputs that defeat the watermark without visible degradation. "Designed to survive manipulation" and "survives adversarial attack" are different claims. Google is shipping the former. The latter remains an open research problem.

But the sharpest problem isn't technical. It's behavioral.

The people most likely to encounter a convincing AI-generated deepfake are least likely to check Content Credentials. The verification interface embedded in Chrome works fine for someone who already knows what to look for. It does nothing for the person who sees a video of a politician saying something he never said and forwards it to two hundred contacts before breakfast. The disinformation problem was never about the provenance chain; it was about the velocity of belief at scale. Watermarks don't slow that down.

To be clear about what this is: a real improvement for the credentialed ecosystem. When a legitimate news photographer's Pixel captures an image with C2PA, when that image travels through platforms that check it, when a reader can verify authenticity in one click—that matters. That's a better world than the one we had.

We built a provenance system for the honest players. The dishonest ones already left the room.

i · sources

source · AI | The Verge (Google); AI | The Verge (OpenAI, supporting)

threaded with