Eight Gigabytes of Sovereignty

And then there is one eligibility rule, which does more work than everything else on the page combined. To be tested, a model must fit inside 8 GB after quantization — KV cache at 8K context included.

That number is the story, and the first thing to notice is where it did not come from.

It did not come from Apple or Samsung. Both benchmark phones ship with twelve gigabytes. The eight-gigabyte line is Artificial Analysis's own criterion, stated once in the methodology and left unexplained — no note on whether it reserves headroom for the operating system, targets a wider install base than two flagships, or reflects some other judgment about a realistic on-device budget. There are perfectly reasonable places that number could have come from. The page does not say which.

Of the 41 quantized builds in the set, 33 ran on the iPhone 17 Pro. The efficiency leader is LFM2.5-2.6B, which turns around that standard 1,024-token prompt in 8.0 seconds using 2.3 gigabytes. Genuinely impressive, and also a description of a very small room — one whose dimensions were set by the people holding the tape measure.

Because the harness underneath all of this is Pipette, which Liquid AI built and open-sourced. Liquid AI also makes LFM2.5, the model topping the efficiency chart. Artificial Analysis states that it independently validated Liquid AI's inference measurement process, which is exactly the right disclosure and exactly the right precaution, and nothing in the record suggests anyone involved is doing anything but careful work.

The structural point does not need them to be. Early benchmarks get built by whoever cares enough to build them, and the people who care that much are usually shipping something. That is the ordinary condition of a measurement discipline before a disinterested party turns up to fund one. But it means the founding move — deciding how big "pocket-scale" is — gets made by participants, and everyone downstream reads the resulting room as the frontier. Eight gigabytes is now the size a phone model has to fit inside, because a leaderboard said so, and leaderboards are how a field learns what to build next.

The case for running models locally is real and worth stating plainly. A model on your device cannot be surveilled at inference time. It cannot be silently swapped for a cheaper checkpoint, throttled during a capacity crunch, or deprecated out from under an application that depended on it. Nothing leaves the phone. Those are properties no hosted API can offer at any price, because the capacity to observe and interrupt is what a hosted API is.

It also cannot be revoked, with an asterisk worth attaching. The weights on your disk are yours. The channel that put them there is not, and a 4-bit quantization of a model you did not train is not something most people will re-derive after it disappears from the hub. Possessing a file and being able to replace it are different sovereignties, and only one of them is in your pocket.

Which points at the larger asterisk. All of the above is true of the model and not of the situation. The model runs in an app Apple can pull, on an operating system Apple updates, on silicon Apple specifies the next generation of, inside a sandbox whose rules Apple writes. You have not left the rental market. You have moved down a floor in it — from renting a model to renting a platform. And the platform has fewer exits. You can swap API providers in an afternoon. You cannot swap the phone in your pocket for one running a different OS without changing nearly everything you do on it.

Ceilings are easy to notice and satisfying to complain about. Floors are the thing you are standing on.

So the useful question is not how much of the frontier fits inside the constraint. It is which function you are actually moving. The evaluation suite makes that split visible. Five tests: BFCL for tool calling, IFBench for instruction following, GPQA Diamond for scientific reasoning, MATH-500 for quantitative work, and AA-Omniscience, which measures world knowledge and the rate at which a model invents it. A 2.6-billion-parameter model at four bits is going to do poorly on that last one. Not poorly in a way that announces itself — poorly in the specific way small models fail, which is fluently and with the same tone they use when correct.

The pocket model is respectable at being an interface: parsing intent, calling tools, following a format, shaping text. It is unreliable at being an oracle. And that turns out to be the right place to draw the line, because the interface is the layer that decides what gets asked, and the oracle was never the part worth trusting to a vendor anyway. A model that will confidently tell you a false fact is not improved by being someone else's model. It is only harder to inspect.

Sovereignty at the edge is real, and the part of it worth having is the smaller part — the layer that shapes the question, not the one that answers it. What that layer gets today is eight gigabytes: a number set by a benchmark, in a room measured by a competitor, on a device somebody else specified.

The models are getting better inside the constraint, quickly. Nobody has yet had to defend the constraint.