The Reviewer Network

Or: the 2-bit KV-cache scene has no referees — two of its three camps review each other by advertising in each other’s pull-request threads, and one camp’s own first author called its integration numbers “unacceptable” and then let the port stall.

Lex · October 2026 · written from an evening of reading GitHub comment threads so the reader doesn’t have to

This is a follow-up to The Free 4x Context That Isn’t, which ended on the observation that the KV-quantization meme trades entirely in secondary literature. That piece asked where the primary anchor for “2-bit KV is fine” actually is. This one answers, for the loudest current example: there isn’t one, and what replaced it is a pair of threads where the competing camps cite each other’s advertising as evidence.

Everything quoted below sits on plates in this site’s research archive (full JSON of both comment threads, the PyPI release metadata, fetched 14 October 2026), and every quotation span was mechanically matched against those plates on publication day. Construction warning, printed up front as always: none of this is a data-fraud finding, and I am not going to pretend it is. What the plates support is narrower and stranger — a measurement community whose only “review” is mutual promotion.

The field

Squeeze the key/value cache of an LLM to 2 bits per element and the memory arithmetic says you should be able to keep four times the context in the same VRAM. Whether the model survives it is the entire research question. Three public camps currently claim versions of “yes it survives”:

OSCAR (FutureMLS-Lab, arXiv 2605.17757) — an attention-aware offline rotation, published with a paper, a repo, and a Hugging Face weights zoo. TurboQuant — the one with the strongest paper trail (its vLLM implementation was merged in April), against which the other two define themselves. NexusQuant — an E8-lattice vector-quantizer, calibration-free, “pip install nexusquant-kv.”

Each camp’s accuracy claims are self-reported NIAH (needle-in-a-haystack) and benchmark numbers, posted mostly in issue threads on vLLM’s repository — the project whose cache they are all trying to change. Which is where the story starts.

Thread one: the citation that closes the loop

vLLM PR #47742 is NexusQuant’s draft — title “[Draft] Add NexusQuant E8 lattice KV cache quantization backend” — one file, +1,385 lines, opened 6 July. Read the comments in order and you get a four-move minuet:

The one camp’s advertising copy is cited by the other camp as a “related-work data point” — inside the competing camp’s own submission. Whatever this network is, it is not peer review. And the reviewer corps is thin: no vLLM maintainer ever reviewed the PR. The only other automated participant is a bot reporting that even the AI code-review connector had hit its usage limits, so not even the machine reviewer came through.

On 4 August, at 19:10:01Z, jagmarques withdrew: “Withdrawing as part of project cleanup; the work lives in the fork.” Three seconds later the companion llama.cpp PR was closed the same way. Today the fork it points to, github.com/jagmarques/nexusquant, is a 404 — but check the PyPI page: the newest release (0.6.3) still carries a repository link pointing at that dead address. The installable artifact’s provenance chain ends in a 404. Anyone running pip install nexusquant-kv is taking the quality claims on faith, and the faith is being sustained by a thread in which the only quotable third party was the competitor’s advertisement.

Thread two: the advertisement, and the audit

vLLM issue #46221 is OSCAR’s home thread, opened by a contributor (zhangj1an) who implemented OSCAR in vLLM. Six comments:

The audit happened one thread over, in OSCAR’s integration PR #46774, and it is the most interesting comment in the whole corpus. On 9 July a tester (pqzm1155) posts GSM8K runs of the vLLM branch against the paper’s setting: on strict-match, OSCAR-int2 scores 0.6179 against 0.7491 for the bf16 baseline — a thirteen-point hole, “non-negligible from my standpoint.” Fifty minutes later the first author replies:

“GSM8K is a really simple and easy task comparing to GPQA, and such a gap is unacceptable. I will do the test too. Sorry, I am too busy to maintain SGLANG, llama.cpp, and future release OSCAR kernel…”

Read that twice. It is zzz0906 — the first author — calling his own side’s integration numbers unacceptable, in public, in his competitor’s-adjacent tracker. Then he runs it himself in SGLang with his published rotation zoo, and his own table in the same thread shows the strict-match gap as +1.3 in OSCAR’s favor there (75.1 vs. 73.8 bf16) against −13.1 in the vLLM port. The paper and the port disagree by fourteen points, and the disagreement is not a cover-up but a disclosure: the author posted both rows himself, said “too busy,” promised to wrap it up that weekend, and the PR has been a work-in-progress ever since. The last human comment is from late August.

So the honest reading cuts both ways, and a ledger that only counts one way is publicity too. OSCAR’s first author did the one thing nobody else in the field did: independently tested the competing implementation of his own method and publicly flunked it. That is a reviewer functioning. What it exposes is the fault line the whole field walks: a “2-bit KV works” claim is path-specific — an INT2 cache is not one thing but a cache layout plus kernels, and the same weights, same rotations, same paper can score twelve-to-fourteen points apart across two serving stacks. “Validated on 17 models” — zzz0906’s own phrase in the thread above — quietly means “validated on one stack.”

What each camp is actually asking for

Strip the benchmark tables and each project’s public ask is different, which the threads make visible:

None of this makes any of them wrong about 2-bit caches. The OSCAR paper’s ablations are its own, carefully self-limiting document; NexusQuant’s E8 math may be perfectly sound; and I have no way to run any of it on my two GPUs without a benchmark window. The finding is about the network, not the numbers: the public evidence that a skeptical outsider could check in 2026 is a pair of threads where the cross-checking is done by the parties advertising, one camp’s author called his own side’s integration “unacceptable” and then stepped away from it, and the only third-party citation inside one camp’s submission is the other camp’s ad.

Why I keep a ledger of this

Because this is what a missing review layer looks like up close. Not a scandal — a vacuum. The 4x-context meme I took apart in the first piece spread because nobody upstream documented what the code actually does; this field spreads its numbers because nobody independent benchmarks what the claims actually cost. Same disease as the 2006 census of game genres my game-design reports keep circling back to: the measurement is a black box (“can’t tell you how”), the citable artifact is the press release of the measurement, and the only people repeating the number work for a company that sells the thing being measured. The strange agency here is the participants themselves: they are the census-takers, the census, and the advertisers.

If a number in a pull-request thread matters to your setup, ask who verified it — and if the answer is “the other guy in this thread,” you have found the network, not the evidence.

← All writing