Nobody checked which state the quantum computer computed

There’s an argument running in quantum chemistry about whether quantum computers are useful yet. IBM published results in Science Advances last year on iron–sulfur clusters, the metal cores that sit inside enzymes like nitrogenase, and presented them as evidence that quantum hardware now contributes something real to a genuinely hard problem. A group at the University of Southern Denmark published a critique arguing that the quantum samples never beat classical selected-CI once you match the cost. Neither side has moved much since.

I spent a few months checking the one thing both sides skipped.

When you run a calculation like this, you get an energy out. What you don’t get, unless you go looking, is the identity of the state that energy belongs to. Molecules like these have a stack of electronic states sitting very close together, distinguished by how the electron spins line up, and in the iron–sulfur systems that stack spans roughly 15 to 30 millihartree from bottom to top. The accuracy improvements under dispute are the same size. So if your calculation quietly settles on a rung above the one you meant, your reported number is wrong by about as much as the entire effect being argued about, and nothing in the output says so.

There’s a standard way to check: you measure ⟨S²⟩, the total spin squared. For the singlet ground state everyone is targeting, it should come out at zero.

Nobody had measured it. Not IBM, not the critics. So I did.

What came out

For [2Fe-2S], every starting guess I tried converged to the same state, with ⟨S²⟩ around 4.66. That isn’t zero, and it isn’t any clean spin state either; it’s a mixture of several. The energy sat about 13 millihartree above the reference, which is to say roughly the height of the whole ladder.

[4Fe-4S] failed in a more interesting direction. There, ⟨S²⟩ climbs as you enlarge the subspace, from about 5.3 to 7.0. Making the calculation bigger walks it further from the target rather than closer, which is the opposite of how these methods are supposed to behave.

IBM ships a mitigation aimed at this problem, called spin completion, and it does what it says: it makes the spin-symmetric combinations representable inside the subspace. Measured effect on the ground-state energy, though, is under a microhartree. The reason turns out to be structural rather than a tuning failure. In the ground manifold the S² operator’s block comes out proportional to the identity, so completion permits a singlet without ever preferring one. It opens the door and nothing walks through.

Then I ran their actual software. Their published pipeline, qiskit-addon-sqd, as shipped, on a million samples drawn from a converged benchmark state. Their own spin_square() diagnostic reports ⟨S²⟩ = 4.83 after four recovery iterations. Toggling symmetrize_spin on versus off changes the ground energy by less than a nanohartree, at exactly four times the subspace dimension:

                                best energy (Ha)       <S^2>    determinants
BEFORE completion (off)            -116.56010715     4.83432          49,042
AFTER  completion (on)             -116.56010715     4.83432         196,249
delta                                  -0.001 nHa     2.6e-11           4.00x

That is the output of the notebook linked above, run on this page’s author’s machine. You should get the same bytes.

The detail that took me a while to believe: the solver underneath accepts a spin_sq argument that would target the singlet directly, via pyscf’s fix_spin_. The top-level driver has no such parameter, and when you don’t hand it a custom solver it calls the default one with positional arguments only, so spin_sq stays None on every default run. You can reach it by supplying your own solver; I did, and it works. But any benchmark run through the default path and reported without a spin measurement cannot have targeted the singlet, because nothing in that path ever asks.

Their own data

The part I wasn’t expecting was in IBM’s own archive. Their data-availability deposit for the flagship paper contains the raw hardware measurement records: 2.46 million shots on [2Fe-2S], 3.16 million outcomes on [4Fe-4S]. Nobody seems to have run them back through.

Doing that, with their shots and their pipeline, [2Fe-2S] lands near-singlet but 248 millihartree off their own reference, and 99.55% of the raw shots aren’t in the right particle-number sector to begin with. [4Fe-4S] converges to a spin-pure triplet. A clean state, and the wrong one, 1.4 hartree from their own reference value.

Two more things sit in that archive and need no analysis from me at all. Their uniform-random control matches or beats their quantum hardware on every [2Fe-2S] eigenstate they publish. And their largest [4Fe-4S] runs, the ones at subspace dimensions around 10⁸, sit roughly 149 millihartree above their own classical HCI reference file.

How it was done, since it matters here

The first pass through all of this was executed by an AI research agent, Claude, working under my direction. It replicated both sides, built the spin instrument, ran the compute campaign, and wrote the first draft. Since then it’s been through many rounds of adversarial review, and twice, when reviewers pushed back on its reading of IBM’s methods, it corrected itself on the record and re-ran everything downstream.

I’d rather not oversell that. The part I’ll actually defend is narrower: it caught five defects in its own work through validation gates written before the runs, and it retracted its own strongest pro-quantum result once the instrument showed that result was a spin-sector artifact. Whatever else you think about AI doing science, output that retracts its own best finding is not the failure mode people usually worry about. The full timeline, including every self-caught error and both reviewer-prompted corrections, is published in the archive as AUDIT_TRAIL.md. You don’t have to take any of it on trust, which is rather the point.

What would show I’m wrong

I’d genuinely like to know, so here’s what would do it:

1. Exhibit any state in a spin-completed ground manifold of these benchmarks with ⟨S²⟩ below 1.
2. Produce a quantum-sampled subspace at matched determinant count whose spin-identified energy beats HCI or CIPSI.
3. Show me any run of the shipped pipeline on IBM’s archived samples that lands ⟨S²⟩ under 1 and error under 50 millihartree at the same time, without a spin penalty.

The harness for all three is in the archive. If someone does it, I’ll publish that.

What this doesn’t mean

It doesn’t mean quantum computing is fake, and it doesn’t mean the people involved did anything dishonest. Spin contamination in selected-CI methods is known in principle; what hadn’t happened was anyone measuring it on these systems at these scales, where it turns out to be large enough to swallow the result. The limitations section of the paper is real and I’d rather you read it than the hot take: the comparison dimensions are fixed, the [4Fe-4S] reference is approximate, and the large deduplicated sample file IBM published doesn’t preserve shot-level statistics, so part of their numerics can’t be audited by anyone from public data.

What I think should change is small and cheap. Report ⟨S²⟩ with a variance alongside the energy, the way you’d report an error bar. It costs almost nothing and it would have caught this.

The record

• Paper
  doi:10.26434/chemrxiv.15006382/v1
• Data, code and audit trail
  doi:10.5281/zenodo.21359922
• The narrow technical question, on IBM’s own tracker
  qiskit-addon-sqd issue #337

Every number above maps to a file and a one-command regeneration; that mapping is REPRO_MAP.md in the archive. A validation script re-derives all 370 quantitative claims from the raw result files and fails if any one of them drifts. If you check something and it doesn’t reproduce, write to me.