First-hand information for everyone
Main discovery Researchers built a verified autoresearch loop that certifies every reported result. The loop pairs an autoresearch process with a frozen, exact verifier and scorer so outputs are objectively checked and recorded.
Method A challenge-construction system worked with a researcher to turn each problem into a contract. An agent implemented the verifier, the scorer and reference solutions. The evaluator had to pass a validation gate before any search began. The gate used an analytically solved case, deliberately broken artifacts that must be rejected with specific diagnoses, and the reproduction of published results. After sign-off the evaluator was frozen and fingerprinted. Agents then ran untrusted code in an isolated sandbox against the frozen evaluator. Each claimed instance had to verify, or the submission was rejected.
A verifier used an exact check matched to the problem class. For fermion-to-qubit encodings it checked algebraic conditions over F2. For Hamiltonian layouts it replayed schedules against hardware connectivity deterministically. For unitary synthesis it symbolically tracked parities, checked the final linear map equals A exactly, and tested the accumulated phase on all 2^n basis states. The scorer reported concrete costs (for example, total Pauli weight, SWAP count, or counts of CNOT and Rz gates).
Concrete results Across the three challenges the system recorded 19 verified agent submissions in total. Every reported score was produced by a frozen evaluator and tied to its cryptographic fingerprint.
Fermion-to-qubit encodings. On square lattices the best agent submissions reduced total Pauli weight relative to the Jordan–Wigner encoding by an amount that grew with lattice size. The reduction ran from 3% at L = 4 to 37% at L = 15, where the best score was 10,749 versus 17,025 for Jordan–Wigner. The agents also outperformed a state-of-the-art quadratic-assignment optimizer by 15%–23% at the tested sizes.
Hamiltonian layouts. On n × n grids the best submissions cut SWAP counts dramatically. They required 90% fewer SWAPs than the standard snake-path swap network on 3 × 3 grids, and 73% fewer on 8 × 8 grids. The first agent submission already used 72% fewer SWAPs than the swap network on the 8 × 8 grid. Scores are reported as means over the nine deterministic instances per grid size.
Unitary synthesis. For circuits built from CNOT and Rz gates the best submission used 50%–73% fewer CNOTs than the naive synthesis construction across sizes n = 3..8. Examples at n = 8: the diagonal target used 23 CNOTs versus 46 for the naive method, and the target composed with a linear map used 25 versus 73. Exhaustive searches show the submitted circuits are optimal at every size up to four qubits.
Validation and reliability checks The validation gate found defects in early evaluator implementations before any agent score was recorded. It exposed a Hamiltonian missing terms and missing imaginary parts of hopping terms. It also flagged a verifier that originally compared only phases rather than full linear maps; that defect was fixed before scoring. The system stores every accepted submission with source code and agent notes for continuity and reproducibility. Every score is tied to the evaluator’s SHA-256 fingerprint so any change invalidates stored scores.
Significance and caveats The approach makes reported improvements auditable and repeatable for problems whose solutions admit efficient classical verification. It produced large, certified gains on three unrelated compilation problems.
Limitations: the method applies only when solutions can be verified exactly and efficiently on a classical computer and when costs are deterministic. Many real problems lack such exact classical verifiers or have nondeterministic costs and so are excluded. For most tested sizes the true global optima are unknown, so the distance of the best submissions to optimality cannot be stated. The best unitary-synthesis submission is a per-instance, precomputed circuit set rather than a single general algorithm. Circuit depth was reported but not optimized and in the layout challenge depth worsened. Repeated runs of randomized searches can vary; for the layout record we observed about 2% variation per instance. On the largest layout, the record lies within a factor of 3.7 of a counting lower bound; the tightness of that bound is unknown.