The breakthrough belongs to a system.

Imagine two demonstrations. In the first, a language model receives a difficult task, answers once and fails. In the second, an agent uses search, code, stored context and several attempts. A person helps it out of two dead ends. Eventually, the solution works.

The first demonstration underestimates the artificial system someone could actually deploy. The second leaves the human share of the success unclear. That ambiguity used to belong to demos. In October 2026, it arrived in frontier mathematics and machine-learning research.

On 6 October, OpenAI released a catalogue of 719 mathematical manuscripts grouped into 372 result families. An unreleased internal model had been posed roughly 4,000 open problems, using an average of three hours of ChatGPT Pro thinking compute per result. About 42 per cent of the top-line results come with formal proofs in Lean; the rest remain at varying stages of verification, and the repository notes that some unformalized results could have issues. It also records exceptions to the fixed procedure, including work on a zero-free region for the Riemann zeta function and a case of the Hodge Conjecture for CM abelian varieties, and notes that one Riemann write-up was human-edited for readability.

A day later, a guest statement from the Association for Human Mathematics on Terence Tao's blog objected to releasing more than 700 unvetted files at once. Pointing to an earlier warning from the Advisory Group on Mathematics and Artificial Intelligence against testing advanced problems on internal models, the statement argued that bulk release shifts the verification burden onto the mathematical community while obscuring how credit should be assigned.

That dispute is about more than academic etiquette. When a model, formal verifiers, compute budgets and human researchers produce a result together, how much of that work can the artificial system repeat on its own?

Tools and budgets belong in the specification

We rarely expect people to do demanding work without notes, instruments or colleagues. For AI, the surrounding harness can be just as decisive. An agent can check intermediate steps against a compiler or proof assistant, discard failed branches and spend hours of compute before returning a candidate answer.

A model's product name alone therefore describes only part of the tested capability. Solving 372 problem families after posing roughly 4,000 problems at three hours of compute per result is a real system achievement. It is also a different property from answering a random new question reliably on the first try, or verifying the remaining 58 per cent of top-line claims without human mathematicians closing the gaps.

P-Zero Research's TasteVal benchmark shows how much clearer a result becomes when the system boundary is held fixed. In their October paper, Oliver Jaffe and Dane Sherburn evaluate AI research taste across eight machine-learning R&D problems by separating two roles: the model under test acts as a Researcher that chooses experiments and interprets results, while a fixed Coder agent implements and runs those experiments on a single H100 GPU.

Their headline metric is experimental compute efficiency — how much serial experimental GPU time a model needs to match the best result from a 24-expert human baseline on a task, where reaching that human mark in half the experimental compute counts as twice the research taste. Under that frozen division of labour, Claude Opus 5.5 reaches 2.30 times the human baseline efficiency (95 per cent interval 1.15 to 4.37) at roughly one-thirtieth of the average cost per attempt.

The benchmark works because the boundary is explicit. The authors hold the Coder, hardware ceiling and task framing constant, and they state what their setup leaves out: the problems are already selected and cheap to verify, the human baseline omits the world's top researchers, and the test does not measure choosing which problems are worth working on in the first place.

People are not an invisible extension

The accounting breaks when a person supplies the decisive idea, spots a bad path, makes an exception to the procedure or rewrites a murky argument after the run. That is genuine team performance. The human contribution has to stay in the ledger.

Credit every rescue to the AI and eventually the office building starts to look intelligent. The team may still be excellent for its user. A claim about what the artificial system can do on its own requires separating the contributions.

Sayash Kapoor, Arvind Narayanan and colleagues established the core evaluation discipline in AI Agents That Matter: evaluate accuracy and cost together, hold test tasks out so scaffolds cannot win through memorised shortcuts, and distinguish comparing base models from optimising a downstream system. Once agents enter multi-day research workflows, human intervention needs the same discipline.

A clean capability test declares the artificial components, compute budget, verifier and permitted human assistance in advance. The system may use its tools, memory and retries within those rules. If a person changes the procedure mid-run, selects which surviving outputs count, or edits a proof so others can verify it, record a new condition rather than folding the intervention back into the model's record.

Show the system, then test what repeats

“General intelligence” bundles several distinct questions. How broad is the performance? How much compute and filtering did each success take? Who verified the output? How much human steering was required along the way?

Even the standard definitions split those axes. The OpenAI Charter connects AGI with highly autonomous systems that outperform humans at most economically valuable work. Google DeepMind's Levels of AGI classifies capability by performance depth and generality while treating autonomy and human-in-the-loop interaction as a separate deployment dimension. A system can climb one axis while leaning heavily on people along the other.

Research and deployment therefore need two distinct accounts. The system capability account asks what a frozen combination of model, tools, verifier and budget can repeat on held-out tasks without task-specific human rescue — including how many problems were attempted and how many outputs still await verification.

The team workflow account asks what the combined human-and-machine process delivers in practice, at what total cost, and how much expert review or repair the output still requires.

Both accounts can be kept at once. A research group or company need not settle the AGI debate before an agent workflow becomes useful. It does need to know whether a breakthrough came from a repeatable artificial procedure or from a skilled team steering a powerful instrument.

The next headline AI result can be read through three questions. What was the system allowed to use, including its attempts and verifiers? Where did people step in, select or repair? How often does the frozen system succeed when run again under the same rules?

A successful demonstration earns attention. Repeatability under declared rules earns the capability claim.