IMO 2026: Models Win on Scaffolding

Newly Released Problems Became a Pop Quiz for Large Models

Soon after the official IMO 2026 problems were released, a series of real-world tests targeting mainstream large models quickly followed. Unlike datasets such as GSM8K and MATH, which have been repeatedly benchmarked, this test directly used the six new problems from this year’s International Mathematical Olympiad to compare the performance of cutting-edge closed-source models, Claude Sonnet, Claude Opus, and the open-weight model GLM under different reasoning environments.

The most noteworthy result was not that yet another model achieved a near-perfect score, but that the same model can appear to become an entirely different model when run in a different way.

According to the testers, two frontier models identified by the code names “sol” and “fable” achieved perfect or near-perfect scores regardless of the harness used—that is, the reasoning scaffolding responsible for prompting, parallel sampling, result verification, and answer synthesis. Claude Sonnet and Claude Opus performed relatively poorly when answering directly through the web interface, improved significantly after switching to Anthropic’s own Claude Code, and scored even higher when connected to AutoFyn, a multi-agent framework developed by the research team.

GLM showed a similar pattern: the bare model performed roughly on par with Sonnet without scaffolding, while connecting it to AutoFyn also produced a substantial improvement. However, even with multi-agent orchestration, it still failed to catch up with the two best-performing frontier models.

One easily overlooked detail must be emphasized here: the public discussion provided only model code names and overall trends, while the complete figures appear in the research appendix. The Sonnet and Opus models in the evaluation should also not be arbitrarily mapped to specific snapshots without version information. These results are therefore better suited to examining differences caused by system design than to producing a “model leaderboard” with scores precise to the single point.

The Value of the IMO Lies in More Than Just Difficulty

The IMO consists of six proof-based problems each year. Each problem is worth 7 points, for a total of 42. It does not require university-level knowledge such as calculus or linear algebra, but it does require contestants to discover structure within limited conditions and construct a chain of proof that can be scrutinized step by step.

For large models, this is far more demanding than outputting a numerical answer.

A single competition problem may require several things at once:

• Identifying a genuinely useful invariant, construction, or auxiliary line;
• Rejecting approaches that seem plausible but cannot be completed;
• Translating intuition into a proof without logical gaps;
• Handling degenerate cases, boundary conditions, and the scope of quantifiers;
• Maintaining consistency in notation and assumptions across thousands or even tens of thousands of tokens.

Traditional mathematics benchmarks often check only the final answer. Even if a model’s intermediate derivation is riddled with flaws, it can still receive credit if it happens to output the correct number. The IMO is closer to code review: it is not enough to see whether the program prints the expected result; one must also verify that the implementation is actually correct and covers edge cases.

New problems from the current year offer another advantage: they greatly reduce the likelihood of training-data contamination. A model may have memorized classic theorems, patterns from previous competitions, and common techniques, but it is unlikely to have encountered the original 2026 problems during pretraining if they were only just released. Testing as soon as possible after the problems become public is effectively a relatively clean, closed-book exam for the model.

However, “new problems” do not guarantee the complete absence of contamination. If testing occurs some time after the problems and solutions have been published, web search, retrieval caches, post-training data, and server-side updates may all introduce information leakage. A rigorous test should at minimum record the model snapshot, run time, internet-access status, system prompt, and complete trajectory. Otherwise, claims of first-time independent problem-solving are difficult to verify.

What the Web Interface Measures May Be Only the Product Default

The most meaningful conclusion from this test is that a web application can no longer represent a model’s upper limit.

A web interface usually gives the model only limited thinking time and a single opportunity to answer. It may use a hidden system prompt and may also restrict output length, tool use, and the number of samples. For ordinary question answering, this is a reasonable trade-off between cost and user experience. For IMO-level proof problems, however, it is equivalent to requiring contestants to submit their answers immediately after seeing the problem, without using scratch paper or reviewing their work.

Agent environments such as Claude Code are different. The model can maintain a working context, repeatedly read and write intermediate results, invoke tools, and adjust its approach after failure. AutoFyn goes one step further by assigning different agents to exploration, proof construction, refutation, scoring, and synthesis.

The three operating modes can be understood as follows:

1. Web interface: One person answers orally on the spot, and the first answer is the final answer;
2. Official agent tool: The person is allowed to use scratch paper, check steps, and answer again;
3. Multi-agent harness: A group of contestants solve the problem independently, after which reviewers identify errors, merge the solutions, and finalize the answer.

The model parameters remain unchanged, but the available test-time compute, sampling breadth, and opportunities for error correction are entirely different. Naturally, the final scores may differ dramatically.

This also explains how “a model solved only two problems through the web interface” and “the same model approached the gold-medal cutoff in an agent system” can both be true. The two tests are not measuring the same kind of product: the former is closer to default reasoning capability, while the latter measures the model’s system-level ceiling as a search and proof engine under a given budget.

Multi-Agent Systems Are Not Magic; They Essentially Trade Compute for Success Rate

The core of frameworks such as AutoFyn is not mysterious. When facing a highly difficult proof problem, the system can generate multiple candidate approaches in parallel: one agent tries proof by contradiction, another searches for invariants, another handles special cases, and still another focuses specifically on finding logical flaws. A synthesis agent then selects the more reliable route and rewrites it as a formal solution.

This addresses one of the most typical weaknesses of language models: the first generation tends to commit prematurely to a particular approach, and the model has difficulty actively overturning conclusions it has already written down.

Parallel exploration reduces the risk that a single sample will go down the wrong path, while verifiers allocate part of the compute budget to rejecting incorrect answers. From an engineering perspective, this is fundamentally no different from unit testing, static analysis, and code review in software systems.

But it also has clear costs:

• Token consumption may be several or even dozens of times higher than that of a single call;
• Generating multiple long proofs in parallel can significantly increase latency;
• The synthesis model may mistakenly reject a correct solution written in an unfamiliar style;
• If multiple agents share the same model, they often produce highly correlated errors;
• Without a reliable formal verifier, “most models agree” still does not mean the proof is valid.

The improvement produced by a harness is therefore not free model intelligence, but rather a higher success rate achieved by investing more inference compute per task. Comparing models without disclosing the number of samples, total tokens, maximum context length, tool permissions, and retry budget is like publishing race-car lap times without mentioning the tires, fuel load, or modification rules.

This is especially relevant for everyday API applications. Spending dozens of calls to solve one IMO problem may be acceptable, but customer-service classification, code completion, and batch data processing may not be able to bear the same cost. Research results demonstrate an upper limit; they do not mean that this capability is already economically viable as a default.

GLM’s Results Are More Complicated Than “Open Source Has Caught Up With Closed Source”

The performance of the open-weight GLM model in this test deserves attention. Without additional orchestration, it performed roughly on par with Sonnet running without scaffolding. After being connected to AutoFyn, it also benefited from parallel exploration and verification. This shows that high-quality models do not need to operate exclusively within proprietary systems owned by their labs to perform complex reasoning; third-party developers can also improve task success rates through orchestration.

However, summarizing this result as “open source has caught up with closed source” would still be overly optimistic.

On the one hand, the strongest models, “sol” and “fable,” maintained near-perfect scores even without relying on a specific harness, indicating that the base models themselves still hold clear advantages in strategy search, long-horizon consistency, and error recovery. On the other hand, the fact that GLM achieved a high score after enhancement by an external framework also means the open-source ecosystem can use greater controllability to compensate for some of the underlying gap—for example, by deploying custom verifiers, adjusting sampling strategies, or configuring specialized agents for geometry, combinatorics, and number theory.

The truly valuable comparison is no longer “who scores higher on a single answer,” but the following set of more engineering-oriented questions:

• Under the same one-million-token budget, which system solves more problems?
• At the same score, which system has lower latency and cost?
• Which system can reproduce its results consistently rather than merely sampling the correct path by chance?
• Can incorrect proofs be identified automatically, or can they slip through by virtue of fluent writing?
• After deploying an open-source model locally, does it still retain an advantage in hardware and operational costs?

Without this information, a 42-point score table can show only that “a particular experiment succeeded.” It cannot fully demonstrate who has stronger, cheaper, or more reliable mathematical reasoning capabilities.

Using Another Model as the Grader Is the Weak Link in This Evaluation

The answers in this test were graded by another frontier model. This approach is highly efficient, but it is also the aspect of the evaluation that deserves the most caution.

IMO proofs cannot be graded through keyword matching. A grader must determine whether a step implicitly relies on a conclusion that has not yet been proved, whether special cases have been omitted, and whether a local error invalidates the entire subsequent proof. Frontier models can perform initial screening, but they may still exhibit three kinds of bias:

• Being fooled by proofs that are fluent and well-structured but substantively incorrect;
• Penalizing unconventional but correct solutions too harshly;
• Preferring answers similar to their own generation style, creating correlated bias.

A more robust process would use model grading for rapid screening, followed by blind review of high-scoring and disputed answers by human experts in competition mathematics. The per-problem answers, grading rationales, and review records should also be made public. Where feasible, some proofs could be translated into formal systems such as Lean for verification. However, formalization itself requires substantial engineering effort and may introduce human guidance during translation, so it cannot simply be treated as a fully automated judge.

In other words, having a model grade proof problems generated by other models is like having a coding model approve its own pull request. It can improve efficiency, but it should not be the sole evidence for claiming that “the model has achieved a perfect IMO-level score.”

What This Round of IMO Testing Really Changed

Over the past two years, leading laboratories have repeatedly demonstrated gold-medal-level model performance on Olympiad mathematics problems. By IMO 2026, the question is shifting from “Can a model solve them?” to “How many attempts and how much compute does it require, and can the answers actually be trusted?”

The lessons from these tests are straightforward for developers:

First, stop conflating the base model with the final system. The same model may cross several scoring tiers when connected to different toolchains. Future evaluations of mathematics, coding, and research agents should report both the model and the harness.

Second, verification is becoming as important as generation. What complex tasks lack most is not another answer, but a way to identify which answers cannot be trusted. A combination of generator, critic, and final judge may be more effective than simply increasing the length of a single reasoning attempt.

Third, the IMO remains a high-value benchmark, but it should not be mythologized. Six problems are too small a sample, and the problem types are concentrated in competition mathematics. They cannot directly represent capability in real-world scientific research, software engineering, or business decision-making. A system may score 42 through massive sampling yet still fail to handle real projects with ambiguous requirements and continuously changing information.

Fourth, future leaderboards must include cost. If Model A scores 35 with one call while Model B scores 42 using 200 parallel samples, both are highly capable, but they are entirely different products in a production environment.

Based on the information disclosed so far, the strongest frontier models still lead in raw capability, while AutoFyn demonstrates that well-designed multi-agent orchestration can significantly narrow the gap between models. IMO 2026 did not provide a simple answer to “Which model is the smartest?” Instead, it revealed a new unit of competition: no longer just the model, but the complete system formed by the model, inference budget, search strategy, verifier, and toolchain.

That matters far more than yet another poster proclaiming a perfect score.

References

• Reddit: We compared different LLMs on IMO 2026: The public discussion of this comparative test, describing the overall performance of different models through the web interface, Claude Code, and AutoFyn, and noting that grading was performed by another frontier model.
• Zhihu: Xiaohongshu’s Large Model Wins IMO Gold With a Perfect Score, Sparking Discussion Over Its Solution to Problem 3: Provides a case study and performance description of a Chinese team attempting this year’s new IMO problems. The reported perfect score should still be treated cautiously and evaluated alongside the original answers and human review.
• Zhihu: Discussion of IMO 2026 and Perfect-Score Testing of Large Models: Summarizes information about models’ first attempts after the release of the 2026 IMO problems and can serve as background on the event.
• Zhihu: From the IMO Grand Challenge to AlphaProof: Reviews the background and development of IMO problem types, formal mathematics, and evaluations of AI mathematical reasoning.