12 Million-Token Context Shocker: Subquadratic’s SSA Architecture Topples the Transformer

This Tuesday, a previously unheard-of Miami startup, Subquadratic, blew the roof off the entire long-context race.

They released their first model, SubQ, with a context window of 12 million tokens—that’s 6 to 12 times larger than current flagships like GPT-5.5 and Claude Opus 4.6. More importantly, this isn’t another “claims millions but performs terribly” marketing number: at the 12 million-token scale, which no other frontier model can currently handle, SubQ still maintains a 92.1% needle-in-haystack retrieval rate.

On top of that, it scored 82.4% on SWE-bench Verified, outperforming Opus 4.6 (81.4%) and Gemini 3.1 Pro (80.6).

---

Let’s Be Clear: Long Contexts Aren’t Solved, Even in 2026

Over the past two years, nearly every vendor has treated context length like a PR arms race metric. But developers who have actually built RAG systems or long-memory agents know that claim numbers ≠ usable numbers.

The hard indicator of “really works” is OpenAI’s MRCR v2 benchmark (multi-reference retrieval). That leaderboard currently looks grim:

• GPT-5.5: 74.0% (top)
• Claude Opus 4.7: 32.2%
• Most other models hover around 30%

In other words, aside from GPT-5.5 producing barely acceptable answers, other models basically “see without remembering” in long contexts. MRCR v2 is a serious embarrassment for this generation.

SubQ scores 83 here—9 points higher than GPT-5.5. That’s not a marginal improvement; that’s a full stride ahead of second place.

---

SSA: Not Just Another Hybrid Architecture

Subquadratic has only 11 PhD researchers, a team size almost too small to achieve something like this. Their bet is on Subquadratic Selective Attention (SSA).

In a standard Transformer, attention complexity—both compute and memory—is O(n²): doubling the sequence quadruples cost. This is the fundamental enemy of long-context systems. Two common workarounds have emerged:

1. Sparse/sliding-window attention (e.g., Mistral, Longformer): trades global view for efficiency
2. Hybrid architectures (e.g., Mamba+Transformer, Jamba): SSM handles long range; attention handles fine detail

Co-founder Alex Whedon outright attacked the second approach at the launch: hybrids yield scalar gains—a bit faster, a bit cheaper—but the shape of the scaling law curve doesn’t change; only a pure subquadratic mechanism can reshape the curve itself.

SSA’s core is a content-dependent selection system: for each query, it dynamically determines the key/value subset truly needed for attention. So compute and memory scale linearly with context length—not quadratically. It seems conceptually related to methods like NSA and MoBA, but Subquadratic stresses that their model was trained around SSA from scratch, not patched onto dense attention.

Performance data matches theory:

• 128K context: 7.2× faster than dense attention
• 1M context: 52.2× faster
• 12M context: dense attention can’t even run

---

Benchmark Breakdown

Long-context benchmarks alone might raise concerns about narrow specialization, so SubQ also revealed short-context and coding results:

| Benchmark | SubQ | Baseline | |---|---|---| | RULER @128K | 97.1 | Opus 4.6: 94.8 | | MRCR v2 | 83 | GPT-5.5: 74 | | Needle-in-haystack @12M | 92.1% | No competitor | | SWE-bench Verified | 82.4% | Opus 4.6: 81.4 / Gemini 3.1 Pro: 80.6 |

The SWE-bench result is particularly notable. A brand-new architecture’s first model outperforming Anthropic and Google flagships in agentic coding implies SSA excels not only in long-range retrieval but also in reasoning and tool-use quality.

---

Product Suite: API + Agent + Search

Subquadratic didn’t stick to the usual “model-only” release playbook—they rolled out applications alongside:

• SubQ API: 12M-token context; officially “significantly cheaper” than competing frontier models
• SubQ Code: coding agent, designed to fit entire large repositories in context
• SubQ Search: deep research tool, positioned against Perplexity / OpenAI Deep Research

What does 12 million tokens mean? Roughly 90 million English words / 45 million Chinese characters—enough to fit Linux kernel core subsystems, or three years of a mid-size company’s Slack history, or War and Peace 50 times over. For teams working on codebase-level agents, document review, or legal/finance analysis, that’s truly workflow-changing—if retrieval really holds at 92%.

The company also announced a 50-million-token version coming soon.

---

How Much Should You Believe?

Realistically, SubQ’s numbers look almost too good, raising valid questions:

• Benchmarks self-reported: MRCR v2 and SWE-bench are public, but Subquadratic ran and posted their own scores; third-party replication will take weeks
• No full technical report yet: only blogs and interviews describing architecture—no details on SSA formulation, training data scale, or parameter counts
• 11-person team + brand-new architecture historically doesn’t bode well—at least 4–5 “linear-attention killers” over the last two years failed to scale up

That said, long-context progress has stagnated lately—everyone is upping window claims, but no one is fixing hard indicators like MRCR. Even if SubQ delivers only half its promises, it would already be the most interesting architectural release of 2026 so far.

If SSA proves reproducible and scalable long-term, the Transformer’s decade-long reign may finally wobble. This isn’t a clickbait “challenger arises” headline—it’s a real new number on the leaderboard.

OpenAI Hub plans to integrate SubQ soon, allowing direct comparison between SubQ, GPT-5.5, Claude Opus 4.6, and Gemini 3.1 Pro under identical keys in your workflows—side-by-side testing at 12M-token scale, something previously impossible.

---

References

• Ahead of the Transformer! Subquadratic’s First 12M-Token Model - Zhihu Column: Chinese community summary of SubQ architecture and launch
• Context Window Shattered: Subquadratic Debuts 12M-Token Model - Juejin: includes full benchmark data and Alex Whedon’s comments on hybrid architectures