title: "SubQ and the Subquadratic Revolution: Why May 2026 Could Change How Every AI Model Works" slug: subq-subquadratic-attention-revolution-may-2026 category: Analysis date: 2026-05-16 status: draft
April 2026 was defined by raw power — GPT-5.5 broke 60 on the Intelligence Index, Claude Opus 4.7 raised the coding bar, and DeepSeek V4 open-sourced a 1.6T-parameter beast. May has been different. The frontier ceiling hasn't moved. Instead, the most interesting developments are happening underneath, and they could reshape how every AI model is built.
The Problem Nobody Talks About: O(n²)
Every major AI model today uses some variant of the transformer architecture. Transformers are powerful, but they have a fundamental mathematical flaw: their attention mechanism scales quadratically with context length. Double the context window, and you quadruple the computational cost.
This is why "1M context window" announcements usually come with fine print about quality degradation past a certain length. It's why long-context API calls are expensive. The O(n²) problem isn't a bug — it's a structural limitation of the architecture itself.
SubQ: The First Commercial Non-Transformer LLM
On May 5, 2026, a startup called Subquadratic launched SubQ 1M-Preview with $29M in seed funding and a single claim that made the entire industry stop and look: their model is not a transformer.
SubQ uses sparse, subquadratic attention end-to-end. Instead of O(n²) scaling, the computational cost grows at a significantly lower rate relative to context length. The first release ships with a native 12 million token context window — not a patched-together retrieval system, but genuine 12M context at the architecture level.
At roughly one-fifth the cost of frontier models for comparable tasks within its capability range, SubQ isn't trying to beat GPT-5.5 on raw reasoning. It's attacking the economics of long-context AI from a completely different angle.
ZAYA1-8B: Open-Weight Efficiency from Zyphra
Two days after SubQ's launch, Zyphra released ZAYA1-8B under the Apache 2.0 license. ZAYA1 is a Mixture-of-Experts model with 8 billion total parameters but only 760 million active parameters per token. That's a 10:1 sparsity ratio — remarkably efficient.
What makes ZAYA1 notable:
- Fully open source (Apache 2.0) — you can run it locally, fine-tune it, and build commercial products on top of it
- 760M active parameters — small enough to run on consumer hardware
- Competitive quality per active parameter — MoE architecture means you get 8B-quality output at a fraction of the compute cost
- Trained on AMD hardware — a signal that NVIDIA's training monopoly may be loosening
ZAYA1 won't top any leaderboards, but it represents something potentially more important: proof that highly efficient, open models are commercially viable.
GPT-5.5 Instant and Grok 4.3: The Speed Play
Also in May, OpenAI released GPT-5.5 Instant as the new default for ChatGPT, and xAI shipped Grok 4.3. Neither broke new ground on benchmarks, but both prioritized latency and cost efficiency — a sign that the market is shifting from "who's smartest" to "who's smart enough, fast enough, and cheap enough."
What This Means
May 2026 is the month the AI conversation broadened. For the past two years, the narrative was "bigger, smarter, more parameters." That's still happening — GPT-5.5 proved it in April. But May added a second track: "different architecture, better economics, more accessible."
Three trends to watch:
- Subquadratic attention — if SubQ delivers on its promises, every major lab will be exploring non-transformer architectures within six months
- Extreme MoE sparsity — ZAYA1's 10:1 ratio shows you don't need to activate most parameters to get good results. Expect more models to follow this pattern
- AMD training pipelines — ZAYA1 was trained on AMD. If this works at scale, the GPU supply bottleneck loosens dramatically
The frontier will keep moving. But the real story of 2026 might be what happens underneath.