How Lumen actually works
Four models, one engine. This is the mechanism underneath — how a single router picks from a shared pool of lanes, how the system learns from real traffic without ever billing you for the experiment, and exactly what it keeps.
01 · The engine
One engine, four tiers
Flash, Standard, Pro, and Ultra are not four different machines. They are four points on one utility curve.
All four tiers share the same router, the same pool of internal lanes, the same learned quality-ratings ledger, and the same duel-based learning plane. What a tier configures is exactly one thing: how the final lane is chosen from that identical, already-scored slate of candidates.
No tier owns a mechanism; no mechanism knows which tier invoked it.
So a tier is a selection policy, not a separate stack. Flash sits at the cost-sensitive end of the curve — the strongest lane whose price clears a tight budget. Ultra sits at the quality end, and can convene a council when a request looks genuinely hard. Standard and Pro are points in between. Same engine, different dial.
It wasn't always built this way, and that's worth saying plainly. Tiers used to mean three genuinely different machines. We collapsed them into one engine with different selection policies — a real architecture change, and a deliberate one. It means every improvement to the engine improves all four tiers at once, and no tier can quietly drift onto a worse code path than its neighbours.
02 · Learning
Shadow duels
The engine learns by running true, same-request experiments on live traffic — where the experiment never touches your answer or your bill.
When a request carries enough information demand to be worth learning from, the engine serves it twice.
- 1A primary lane streams the real answer to you, exactly as it normally would. Nothing about your response changes.
- 2In parallel, a shadow lane answers the same request — off your path, invisible, never shown to you.
- 3A judge compares the two answers pairwise, position-swapped: it reads both answers in both orderings, so the order they arrive in can't bias the verdict. The judge is family-disjoint — never drawn from the same lane-family as either answer it grades, so it can't favour its own kind — and it is validated offline against a fixed answer-key set before it's ever trusted on live traffic.
- 4Winner and loser nudge the quality ratings at that neighbourhood of requests. Then the two answers are dropped.
The shadow eats the cost of trying something uncertain — not you.
Three properties fall out of this design. There is no exploration tax: the risk of trying an unproven lane is paid by the invisible shadow, never by your latency or your answer. The comparisons are true counterfactuals — the same request, at the same moment, on real production traffic, not a synthetic benchmark. And nothing is kept: the duel is scored, and only anonymous scalar ratings survive it — which is the next section.
03 · Governance
Gates in, rollback out
A lane earns its way into live traffic on machine-checked evidence — and loses its seat automatically the moment that evidence turns.
A new internal lane doesn't serve real traffic the moment it's added. It enters an examining stage, where it accumulates real duel evidence — judged head-to-head against whatever is actually serving — with provably zero effect on what you receive. It is being measured, not deployed.
Promotion to a serving seat requires machine-checked evidence, not a hunch:
- ✓A minimum sample of judged duels — enough signal to trust the result, not a lucky streak.
- ✓A win rate at or above the current pool's median — a newcomer has to be at least as good as a typical incumbent, not merely clear some fixed floor.
- ✓It ships as a reviewed, auto-generated config change — the same version-controlled path as any other change to the system, never a silent runtime write.
And the gate runs in both directions. If live telemetry on something already promoted starts to degrade, it auto-rolls back to the last known-good configuration — no human in the loop, no waiting for a report.
Quality is a gate on the way in and a tripwire on the way out.
04 · Retention
What the learning keeps
Everything above runs on your real traffic. So it's fair to ask what survives it.
The answer is: only anonymous quality scores. Never prompts, never outputs, never session identifiers. The primary answer, the shadow answer, the judge's transcript — none of it becomes a record.
This is a structural property, not a policy promise layered on top. Anything that touches content lives in short-TTL, non-durable storage — it physically isn't kept long enough to become a record. The scalar ratings that update the ledger carry no way back to the request that produced them.
Only the score is durable. The content that produced it isn't.
05 · Latency
The latency law
Every tier streams its answer the moment its lane produces it. The only wait you ever incur is one you explicitly chose.
Nothing waits on the serving path that you didn't explicitly buy.
In every tier and every mode, the engine streams the answer as soon as the chosen lane produces it. There is no verification pause, no second-guessing re-generation, no quality checkpoint sitting between the lane and you. The learning plane is entirely off to the side — that is what "shadow" means.
There is exactly one legal exception, and it's opt-in: Ultra's council. It convenes only when a request genuinely looks hard or unfamiliar — the model's own confidence signal says so — and then fans out to multiple lanes, with an independent judge picking the winner before you see an answer. You pay the council's latency only when the system itself is uncertain.
So easy Ultra traffic is exactly as fast as Pro. You aren't buying a slower tier; you're buying a fallback that fires only when it's warranted. And the council's share of traffic shrinks over time — as the family's ratings converge, the system is uncertain less often, and needs the council less.
06 · Streaming
Resumable streams
A dropped connection mid-answer no longer means starting over — and you don't pay to regenerate what was already produced.
A network blip, a rolling deploy, an edge hiccup — any of them can cut a stream mid-response. Every streaming response can carry a resume capability so that doesn't cost you the answer. At the start of the stream the server hands back an x-sylphx-resume-token header, and every event carries a sequence id. If the connection drops, you reconnect with that same token and tell the server the last event you saw — using the standard Last-Event-ID header — and it replays exactly what was already produced, picking up from where you left off, plus a clean signal for anything the original generation hadn't finished yet.
Last-Event-ID — so the server replays only what the client hasn't seen. The exact bytes are below.# Start of stream — the server hands back a resume capability HTTP/1.1 200 OK content-type: text/event-stream x-sylphx-resume-token: <token> (opaque, expires in minutes) id: 41 data: … the answer streaming to you … id: 42 data: … more of the answer … # ── connection drops here ── # Reconnect: same token, and the last id you actually saw. # Standard SSE — the Last-Event-ID header, no bespoke protocol. Last-Event-ID: 42 id: 43 data: … picks up exactly here. Nothing already sent is regenerated. …
Two honest caveats. This uses standard SSE reconnection semantics — the same Last-Event-ID mechanism a browser's EventSource already speaks — not a bespoke protocol you have to implement. And it is a reconnect-and-continue from a journal, not a live hand-off of a still-running generation between machines: it replays what was already generated and journaled. That journal is capped and short-lived — kept on the order of minutes, just long enough to serve a reconnect, then gone. Never durably stored.
07 · Evergreen
Evergreen by construction
Capability only ratchets up — behind the same model id you already integrated against.
New lanes are admitted, rated, and promoted as the market moves. Improvements land as audited releases, automatically, behind the same model id you integrated against on day one — you don't chase version numbers to stay current.
And "capability only goes up" is enforced in code, not asserted in a blog post. Take the advertised context window:
A lane can only hold a serving seat if its real capability meets or exceeds what's advertised. The advertised number can only be raised, never silently lowered — and it can only be raised once every serving lane already clears the new floor. The same discipline you saw for quality — gates in, auto-rollback out — applies to the specs on the box: the number you integrated against is a promise the code keeps, not a ceiling that can quietly slip.
08 · Certification
How we keep ourselves honest
Tier separation isn't asserted once and left alone. It's a hypothesis the production data has to keep earning.
The whole ladder rests on one claim: that Flash, Standard, Pro, and Ultra actually separate in quality, in that order. We don't take that on faith. It's verified continuously from judged production traffic — the same duels that teach the engine also measure whether each tier still out-separates the one below it.
A tier holds its slot only while it separates.
If a tier stops earning its place above its neighbour, that isn't a marketing problem to smooth over — it's a signal the ladder has to answer to. Certification, not claims: everything on this page is something the system checks on itself, on live traffic, on repeat.
The full trajectory of these certifications, run over run, is on evolution. See the four tiers side by side on the models page, or how the flat per-tier rates work on pricing.