NVIDIA ships a decode chip with a competitor's name on it

NVIDIA said on Monday that Groq 3 LPX, an inference accelerator that carries a competitor's name under licence, has entered full production. The announcement landed on the first full conference day of Hot Chips 2026 at Stanford, where NVIDIA is presenting both its Vera CPU and its Rubin GPU, and it arrived bundled with efficiency figures that are large, entirely NVIDIA-measured, and not yet reviewed by the party whose benchmark produced them.

LPX is not a general-purpose accelerator and is not meant to replace one. It targets a single stage of inference, and understanding which stage is the whole point of the product.

Splitting inference in two

Serving a model involves two phases with opposite hardware appetites. Prefill ingests the prompt and any accumulated context: it is throughput-bound, embarrassingly parallel, and GPUs are good at it. Decode emits the answer one token at a time, each token depending on the last: it is latency-bound and memory-bandwidth-bound, and it is where a GPU spends much of its time waiting rather than computing.

NVIDIA's framing is that agents made this asymmetry expensive. An agent generates tokens across hundreds or thousands of steps, and a small per-token delay compounds through every link in the chain. NVIDIA cites OpenRouter data putting agentic workloads at 15x the token consumption of a simple chat request. So the company has split the two phases across different silicon: Rubin GPUs in the Vera Rubin NVL72 rack handle large-scale context processing, and LPX handles latency-sensitive decode. A rack-scale LPX deployment can hold 256 LP30 accelerators wired together over direct chip-to-chip links, which NVIDIA describes as operating like one large processor for deterministic inference.

This is the same disaggregation idea that inference-serving stacks have been pursuing in software for a while. The change is that it is now a hardware boundary with a purchase order attached.

The numbers, and who produced them

NVIDIA's headline result is 3,400 output tokens per second running Gemma 4 31B with a 100,000-token context, which it attributes to Artificial Analysis benchmarking and calls the fastest performance ever recorded for that model. It also claims 4x better responsiveness than what it calls the nearest alternative platform, without naming the comparator.

The efficiency claims come from a separate NVIDIA post and are the ones that most deserve care. NVIDIA reports that Vera Rubin NVL72 delivers up to 30x higher throughput per megawatt and up to 35x lower cost per million tokens than its own GB300 NVL72, on the DeepSeek V4 Pro model, using the SemiAnalysis AgentX workload of recorded real-world agentic coding sessions with context growth, tool calls and sub-agent spawning preserved. For reference, NVIDIA puts GB300 NVL72 at up to 15x better throughput per megawatt than Hopper on the same model.

Four qualifiers travel with those figures, and NVIDIA states all of them itself:

• The data was measured by NVIDIA, not by SemiAnalysis, whose workload it uses.
• The results are, in NVIDIA's words, "currently pending SemiAnalysis review."
• They do not yet reflect Vera CPU performance for tool calling, a material part of any real agent loop.
• The comparison baseline is NVIDIA's own previous generation, which is a generational claim, not a competitive one.

A 30x generational jump in throughput per watt would be unusual on architecture alone. Part of it comes from elsewhere in the stack: NVIDIA says its DSX MaxLPS power management provisions up to 40% more GPUs inside the same megawatt budget. Treat the number as a vendor figure on a vendor-run test until the benchmark's author says otherwise.

A competitor's name on NVIDIA's box

The oddest detail is in the legal boilerplate at the foot of the press release: "Groq and LPU are used under license from Groq, Inc." NVIDIA is shipping a product branded with the trademark of the startup that spent years positioning itself as the fast-inference alternative to NVIDIA.

That follows the licensing arrangement reported at around $20 billion, under which NVIDIA took a non-exclusive licence to Groq's inference technology and hired founder and chief executive Jonathan Ross. TechCrunch reported on 17 August that Groq has since raised a $350 million Series A at a $3.5 billion valuation, down from $6.9 billion in September 2025, and has repositioned itself as a neocloud running NVIDIA systems across 13 data centres, with plans to grow from 54 megawatts to more than 200 megawatts by 2027.

Which produces the closing line of NVIDIA's own announcement: after Nebius, the first cloud to adopt LPX, Groq itself plans to be among the earliest adopters. The company will buy NVIDIA racks carrying its own licensed brand name. Whatever else it is, it is a clean illustration of how the inference-challenger thesis resolved.

What this changes for developers

Today, nothing. LPX is rack-scale infrastructure sold to clouds, and the first deployment is Nebius, which plans to expose it through Nebius Token Factory. Nebius CTO Danila Shtan makes a point of saying it arrives "through the same API developers are already using, with no migration to a new stack" — meaning there is no LPX SDK to learn and no code to port.

What is worth tracking is second-order. If decode acceleration reaches production clouds at the speeds claimed, the cost and latency profile of long-running agent loops changes more than that of single-shot chat calls, because agents are decode-heavy and their step latency accumulates. That is precisely the workload where developers currently trim scope: fewer sub-agents, shorter reasoning chains, smaller retained context. A meaningful drop in per-token decode latency would loosen those constraints before it does anything for a one-shot summarisation call.

It also reinforces a split already visible in model routing. If prefill and decode run on different silicon with different economics, providers have more reason to price long-context ingestion separately from generation — which is roughly what tiered context pricing and prompt caching already do, expressed in hardware.

Around the announcement

NVIDIA released a second item the same morning: SpaceXAI will deploy NVIDIA Vera CPUs for the agentic AI infrastructure behind Grok. NVIDIA describes Vera as its first CPU designed for AI agents, aimed at the CPU-bound work surrounding inference — tool execution, code running, data processing and orchestration — with 88 NVIDIA-designed Olympus cores, up to 1.2TB/s of memory bandwidth and up to 1.8x faster task completion than x86 CPUs, again by NVIDIA's measurement. SpaceXAI president Mike Nicolls frames it as keeping "GPUs doing what they do best" while the CPU absorbs orchestration.

Also announced: Spectrum-X Multiplane, which NVIDIA says scales Ethernet to 512,000 GPUs without a third network tier and retains about 90% of bandwidth if one plane fails in an eight-plane topology, now in production at CoreWeave; and NVLink Fusion, which connects third-party custom XPUs and CPUs to NVIDIA's rack architecture. The through-line is consistent: NVIDIA is selling the whole factory rather than the chip.

Inference is the growth engine of AI. Vera Rubin extends that vision with workload-optimized AI factory configurations designed for the era of agentic AI, advancing the performance frontier with LPX for ultrafast token generation.

The strategic claim is doing more work than the silicon claim here. NVIDIA's argument is that inference is now a systems problem — context, networking, orchestration and generation codesigned together — and that no single component wins it. That is a convenient argument for the only vendor selling every component. It is also, on the evidence of where agent workloads actually stall, not obviously wrong.