Nvidia unveiled the Groq 3 language processing unit (LPU) at GTC 2026. This marks the first chip produced under Nvidia’s $20 billion licensing and talent deal with Groq.
The SRAM-based Groq 3 LPX acts as a decode-phase co-processor in the Vera Rubin platform. This post breaks down its architecture, performance, and what it might mean for AI inference in 2026 and beyond.
Groq 3 LPX: Design, placement, and performance
The Groq 3 LPX is all about SRAM and works as a decode-phase co-processor alongside Vera Rubin GPUs. Samsung manufactures it on a 4‑nm process, and shipments are expected in Q3 2026.
Each LP30 die packs 512 MB of on-chip SRAM. It delivers a wild 150 TB/s of memory bandwidth, easily outpacing Rubin’s HBM4 at 22 TB/s.
If you fill a LPX rack with 256 LPUs, you get 128 GB of SRAM and 40 PB/s of bandwidth. Nvidia claims that pairing an LPX rack with a Vera Rubin NVL72 gives you about 35× higher throughput per megawatt for trillion-parameter models, compared to NVL72 alone.
- On-die SRAM per LPU: 512 MB
- Per-die bandwidth: 150 TB/s
- Rack capacity: 256 LPUs, 128 GB SRAM, 40 PB/s
- Throughput uplift (with Rubin NVL72): ~35× per megawatt
- Perf metric: 1.23 FP8 PFLOPS per LPU
The design sticks to Groq’s SRAM-centric, pre-scheduled VLIW approach. It expands per-die capacity and keeps latency predictable, which should help with big autoregressive workloads.
Rubin GPUs still handle the heavy compute during prefill phases and manage long contexts. Groq LPUs, on the other hand, crank out output tokens quickly and with low latency.
The Dynamo orchestrator keeps this mix running smoothly, tying together different hardware units for unified inference.
System integration and workload partitioning
By splitting compute-heavy prefill tasks from token generation, Groq 3 LPX aims to boost both latency and throughput for massive models. The SRAM-based, pre-scheduled execution model keeps latency predictable—super important for streaming inference and real-time use cases.
Rubin GPUs focus on the long-context work. Groq LPUs handle the rapid-fire token generation, all under Dynamo’s direction.
Supply chain, partnerships, and roadmap shifts
Supply and deployment around Groq 3 are moving fast. Samsung increased wafer production from around 9,000 to 15,000 units as the chip transitions from samples to commercial scale.
AWS plans to roll out Groq 3 LPUs alongside more than a million Nvidia GPUs. That’s a pretty broad integration into hyperscale data centers.
Interestingly, the Rubin CPX GDDR7 inference accelerator—previously on Nvidia’s roadmap—seems to have been dropped for the Groq LPU. It’s another sign of the industry’s habit of snapping up inference-chip startups and baking their tech into bigger platforms.
- Wafer production: ~9,000 → ~15,000
- Customer deployments: AWS integrating Groq 3 LPUs with Nvidia GPUs
- Roadmap change: Rubin CPX GDDR7 accelerator appears sidelined in favor of Groq LPU
Industry context and strategic implications
Analysts call 2025 a turning point. Incumbents started snapping up inference-focused startups to beef up their platforms.
The non-exclusive licensing between Nvidia and Groq keeps a hint of competition alive, but honestly, the big players are just weaving startup tech into scalable, money-making architectures. The Groq-Nvidia deal is a good example of this consolidation wave, with platform integration taking priority over standalone chips.
Implications for AI inference infrastructure in 2026
Data centers looking at trillion-parameter model workloads have a new option: the Groq 3 LPX. This platform mixes deterministic latency with high memory bandwidth and pretty solid power efficiency.
It offloads latency-sensitive token generation to SRAM-based LPUs. Meanwhile, Rubin GPUs take care of the heavy compute.
The Vera Rubin platform could open up new possibilities for real-time AI services. These days, cloud providers are throwing GPUs, CPUs, and accelerators together in all sorts of ways.
The Groq 3 LPU stands out as a startup innovation that’s actually making its way into enterprise AI infrastructure. That’s not something you see every day.
Bottom line: Nvidia’s Groq 3 LPX is an important move. It combines high‑bandwidth SRAM co-processors with compute GPUs, brings in more partners, and seems to fit right in with the bigger trend of hardware consolidation in AI inference over the next few years.
Here is the source article for this story: How Nvidia’s $20 billion Groq 3 LPU deal reshapes the Nvidia Vera Rubin Platform — Samsung 4nm process serves as bedrock for SRAM-based AI accelerator chip