At present, the focus of AI is shifting from large-scale training to agentic inference, and traditional hardware design centered on GPU compute struggles to capture the real workload characteristics of AI agents. An agent's workflow consists of numerous iterative loops of reasoning, tool invocation, code execution, and result verification. This entire pipeline generates massive amounts of tokens, intertwining long-context processing with decoding latency. Minute latencies progressively amplify across multiple loops, representing the underlying hardware bottleneck encountered in the deployment of many current agent projects.
Therefore, the highlight of the complete Vera Rubin solution unveiled by NVIDIA at Hot Chips 2026 is not the spec leap of a single chip, but rather a full-stack systems engineering approach tailored for agentic scenarios, spanning chips, networking, and full rack systems.
Groq 3 LPX Enters Mass Production
The centerpiece of this release is undoubtedly the full-scale production of the Groq 3 LPX. It is not a standalone accelerator, but an interactive inference extension for the Vera Rubin NVL72 platform—specifically designed to address two computational challenges faced by agentic AI: processing massive contexts and generating tokens with extremely low latency.
In the Artificial Analysis benchmark, while running the Gemma 4 31B open-source agent model and processing a 100K token context, the Groq 3 LPX achieved a record of 3,400 output tokens per second. This is the fastest performance ever recorded for this model. More critically, it responds 4 times faster than comparable platforms—meaning tasks such as agentic programming can be reduced from hours to minutes.
This performance stems from architectural division of labor. The Rubin GPU handles large-scale context processing, while the LPX specifically accelerates latency-sensitive decode workloads. The two collaboratively compute every layer of the model, thereby eliminating the trade-off between speed and throughput in traditional architectures.
In terms of deployment, Nebius was the first to adopt the Groq 3 LPX, integrating it into the Nebius Token Factory production inference platform. Developers do not need to migrate their tech stacks and can achieve ultimate token generation speeds through the same set of APIs. Additionally, Groq closely followed as an early adopter.
Vera CPU Built Specifically for AI Agents
If the Groq 3 LPX solves the speed issue during the generation phase, then the Vera CPU addresses the efficiency of the agent's "thinking" between model invocations.
Agentic AI increasingly relies on CPUs to orchestrate tools, execute code, process data, and run simulations—tasks that occur in the gaps between GPU inference steps. However, traditional x86 CPUs do not perform ideally in these scenarios.
Consequently, NVIDIA introduced the Vera CPU, the world's first CPU built specifically for AI agents. It features 88 custom-designed Olympus cores, utilizes spatial multithreading technology, and is equipped with LPDDR5X memory providing 1.2 TB/s bandwidth.
Spectrum-X Multiplane: Breaking Through Networking Bottlenecks
When AI factories scale to a certain size, traditional three-tier network topologies become bottlenecks for both performance and cost. Adding a third network tier means increased latency, unpredictable performance, and soaring costs for cabling, optical modules, and power.
Spectrum-X Multiplane adopts a different approach: it splits the network connections of each server into multiple independent paths (planes), with each plane running a lightweight two-tier network. The result is a flat, simple network topology that can scale to 512,000 GPUs while avoiding the overhead of a third-tier network. In an eight-plane topology, even if one plane fails, the network can still maintain approximately 90% of the total bandwidth—hardware recovery is 11 times faster than software multi-plane load balancing, which directly translates to a 1.6x increase in AI factory output.
CoreWeave has already deployed Spectrum-X Multiplane in production environments to connect Vera Rubin racks. This solution is based on the Spectrum-6 Ethernet ASIC (102.4 Tb/s) and ConnectX-9 SuperNIC (1,600 Gb/s per GPU). From the switch silicon to the SuperNICs and software, everything is a co-designed product tailored for the Vera Rubin NVL72.
Scale-In: Infrastructure as Part of Compute
NVIDIA also introduced the fifth pillar of AI networking—Scale-In. The very proposal of this concept is highly indicative: traditional "north-south" access networks must be redefined in the era of agentic AI.
Agentic AI factories converge large-scale accelerated computing, massive user bases, and autonomous agents, which continuously interact with data, storage, and services. Networking, storage, security, and operations must accelerate in sync with AI compute, and can no longer be "bolted on" to the system as independent modules.
Driven by the BlueField-4 DPU and the DOCA software platform, and connected via Spectrum-X Ethernet, Scale-In provides multi-tenant networking, high-performance storage access, silicon-level security, resilient configuration, and real-time observability. All infrastructure services remain independent of host compute resources but are co-designed and accelerated as an organic part of the AI factory.
NVLink Fusion: An Open Path for Semi-Custom AI Factories
NVLink Fusion represents thinking from another dimension: not all AI factories require a completely homogeneous hardware stack. Hyperscalers and AI-native companies have their own custom XPUs, but rapidly bringing custom chips to large-scale data center deployment faces enormous engineering hurdles—ranging from CPU and scale-up interface integration, to rack thermal and power design, to vendor ecosystem management.
The solution provided by NVLink Fusion is to connect custom XPUs and CPUs into NVIDIA's scale-up interconnect domain. The sixth-generation NVLink supports an interconnect domain of 72 XPUs, reducing end-to-end latency to one-third of generic Ethernet alternatives and increasing packet rates by 10 times. NVLink-C2C can connect XPUs to the Vera CPU or other ecosystem CPUs, achieving energy efficiency up to 6 times that of PCIe interfaces.
More importantly, customers adopting NVLink Fusion can directly leverage NVIDIA's MGX rack-level solutions. QCT revealed that the production line system build for the Vera Rubin NVL72 is nearly 100% automated. This means operators can begin data center expansion before the chips are even ready, and later flexibly allocate the ratio of GPUs to XPUs based on workload demands and chip supply. MediaTek and GUC have already joined as partners.
Conclusion
Looking at the entire Vera Rubin platform holistically, it becomes evident that evaluating hardware in the agentic era can no longer rely solely on single-chip compute numbers. What actual businesses truly care about are tokens output per unit of time, compute output per watt, cost per token, and actual cluster utilization. The Groq 3 LPX focuses on decoding, the Vera CPU handles scheduling and orchestration, Spectrum-X Multiplane solves large-scale networking, and NVLink Fusion bridges the custom chip ecosystem. Ultimately, all components serve to build a "token factory" capable of stably running closed-loop agent operations.
Jensen Huang stated that inference is the growth engine of AI. In the present context, growth is no longer just about running models, but about efficiently and with low latency completing thousands of agent loops. Vera Rubin achieves this by co-designing the AI factory as a complete system, rather than simply stacking a bunch of high-performance chips together.