OpenAI Jalapeño Chip: Specs, Benchmarks, Roadmap

Jalapeño is OpenAI's first custom inference chip, co-developed with Broadcom, and at the Hot Chips conference OpenAI published its first benchmark numbers: 1.5-1.9x more work per watt at peak throughput and 1.7-3.6x lower end-to-end latency than Nvidia's GB200/GB300 systems, measured on real model workloads. The chip carries 216 GB of HBM4, delivers up to 13.4 PFLOPS of MXFP4 compute at a 700W rating, and starts deploying into OpenAI's own infrastructure by the end of 2026. Those are first-party numbers and deserve the usual skepticism, but people who benchmark hardware for a living do not consider them fantasy.

Hot Chips 2026 brought major AI inference announcements. Nvidia put its Groq-derived LPX accelerator into full production, Cerebras laid out two future generations of wafer-scale systems, and Apple showed local clustering tooling for its new Mac silicon. All four companies are trying to serve more tokens per watt and per dollar than a general-purpose GPU can.

What OpenAI actually built

Jalapeño is an inference ASIC, meaning a chip designed to run trained models. Nvidia GPUs support both training and inference. According to Tom's Hardware's unpacking of the Hot Chips presentation, the design uses a NUMA-style architecture with 64 memory/core slices, pairing 216 GB of HBM4 with up to 13.4 PFLOPS of MXFP4 compute at 700W. ServeTheHome's coverage adds the system-level picture: a pod scales to 2,048 accelerators, which works out to 27 EFLOPS of MXFP4 compute, 432 TB of HBM4, and 32 PB/s of aggregate memory bandwidth.

The chip went from unveiling (June 2026, per TechCrunch) through a 9-month tape-out to published benchmarks in under a year, with limited deployment expected by end of 2026. Sam Altman confirmed Gen 2 is already deep in development with Gen 3 underway, establishing a production roadmap.

How the benchmarks compare to Nvidia GB200/GB300

OpenAI's claims, published by OpenAI and covered by Tom's Hardware, break down like this:

A chip rated at 700W that sustains real workloads at 550W or below gives data center operators thermal and provisioning headroom. When compute buildouts are measured in gigawatts, and the economics of an AI data center are set by inference throughput per gigawatt, a 1.5-1.9x perf-per-watt improvement creates large capex and opex differences at fleet scale.

Jalapeño reportedly reduces the usual throughput/latency tradeoff, according to the Latent Space roundup of reactions. In some tested setups it reportedly hit these numbers without aggressive prefill/decode disaggregation (splitting prompt processing and token generation onto separate hardware) or speculative decoding (drafting tokens with a small model and verifying with the big one), while beating systems that used those techniques. SemiAnalysis called it unusually strong for a first-generation ASIC and compared it directly against Blackwell and Rubin-class systems.

The standard caveats apply. These are OpenAI's own benchmarks on OpenAI's own workloads, the full Hot Chips presentation was not yet public when the numbers landed, and several commentators noted that packaging and foundry capacity (TSMC CoWoS in particular) remain a hard bottleneck regardless of how good the silicon is. Without volume manufacture, silicon quality will not alter 2027 buying decisions.

The kernels were partly written by models

OpenAI says GPT-Astra and Codex helped write and optimize Jalapeño's low-level kernels, and brought three previously unplanned open-weight models to high performance on the chip in about two months. For selected attention and MoE blocks, the model-generated implementations reportedly ran 1.5-1.8x faster than existing human-expert-written code.

Kernel engineering for novel accelerators has historically kept everyone locked to CUDA. Nvidia's software advantage included two decades of hand-tuned kernels and the scarce humans who could write them. If frontier models can now port and optimize workloads for new silicon in months rather than years, the switching cost protecting the incumbent drops for whoever owns both the models and the chips. OpenAI can use its models to optimize its chips, serve those models more cheaply, and fund the next generation with the savings.

Nvidia's Groq architecture

Nvidia presented Groq 3 LPX, now in full production, as an extension of the Vera Rubin platform, built on the LPU technology Nvidia licensed from Groq for $20 billion in December 2025. Each LP30 chip carries about 500MB of on-die SRAM and no HBM, with 350ns chip-to-chip latency. A rack holds 256 LPUs with 128GB of total SRAM, 40 PB/s of aggregate SRAM bandwidth, and 315 PFLOPS of FP8 compute.

Artificial Analysis measured a Groq 3 LPX rack at 3,431 output tokens per second on a 100K-context Gemma 4 31B reasoning workload, roughly 4x the 870 tokens per second of the next-fastest public endpoint. That is a third-party benchmark, which Jalapeño does not yet have, and it targets long-context agentic inference where per-user token speed determines whether an agent loop is usable.

Nvidia's $20 billion license brought a pure-SRAM, deterministic-latency design into its lineup. The company that defined the AI hardware market now ships a heterogeneous lineup because no single architecture wins every inference regime.

Cerebras's wafer-scale approach

Cerebras used Hot Chips to lay out its next two generations. The CS-5, targeted for 2027, aims for up to 5,000 output tokens per second per user and 3 million tokens per second per megawatt on frontier models like Kimi and GPT-5.6 Sol. Current CS-4 racks already pack three WS-3T wafer-scale engines with a 53 PB/s on-wafer fabric. Beyond that, the CS-6 roadmap adds 3D-stacked wafer-scale SRAM and vertical power delivery.

Cerebras markets hardware in units of throughput per unit of power, rather than FLOPS. Its customers are asking about performance per watt, the axis of competition across all four companies at the conference because power constrains every buildout currently under construction.

Apple's local cluster approach

Apple's Hot Chips presence centered on local AI for the new M5 Ultra Mac Studio and M6/M5 Pro Mac Mini, including RDMA support over Thunderbolt 5 for clustering Macs. The exo project, which Apple featured on its product pages, uses that low-latency link to run models like Kimi K3 and GLM-5.3 across small Mac clusters at API-like speeds, with four M5 Ultras scaling to roughly 4.8 TB/s of aggregate memory bandwidth.

Apple's local clusters target a different market. The pressure driving OpenAI to build its own silicon, inference cost per token, also drives a parallel movement downmarket, where a 2.78 trillion parameter MoE can run on modest hardware with enough systems cleverness. Apple supplying RDMA-over-Thunderbolt clustering as a first-party feature indicates that it sees small local inference clusters as a product category beyond hobbyist use.

Conclusion

Hot Chips 2026 showed that the inference monoculture era is ending. OpenAI will serve a growing share of its own traffic on silicon it designed, with a multi-generation roadmap already funded. Nvidia licensed technology for $20 billion and now sells SRAM-only racks alongside HBM GPUs. Cerebras is quoting throughput per megawatt for 2027 delivery. Nvidia retains its training position, and foundry capacity still gates everyone equally.

The inference market now includes multiple architectures competing on tokens per watt at the latency an agent loop demands. The first-party Jalapeño numbers need independent verification, and the Artificial Analysis result for Groq 3 LPX shows the kind of verification available when a vendor invites it. If the 1.5-1.9x perf-per-watt claim survives third-party measurement on non-OpenAI workloads, a chip unavailable for purchase will influence inference pricing in 2027.