In just 9 months, has OpenAI's in-house ASIC chip already surpassed Nvidia's Blackwell?

Original Author: Dong Jing

Original Source: Wallstreetcn

OpenAI's first self-developed chip, Jalapeño, has burst onto the scene, upending the established landscape of the AI chip industry at a breathtaking pace — this is not merely a product launch, but a signal: AI is reshaping the very way chips are designed.

According to a Wallstreetcn article, citing a Bloomberg report from August 25, OpenAI stated that Jalapeño leads Nvidia's GB300 in two key metrics: AI workload processed per unit of power and response speed. Developed in collaboration with Broadcom and specifically designed for the AI inference stage, the chip is expected to enter practical use as early as later this year. OpenAI's chip chief, Richard Ho, stated that Jalapeño delivers robust performance at a low power draw of 700 watts, helping to significantly reduce data center electricity costs.

According to semiconductor research firm SemiAnalysis, the chip went from design initiation to tape-out in only about 16 months, with the critical CoWoS packaging tape-out node completed in November 2025, just 9 months ago — far shorter than the industry-standard 18 to 36-month cycle.

SemiAnalysis researchers visited the OpenAI lab in person and conducted hands-on testing of Jalapeño using their proprietary benchmark suite, InferenceX. The conclusion was direct: the chip outperformed every Nvidia, AMD, and Google chip they had previously tested on the performance-per-watt (perf/W) metric.

What's even more noteworthy is that this result was achieved without Jalapeño having speculative decoding enabled or optimized techniques like prefill/decode disaggregation deployed — while the competing chips were all running their optimal configurations.

It's no wonder that even renowned semiconductor analyst Dylan Patel remarked bluntly, "It's not just Blackwell that's been toppled — even Nvidia's Rubin chip has been surpassed!"

Analysts believe this result poses a direct challenge to Nvidia's market position and is prompting the market to reassess the competitive landscape of AI chips.

Benchmark Data: Jalapeño Overwhelms Blackwell on Multiple Key Metrics

SemiAnalysis's test results show that Jalapeño holds a significant advantage in inference efficiency.

On the GPT-OSS 120B model, Jalapeño can output approximately 1,459 tokens per second, compared to just 535 for Nvidia's GB200; in terms of end-to-end latency, Jalapeño completes a task in just 1.65 seconds, while the GB300 requires nearly 6 seconds — a 3.6x gap. In high-interactivity scenarios, when the GB300 is pushed to its fastest decode speed (169 tokens per second), Jalapeño's throughput is 104.3 times greater.

On the core metric measuring data center energy efficiency — tokens output per megawatt per second — Jalapeño achieves approximately 53 million tokens/MW/s on the GPT-OSS model, while the GB200 NVL72 manages only about 10 million.

SemiAnalysis notes that this metric is essentially equivalent to tokens produced per joule, directly determining a data center's revenue ceiling — and in an era where compute is constrained by power availability, this advantage is particularly critical.

From a system-level cost perspective, after factoring in power supply, cooling, and networking, SemiAnalysis estimates Jalapeño's total cost of ownership (TCO) per chip per hour at approximately $1.56 — nearly on par with the H100's $1.55 — while Nvidia's Vera Rubin comes in as high as $3.61.

Notably, SemiAnalysis also raised several important caveats.

First, the model used in testing is not the most cutting-edge currently available. Nvidia and AMD have already published results based on the AgentX suite on larger models (such as DeepSeek V4 Pro and Kimi K3), while Jalapeño has not yet completed AgentX testing — a suite that better reflects real-world production scenarios involving multi-turn, long-context interactions.

Second, a fairer comparison would be against Nvidia's Vera Rubin, which also uses HBM4, rather than Blackwell; Vera Rubin's performance per watt is approximately 5.4x higher than the GB200 NVL72, and compared to Jalapeño, the two are nearly on par in terms of per-token total cost of ownership (TCO).

Third, Jalapeño is still in the engineering sample stage, with mass production not expected to ramp up until 2027.

AI Designing Chips: The "Flywheel Effect" of GPT-Astra and Codex

The deep involvement of AI tools is one of the core reasons Jalapeño was developed at such astonishing speed. According to SemiAnalysis, OpenAI made extensive use of internal AI models during the chip design process, including the widely discussed GPT-Astra.

X user Andrew Curran, citing OpenAI information, noted that GPT-Astra was deeply involved throughout Jalapeño's development:

"The team used Codex and GPT-Astra to optimize three open-source models — which were not originally part of Jalapeño's production plan — to high-performance status within two months."

This means AI is not just accelerating chip design itself, but also rapidly expanding the range of models the chip can support.

SemiAnalysis's data further quantifies this contribution:

AI-assisted design reduced SIMD unit area by 8% and matrix engine area by 10%, while also delivering improved timing and power consumption compared to the initial version.

On the software side, OpenAI used an internally extended version of Codex to write Jalapeño kernels, with some kernel code reaching approximately 3,000 lines; on the most performance-critical attention mechanisms and MoE modules, AI-generated code runs 1.5 to 1.8 times faster than implementations by top human engineers.

SemiAnalysis's assessment is characteristically sharp: OpenAI models running on Nvidia GPUs (such as GPT-5.6 Sol) are being used to design a chip that poses a genuine threat to the CUDA moat — "Nvidia's own GPUs are, in real time, spawning their own potential successor."

Architecture Analysis: Why Is "General-Purpose" Actually Faster?

There was a widespread assumption that Jalapeño is a chip deeply customized specifically for OpenAI's own models. But SemiAnalysis's conclusion is quite the opposite: Jalapeño is a general-purpose chip designed for AI inference, capable of running a wide variety of models and workloads — even including a version of Doom ported using Codex.

At the architectural level, Jalapeño's core design philosophy is "eliminating fixed latency." Unlike GPUs that rely on complex memory hierarchies, Jalapeño directly binds compute cores to HBM slices, with cores synchronized through a dedicated high-bandwidth collective network, dramatically reducing memory access latency.

Furthermore, Jalapeño employs out-of-order (OoO) execution cores paired with L1 caches, rather than the software-managed scratchpads commonly used by other accelerators. This allows it to get closer to theoretical hardware peak performance in small-batch, low-latency scenarios.

In terms of memory bandwidth, Jalapeño uses HBM4, achieving 15.4TB/s of single-package memory bandwidth, with 22 GB/s of HBM bandwidth per watt — compared to 11.1 for Nvidia's Rubin and just 5.71 for the GB300.

SemiAnalysis notes that Jalapeño's HBM4 pin speed reaches 10Gbps, slightly higher than Nvidia Rubin's 9.6Gbps, with Samsung likely serving as the HBM supplier.

Notably, Jalapeño opts not to implement prefill/decode disaggregation (PDD). SemiAnalysis provides a detailed explanation:

In real production environments, parameters such as input/output ratios, concurrency levels, and cache hit rates fluctuate continuously. Fixed resource partitioning leads to lower global utilization; in contrast, a homogeneous resource pool can flexibly respond to traffic variations, dynamically balancing between latency-sensitive requests and high-throughput batch processing.

The CUDA Moat: Cracks Appearing?

In its report, SemiAnalysis makes a weighty assertion: the CUDA moat may already be dead.

The basis lies in a comparison of software ramp-up speed. Nvidia's Rubin completed its CoWoS tape-out one month earlier than Jalapeño, yet to date, the only public Rubin benchmark data available comes from CoreWeave's engineering samples. Nvidia has not opened its doors to third-party labs for free testing the way OpenAI has. SemiAnalysis argues this reflects not a gap in hardware, but a gap in software maturity.

Building the software stack from scratch actually freed OpenAI from legacy baggage, enabling cleaner architectural decisions. OpenAI uses its self-developed kernel programming language Gluon (built on Triton) along with an internal inference engine called "Teacup," leveraging Codex to rapidly complete kernel optimization. SemiAnalysis observed that in less than two weeks, Jalapeño's throughput at a specific interaction speed improved by more than 2x; within 8 days, the team scaled tensor parallelism from TP8 to TP32, achieving full-rack, cross-rack deployment across an entire cabinet.

However, SemiAnalysis also clearly notes that current testing only covers the relatively simple 8k1k workload, and the AgentX multi-turn long-context tests have not yet been completed. Under more complex agentic workloads, the performance of components such as routers and prefix caches will present new challenges.

Next Steps: B0 Already in Fab, 10GW Blueprint Unfolds

Jalapeño's story is far from over.

According to SemiAnalysis, all publicly tested chips are A0 stepping samples, while the B0 stepping has already entered the fab, with an expected ~25% improvement in performance per watt over A0 — the B0's single compute die will deliver 13.4 PFLOPs of MXFP4 compute, with TDP maintained at 700W.

At the system level, OpenAI has partnered with Celestica to design a complete rack solution: each ASIC cabinet contains 128 Jalapeño chips, with a dual-cabinet system drawing approximately 160kW of total power — comparable to Nvidia's GB300 dual-width rack.

On the scale-out side, a single expansion network domain can connect up to 2,048 Jalapeño XPUs across 16 racks. Regarding mass production, SemiAnalysis expects a gradual ramp-up starting in 2027, with the next milestone target being a 100MW deployment scale.

The broader strategic picture: OpenAI and Broadcom have signed a 10GW custom accelerator cooperation agreement — a scale roughly equivalent to the full power output of ten nuclear power units.

As the Wallstreetcn article notes, OpenAI's chip chief Richard Ho stated that the company has reached a power and cost level that can meaningfully reduce infrastructure expenses — "and this is just the first step." He also emphasized that Nvidia remains an important partner, and given OpenAI's enormous demand for compute, it will not abandon its existing suppliers in the near term.

Analysts point out that Jalapeño's significance may not lie in how many Nvidia chips it can replace today, but in what it proves — something previously widely doubted: an AI company, using AI tools, built a genuinely competitive chip in record time. This flywheel has already begun to turn.