OpenAI Personally Steps into Chipmaking, Should Nvidia Be Nervous?

On June 24, OpenAI and Broadcom jointly released a chip.

The name is quite interesting—Jalapeño, Mexican chili pepper. Whether it's spicy or not aside, the signal from this chip is striking: OpenAI finally has its own silicon.

This is an AI accelerator designed specifically for large-scale language model inference, and is the first product from multiple generations of computing platform cooperation between the two parties. According to details disclosed by The Wall Street Journal, the two companies will cooperate over the next four years to develop and deploy 10 gigawatts of custom AI chips and computing systems.

What’s the scale of 10 gigawatts? Roughly equivalent to the output power of 10 large nuclear power plants. This isn’t just making chips—this is building power-plant-level computing infrastructure.

Why Now?

The answer is simple: money and supply.

Sam Altman has publicly complained about two things: Nvidia GPU shortages, and the “dizzying” cost of running such hardware.

This is no exaggeration. OpenAI is one of Nvidia’s largest customers. Since 2020, it has been training models on a Microsoft-built supercomputer that used 10,000 Nvidia GPUs. By the time of GPT-4, that number had multiplied several times. Now, with GPT-5 iterations and explosive ChatGPT user growth, inference-side computing consumption has far exceeded training.

Here we need to distinguish between two concepts: training and inference.

Training is teaching the model skills, with massive one-time computing investment; once done, model weights are fixed. Inference is getting the model to do tasks—each user query consumes compute, smaller per request but with very high frequency. For a product like ChatGPT with over 100 million daily active users, inference cost is the real bottomless pit.

Nvidia’s GPUs are general-purpose, capable of both training and inference, but “can do both” often means “neither is optimal.” Chips designed specifically for inference can achieve higher energy efficiency ratios for specific workloads—more queries for the same electricity cost.

Google figured this out long ago. TPU (Tensor Processing Unit) development began in 2015 and is now on its fifth generation, powering Google Search, YouTube recommendations, and Gemini model inference. Microsoft released Maia 100 in 2023, optimized for AI inference on Azure cloud. Amazon has Inferentia and Trainium. Meta is also working on its own chips.

Among large companies, OpenAI was the last to rely completely on purchased chips.

Now it has filled that gap.

What Exactly is Jalapeño?

From publicly available information, Jalapeño is an ASIC (application-specific integrated circuit), neither a GPU nor a general-purpose processor.

Its design goal is very clear: in large-scale LLM inference scenarios, provide higher throughput and lower unit cost compared to general-purpose GPUs. Specific architectural details have not been fully disclosed, but based on the cooperation model, the chip likely follows these technical routes:

1. Compute Units Optimized for Transformers

Modern LLMs are almost all based on the Transformer architecture, whose core computations are matrix multiplication and attention calculation. General GPUs are designed to handle many workloads, resulting in a lot of “waste.” Purpose-built ASICs can dedicate transistors solely to the computations Transformers need most, removing unneeded functional modules.

2. Large-Capacity, High-Bandwidth Memory

Inference bottlenecks are often not compute but memory bandwidth. The larger the model, the more parameters, and each inference requires reading proportionally more data from memory. Jalapeño likely has large-capacity HBM (high bandwidth memory), optimized for LLM memory-access patterns.

3. Rack-Level System Integration

According to Zhihu information, this cooperation encompasses “far more than just a single chip.” Broadcom provides end-to-end Ethernet, PCIe, and optical connection solutions—a complete rack-level system. This means OpenAI is not just designing a chip but an entire data center compute unit.

Even the most powerful chip will encounter bottlenecks in large-scale deployment if network interconnect lags. This full-stack cooperative model reveals substantial ambition from OpenAI at the infrastructure level.

4. Manufactured by TSMC

Chip design is one thing, manufacturing another. According to prior reports, Jalapeño will be manufactured by TSMC, with mass production expected in the second half of 2026. Given that it’s already June 2026, this means the first batch may already be in tape-out or small-volume production.

Broadcom’s role here is akin to a “chip design service provider.” It doesn’t sell finished chips like Nvidia, but helps clients design custom chips and coordinate manufacturing. Google’s TPU and Meta’s in-house chips both have Broadcom’s involvement.

What Does This Mean for Nvidia?

In the short term, limited impact.

OpenAI won’t stop buying Nvidia GPUs just because it has Jalapeño. Training large models still requires general-purpose GPU compute, and OpenAI’s inference demand may grow faster than production ramp-up for its own chips. Altman himself said he hopes “to maintain good relationships with chip manufacturers, especially when using the next-generation Blackwell chips.”

But in the long term, this is a dangerous signal.

Nvidia’s moat has two layers: first, leading hardware performance; second, CUDA ecosystem lock-in. Hardware leadership can be caught up with via R&D investment. Ecosystem lock-in is the real barrier—developers accustomed to CUDA face high migration costs and are unwilling to switch even if alternatives exist.

But cloud providers and large model companies are different beasts.

They have the engineering capacity to build their own software stacks, the scale to dilute custom chip R&D costs, and the motivation to eliminate dependence on a single supplier. Google has proven TPUs can support world-class AI systems, now OpenAI wants to prove the same.

If OpenAI’s custom chip succeeds, what will other large model companies think?

Anthropic, xAI, Mistral, DeepSeek… all are currently working off Nvidia. If they see OpenAI’s path succeed, will they be tempted?

Chip industry logic is about economies of scale. Design cost is one-time; the higher the output, the lower the per-unit cost. If more companies start building their own inference chips, design service providers like Broadcom become more competitive, TSMC’s capacity shifts toward custom ASICs, and Nvidia’s market share gets nibbled away.

It won’t happen tomorrow—but perhaps in five years.

OpenAI’s Chip Ambition

Looking back at OpenAI’s hardware layout, it’s clear this cooperation was not impulsive.

In early 2024, reports emerged that OpenAI was exploring in-house chip design, with a team of about 20, including engineers involved in Google TPU projects. At the time, reports also said OpenAI planned to build its own fabrication plant but dropped the idea due to high costs and long cycles.

Partnering with Broadcom is a more pragmatic choice. OpenAI focuses on chip architecture design, handing physical implementation and manufacturing to professional partners, enabling faster time-to-market.

From a strategic perspective, OpenAI needs competitive advantage at three levels:

Model layer: Where’s the capability boundary for the GPT series? Can it stay ahead?

Application layer: Can ChatGPT hold onto user mindshare? Can it break into enterprise markets?

Infrastructure layer: Can compute cost be reduced? Can supply chains be autonomous and controllable?

The first two layers are things OpenAI has been working on; the third is just beginning. Infrastructure is often the easiest to overlook yet most decisive for long-term victory.

Think about it—if OpenAI’s inference costs are 30% lower than competitors, what could it do?

• Maintain higher profit margins at the same pricing
• Or lower prices to grab market share, squeezing competitors through price wars
• Or invest saved funds into larger-scale training, widening model capability gaps

Each of these options is lethal.

The 10-Gigawatt Ambition

Back to that number: 10 gigawatts.

Such scale of compute deployment can’t be fully consumed by OpenAI alone. Even if ChatGPT user numbers multiply several times, even if API calls grow exponentially, it still won’t use that much.

A reasonable guess is that OpenAI is preparing for something bigger.

Possibly AGI. Altman has always said OpenAI’s goal is “to build artificial general intelligence at all costs.” What compute does AGI need? No one knows—but certainly more than today.

Possibly AI agents. If AI evolves from “conversation assistant” to “autonomous agent,” capable of long-term continuous operation and proactively completing complex tasks, inference demand will grow geometrically. A user going from asking a few questions daily to having AI working 24/7 could increase compute consumption dozens of times.

Possibly multimodal. When models process text, images, audio, and video simultaneously, compute will exceed that of pure text models. OpenAI’s Sora already showcases video generation capabilities; if that tech scales commercially, inference cost will be a huge challenge.

Whatever the case, 10 gigawatts of compute reserve shows OpenAI is betting on a much larger future.

How Will the Industry Change?

After this release, AI chip competition will grow more complex.

First tier: Nvidia

Still the undisputed leader; general-purpose GPU + CUDA ecosystem is unbeatable short term. But market share ceilings may now emerge.

Second tier: In-house cloud chips

Google TPU, Amazon Inferentia/Trainium, Microsoft Maia—these chips mainly serve their own cloud services, not sold externally. Advantage: deep optimization for their workloads. Disadvantage: scale limited by their own businesses.

Third tier: In-house chips by model companies

OpenAI’s Jalapeño is the first move. If successful, more model companies will follow. This tier’s characteristics: clear demand (running their own models) but less chip design experience, needing partners like Broadcom.

Fourth tier: Independent chip companies

AMD, Intel, plus numerous startups. AMD’s MI300 is catching up to Nvidia; Intel’s Gaudi is trying. Among startups, Cerebras, Groq, SambaNova each have unique approaches, but scale remains small.

In coming years we may see a more fragmented market. Different companies will choose different chips for different scenarios—no more “Nvidia dominates all.” This benefits the industry: competition drives innovation and lowers prices.

For developers, it means more choice but also higher learning costs. Different chips have different software stacks; code may require optimization for specific hardware. The good news: mainstream frameworks (PyTorch, TensorFlow) are working to abstract away hardware differences, letting developers use unified interfaces to call different backends.

Final Thoughts

OpenAI making chips has long been inevitable.

When a company’s core business heavily depends on a scarce resource monopolized by another company, vertical integration is the inevitable choice. This isn’t unique to OpenAI—it’s basic business logic.

Why did Apple design M-series chips? To avoid being choked by Intel. Why did Tesla design FSD chips? Because Mobileye couldn’t meet demands. Now why is OpenAI designing inference chips? Because Nvidia is too expensive, too scarce, and too uncontrollable.

Whether Jalapeño succeeds still needs time to be proven. Chip industry pitfalls abound—from design, to tape-out, to mass production, to deployment, each step can go wrong. But at least, OpenAI has taken the step.

For developers, short-term impact will be minimal. The API you call is still the same, the model still the same. But over time, if OpenAI’s custom chips truly lower inference costs, it will eventually reflect in API pricing.

Currently, OpenAI Hub already supports calls to the full GPT series, including the latest GPT-5.5. If you’re in China, calling via an aggregation platform may be the more stable choice.

The chip story is just beginning.

---

References

• OpenAI and Broadcom Launch LLM Inference Chip "Jalapeño" - Zhihu: Detailed description of the partnership’s full-system solution, including Broadcom’s end-to-end connectivity solutions