OpenAI's Jalapeño chip changes the inference-cost conversation

OpenAI has published the first measured results for Jalapeño, its custom inference chip. The company reports 1.5 to 1.9 times more AI work per watt at peak throughput and 1.7 to 3.6 times lower end‑to‑end latency than the comparison systems across three open‑weight models. Independent reporting confirms the headline advantage while noting that the test is against currently available hardware and that Jalapeño is not a training chip.^{1 2}

Inference is the work performed after a model is trained: processing a prompt and generating a response. OpenAI tested Jalapeño on SemiAnalysis's public InferenceX framework, which measures the full serving path rather than a narrow arithmetic peak. That makes latency, throughput and power relevant together, especially for agents that must wait on many sequential model calls.¹

OpenAI says the chip delivered a better combination of throughput and latency on GPT‑OSS 120B, DeepSeek R1 670B and Kimi K2.5 1T. On Kimi, it reports roughly 1.5 times higher peak performance per watt and 3.4 times lower end‑to‑end latency than the comparison system. The practical prize is not a benchmark trophy: it is the possibility of serving more requests inside the same power envelope.¹

TechCrunch reports that the comparison includes an Nvidia Blackwell system, but also stresses that the competitive field may advance before Jalapeño reaches broad deployment. OpenAI has not disclosed customer pricing, fleet share or a date when users should expect a measurable product‑level change. Buyers should therefore treat the numbers as credible technical evidence, not as a current price cut.²

When a model provider controls the serving stack, it can tune memory placement, networking and software around real workloads. OpenAI says Jalapeño keeps model state local and balances the compute‑heavy prefill phase with the memory‑heavy decode phase. If those gains survive deployment, the provider gains more control over latency, capacity and margins rather than depending entirely on a general‑purpose accelerator roadmap.¹

Jalapeño is designed for inference, not for training frontier models. Axios and TechCrunch both frame it as a way to reduce part of OpenAI's dependence on Nvidia, not eliminate it. Organizations evaluating vendor concentration should distinguish the silicon that creates a model from the silicon that serves it.²

The meaningful follow‑up metrics are production availability, sustained utilization, price per useful task and application‑level latency under load. For now, Jalapeño strengthens the case that inference economics will increasingly be set by vertically integrated systems. It does not yet tell a customer what their bill or response time will be.