OpenAI and Broadcom Unveil Custom Chip for Large Language Model Inference

In a significant development for the tech industry, OpenAI and Broadcom have collaborated on a custom-designed chip called Jalapeño, tailored specifically for large language model inference in data centers. This innovative chip is the result of a nine-month design and production process, informed by detailed insights from OpenAI researchers and the company's roadmap for future models. As a purpose-built Application-Specific Integrated Circuit (ASIC), Jalapeño promises to outperform existing chips in current data centers, delivering substantially better performance per watt. According to OpenAI, early testing indicates a notable improvement, although a comprehensive technical report is forthcoming.

The development of Jalapeño marks the first generation in a long-term project, with both companies committed to refining the chip over time. By creating a more specialized and efficient solution for large language model inference, OpenAI and Broadcom aim to support the growing demands of data centers and pave the way for future advancements in AI technology. As the tech industry continues to evolve, the introduction of Jalapeño underscores the importance of collaboration and innovation in driving progress and improving performance.

Source: Ars Technica