OpenAI partners with Broadcom to develop specialized AI inference chip

 By Ed Ludlow, Rachel Metz and Ian King 

OpenAI is working with Broadcom Inc. to develop a new artificial intelligence chip specifically focused on running AI models after they’ve been trained, according to two people familiar with the matter. 

The AI startup and chipmaker are also consulting with Taiwan Semiconductor Manufacturing Co., the world’s largest chip contract manufacturer, said the people, who asked not to be identified because the discussions are private. OpenAI has been planning a custom chip and working on such uses for the technology for around a year, the people said, but the discussions are still at an early stage. 

 

OpenAI declined to comment. A representative for Broadcom didn’t respond to a request for comment, and a TSMC spokesperson said the company doesn’t comment on rumors and speculation. Reuters reported on OpenAI’s ongoing talks with Broadcom and TSMC on Tuesday. The Information reported in June that Broadcom had discussed making an AI chip for OpenAI.  The process for taking a chip from design to production is long and expensive. OpenAI is less focused on graphics processing units, chips that are used to train and build generative AI models — a market that has been cornered by Nvidia Corp. Instead, it’s looking for a specialized chip that will run the software and respond to user requests, a process called inference. Investors and analysts expect the need for chips to support inference will only grow as more tech companies use AI models to field more complex tasks.

 

OpenAI may continue to research setting up its own network of foundries, or chip factories, one of the people said, but the startup has realized that working with partners on custom chips is a quicker, attainable path for now. Reuters earlier reported that OpenAI was pulling back from the effort of establishing its own chip manufacturing capacity.

 

Broadcom shares closed up 4.2 per cent to $179.24 in New York trading on Tuesday. Even before the latest gain, they were up 54 per cent this year. TSMC’s US-traded shares closed more than 1 per cent higher.

 

Broadcom is the largest designer of application-specific integrated circuits — chips designed to fit a single purpose specified by the customer. The company’s biggest customer in this area is Alphabet Inc.’s Google. Broadcom also works with Meta Platforms Inc. and TikTok owner ByteDance Ltd.

 

When asked last month whether he has new customers for the business, given the huge demand for AI training, Broadcom Chief Executive Officer Hock Tan said that he will only add to his short list of customers when projects hit volume shipments.

 

“It’s not an easy product to deploy for any customer, and so we do not consider proof of concepts as production volume,” he said during an earnings conference call. 

 

OpenAI’s services require massive amounts of computing power to develop and run — with much of that coming from Nvidia chips. To meet the demand, the industry has been scrambling to find alternatives to Nvidia. That’s included embracing processors from Advanced Micro Devices Inc. and developing in-house versions.

 

OpenAI is also actively planning investments and partnerships in data centers, the eventual home for such AI chips. The startup’s leadership has pitched the US government on the need for more massive data centers and CEO Sam Altman has sounded out global investors, including some in the Middle East, to finance the effort.

 

“It’s definitely a stretch,” OpenAI Chief Financial Officer Sarah Friar told Bloomberg Television on Monday. “Stretch from a capital perspective but also my own learning. Frankly we are all learning in this space: Infrastructure is destiny.”