Huawei seeks to grab market share in AI chips from Nvidia in China

Huawei is seeking to capture a larger share of the Chinese market for artificial intelligence chips dominated by Nvidia by helping local companies adopt rival silicon for so-called “inference” work.

China’s leading AI companies rely on graphic processing units (GPUs) made by Nvidia to “train” large language models, with the $3.4 trillion US chipmaker’s products seen as critical to developing the technology.

Instead of challenging Nvidia in training, Huawei is positioning its latest Ascend AI processors as the hardware of choice for Chinese groups running “inference,” the calculations adopted by LLM to generate a response to a prompt.

The Chinese tech giant is betting that inference will be a big source of future demand as model training slows down and AI applications like chatbots become more widespread.

“Training is important, but it only happens a few times,” said Georgios Zakaropoulos, a senior AI researcher working on inference acceleration at Huawei’s Zurich lab. “Huawei is mostly focused on speculation, which will ultimately serve more customers.”

According to company employees and Ascend customers, it focuses on the technically less challenging but potentially profitable path of retrofitting AI models trained on Nvidia products. Since Nvidia GPU and Ascend work Various softwareThe Huawei company is helping to use another software tool to make the two systems compatible

Huawei’s push comes with top-down government support. Chinese officials have urged local tech giants to buy more of Huawei’s AI chips and move away from Nvidia.

A person familiar with Nvidia’s operations in China said that Huawei, viewed internally as the country’s most serious competitor, had “advanced” chip design capabilities.

Washington has sought to curb Beijing’s AI development with export controls aimed at hindering the development of sensitive technology in China.

Unlike their US rivals like OpenAI and Google, companies are unable to access the most modern GPUs in China. But while Chinese groups are only able to acquire Nvidia’s low-end H20 chips designed to meet export controls, the less powerful GPUs are in high demand because they are considered better than local alternatives.

Analyst and Huawei researchers Ascend was not yet ready to replace Nvidia for model training due to technical issues, such as a breakdown in the way chips communicate with each other within a wide “cluster” of AI chips when training large models.

“While Ascend chips perform well on a per-chip basis, there is a bottleneck with inter-chip connectivity,” said Lin Qingyuan, China semiconductor analyst at Bernstein. “When training a large model, you must break it down into smaller tasks. If one chip fails, the software has to find a way for other chips without delay.”

The other challenge for Huawei is convincing developers to switch from Nvidia’s Cuda software, which is known as the company’s “secret sauce” for being easy for developers to use and able to greatly speed up data processing.

But Huawei’s soon-to-be-released and updated version of its AI chip, the Ascend 910C, is expected to address these concerns. An unnamed Huawei employee said, “We expect this new generation of hardware to come with improved software that makes it more accessible to developers.

Huawei and Nvidia face stiff competition. Chinese internet group Baidu and chip designer Cambricon have made progress in developing AI chips. Meanwhile, in the US, Amazon and Microsoft are also betting that they can capture more market share in chips for inference as AI applications become more widespread.

Estimates from SemiAnalysis, a chip consultancy, show Nvidia made $12bn in sales in China last year delivering its H20 chip to the country, twice as much as Huawei sold AI chips with the Ascend 910B.

“Nvidia’s China-specific H20 GPUs make up most of the AI ​​chips sold in China.” But the lead is shrinking rapidly as Huawei ramps up production capacity,” said Dylan Patel, principal analyst at SemiAnalysis.

Industry insiders warned that Huawei’s AI chip push was also limited by insufficient supply, with two potential customers telling the Financial Times they were unable to secure the chips.

Huawei did not respond to a request for comment. Nvidia declined to comment.

Analysts said Huawei’s manufacturing is likely to face challenges due to US export controls that have left Chinese fabs dependent on outdated chip manufacturing equipment.

The focus on assumptions also points to an evolving dynamic in Chinese AI that differs from that in the United States. Washington’s export controls mean Chinese AI players are not engaged in the same competition as Silicon Valley rivals Meta, Elon Musk’s x.AI and OpenAI to build large mega-clusters of Nvidia’s most advanced GPUs.

“Chinese companies are playing a different game. They are focusing much more on speculation than the US because it is possible to achieve large efficiencies with less powerful chips, which means they can achieve commercialization faster,” Bernstein analyst Lin said.

Chinese companies are betting that they can stay competitive in AI by lowering guesswork costs, making AI applications cheaper to run, he said.

Last month, Hangzhou and Beijing-based start-up Dipsic released its V3 model, which has attracted attention for its lower training and estimation costs than comparable models in the US.

The company proposed a new way for an AI model to focus on specific parts of the input data as a way to reduce the cost of running the model. It also used the “expert mix” technique popular with other Chinese AIs Start upwhich helps speed up estimation because only part of the model is used to generate a response.

Dipsic said Huawei successfully adapted the V3 to the Ascend, providing detailed instructions for developers on how to use the chip. Before FT Report That Huawei sent engineers to help customers migrate from Nvidia to Ascend.

Additional reporting by Jijing Wu in Hong Kong