China's efforts to match the computing power of the USA in artificial intelligence (AI) are being slowed down by faulty software. Customers of China's leading AI chip manufacturer, Huawei, are complaining about performance issues and the difficulty of switching from Nvidia products to Huawei's Ascend chips.
After the USA further tightened export controls for high-performance chips last October, Huawei has established itself as a leading player in the development of a domestic alternative to Nvidia. The Ascend chips, in particular, are increasingly used by Chinese AI groups for inference, a central process in applications such as OpenAI's ChatGPT.
However, several industry insiders, including an engineer from a partner company, report that the Ascend chips are still far behind Nvidia's products in the initial model training phase. The main causes are stability issues, slow inter-chip connectivity, and Huawei's inferior software platform Cann. While Nvidia's software Cuda is known for its user-friendliness and data processing acceleration, Huawei is struggling to achieve similar success.
Even internal voices at Huawei criticize Cann. A researcher who wished to remain anonymous described the product as "heavy and unstable in application" and reported difficulties during testing. "With random errors, it is very difficult to identify their origin due to poor documentation. It requires talented developers to analyze the source code, which slows down the process," he said. Another Chinese engineer, who was informed about the use of Huawei processors at Baidu, confirmed frequent crashes of the chips, which significantly hinders AI development.
To tackle the problems, Huawei sends engineers directly to customers to support the transition from Cuda to Cann, according to several sources. Among the supported companies are Baidu, iFlytek, and Tencent. A former Baidu employee emphasized, "Huawei excels in customer service, so it's natural for their engineers to be on-site with major customers to assist them in using the chips.
Huawei leverages its large workforce to expedite the transition. More than half of the 207,000 employees work in research and development, according to the company, including the engineers responsible for customer installations. Tilly Zhang, a technology analyst at consulting firm Gavekal, stated: "Huawei's advantage over Nvidia is its close collaboration with its customers. Unlike Nvidia, the company has a large team of engineers who help customers solve problems and support them in transitioning to their hardware.
Following the stricter US export controls in October, Huawei increased the price of its training chip Ascend 910B by 20 to 30 percent, according to insiders. Despite high demand, customers expressed concerns about potential supply shortages due to production difficulties, as Chinese companies are prohibited from purchasing advanced chip manufacturing machines from Dutch company ASML.
Despite the challenges, Huawei reported strong demand for its AI chips and a 34 percent increase in revenue in the first half of the year, without, however, disclosing detailed figures for individual business segments. More than 50 foundational models have been "trained and developed" on the Ascend chip, said Huawei CEO Zhang Ping'an at the World Artificial Intelligence Conference in Shanghai in July. iFlytek reported that its large language model was trained exclusively on Huawei chips after Huawei sent a team of engineers to its headquarters in Hefei last year to integrate the technology.
