Nvidia, the undisputed market leader in AI chips, is increasingly facing growing competition. But while competitors hope to gain a foothold in this rapidly evolving market, Nvidia shows no signs of losing its dominant position. On the contrary, the company continues to expand its presence and influence.

A decisive turning point in the competition for the AI market may be imminent. However, the question of whether and when Nvidia might lose market share to its competitors remains open. The AI market is evolving rapidly, and while training large "foundation" models that underlie modern AI systems continues to require immense computational power, the so-called "inference" – running queries against these models – could present an opportunity for competitors to bring less powerful and cheaper chips to the market.

However, it does not seem that Nvidia is losing its lead. When announcing its most recent quarterly results, the company stated that more than 40 percent of its data center revenue last year was attributable to inference, amounting to more than 33 billion US dollars. This figure exceeds Intel's total data center revenue by more than two and a half times.

But how the inference market will develop remains uncertain. Two crucial questions are up in the air: Will the AI market continue to be dominated by a race for ever larger models, and where will inference take place in the future?

Nvidia has greatly benefited from this race for ever-larger models. Jensen Huang, CEO of Nvidia, stated that each new generation of large AI models requires "10, 20, 40 times more computational power," which guarantees enormous demand for Nvidia's upcoming Blackwell chips. These new processors are also supposed to provide the most efficient way to perform inference against these "multi-trillion parameter models.

It is, however, unclear whether ever-larger models will continue to dominate the market or if these models will eventually hit efficiency limits. At the same time, smaller models that promise similar benefits, as well as specialized models for narrower application areas, are gaining in popularity. Meta, for example, recently claimed that its new Llama 3.1 model can match the performance of advanced models like OpenAI's GPT-4 despite its smaller size.

Improved training methods and the use of larger amounts of high-quality data have also contributed to increased efficiency. After training, the largest models can be "distilled" into smaller versions. These developments could result in more inference work being done in smaller data centers or on devices such as smartphones and PCs. "AI workloads will increasingly move to where the data or the users are," says Arun Chandrasekaran, an analyst at Gartner.

The number of competitors targeting this new market is rapidly growing. For example, the company Qualcomm was the first to develop chips for a new class of AI-enabled PCs, challenging the long-standing PC chip market leader, Intel.

It is inevitable that Nvidia will lose market share as AI inference shifts to devices where the company is not yet represented and to the data centers of cloud companies that prefer their own chip designs. To defend its position, Nvidia relies heavily on its software strategy, which has long served as a protective barrier around its hardware and makes it easier for developers to use its chips.

Nvidia is currently working on a broader range of enterprise software to help companies develop applications that optimally utilize AI. This would also secure the demand for Nvidia’s chips. The company expects the revenue from this software to reach an annual run rate of $2 billion by the end of the year. Although this is relatively small compared to the expected total revenues of over $100 billion, it indicates an increasing entrenchment of the technologies, which enhances the "stickiness" of the products.

Although the AI chip market could enter a new phase, Nvidia's strong grip shows no signs of loosening.