NVIDIA's $5 trillion wave! CUDA's moat locks down global AI companies with no one able to escape.

On October 30, Nvidia's market capitalization surpassed $5 trillion, making it the only company in the world to achieve this milestone. However, amidst the shock, a huge confusion arose: if the AI chip industry is so profitable, why is only Nvidia making money? The answer lies in the moat that Nvidia has built over nearly 20 years—CUDA, a unified computing architecture that has locked in over 4.5 million developers worldwide.

NVIDIA's competitors are all top experts, why is there still no one who can rival them?

Does NVIDIA have any competitors? Yes. Many. And each one is a top expert. Looking around, we can at least count three heavyweight “competitors”. For example, AMD. It has been a longtime rival of NVIDIA in the semiconductor field for decades. In terms of technology and experience, it is definitely the most qualified player to go head-to-head with NVIDIA. For instance, Intel. It was once the “chip giant”. With strong manufacturing capabilities and a large customer base. For example, Google. It has almost unlimited resources and the world's top AI team. Moreover, it is also developing its own dedicated AI chips.

Look, among these three giants, which one does not have a name that resonates globally? However, this also makes the problem more acute: clearly, all are masters at the table, but the chips seem to be in the hands of just one player. Why is that?

Let me get straight to the point: NVIDIA has locked in its customers with a system that has extremely high “switching costs.” You might say it's because NVIDIA's chips are the fastest and the best. But in reality, this answer is not rigorous enough. Imagine this scenario: AMD's CEO, Lisa Su, walks into OpenAI's office and makes a very enticing proposal to CEO Sam Altman. AMD has brand new GPU chips that outperform NVIDIA's B200 by 30% and are half the price.

If you were Sam Altman right now, would you sign this “order” or not? If I had to guess, it's likely that Sam Altman wouldn't sign it. At the very least, he would be very, very hesitant. Why is that? Because a cheaper and more efficient option is right in front of him, yet it doesn't entice him? Because the price is just a small part of the whole equation. In business technology investments, you need to consider it from the TCO framework (Total Cost of Ownership). This includes not only the most direct price tag but also various indirect and hidden costs.

the hidden trap with migration costs reaching billions of dollars

If OpenAI really switched from NVIDIA to AMD, what would happen? To get straight to the conclusion, the main costs of labor, code migration, operation, and opportunity will all increase significantly. And any one of these changes could directly determine life or death.

Think about it, if the platform changes, what will happen to those thousands of top engineers? Should they throw away their more than ten years of experience overnight and start learning all over again? How much would that training cost? And what about those millions of lines of code? It's not just a simple “copy and paste.” It's like trying to plant southern lychees in the north. It's not just about digging them up; it requires a lot of research, testing, and significant time investment. AI is the same way; in the end, whether it can succeed is still uncertain.

Additionally, switching platforms means having to operate and maintain two completely different platforms at the same time. The costs involved could very likely double. The most critical and significant risk is the high opportunity cost. In the AI race, every second counts. If switching platforms causes research and development to lag or if the model is released several months late, it could very well turn a leader in the industry into a follower.

OpenAI Hidden Cost List for Platform Switching

Labor Cost: Retraining a thousand engineers, resetting experience, time cost of several years.

Code Migration: Millions of lines of CUDA code need to be rewritten, and testing and verification are time-consuming and labor-intensive.

Dual Platform Maintenance: During the migration period, both systems are operated simultaneously, doubling the cost.

Opportunity Cost: Falling behind competitors in R&D progress may shift from being a leader to a follower.

Risk Cost: Migration failures may lead to a decline in model performance, and the business impact is difficult to quantify.

Therefore, after adding up various direct and indirect costs, the result of “locking in suppliers” is achieved. In simple terms, it means that I am tightly binding everything from software to hardware with you. In the long run, this is actually the optimal solution for AI companies, as they won’t have to worry about hardware during the contract period in the future. Now, looking back at AMD's order of “30% higher performance at half the price”, does it still seem appealing? The answer is no. The millions of dollars saved on hardware are trivial compared to the potential migration costs and strategic risks that could reach billions.

CUDA Ecosystem AI Era Windows

(Source: X)

By now, you should have realized that what really locks customers in for NVIDIA is not its hardware, but an invisible and intangible “cage”. This is known as the “NVIDIA moat” called CUDA. CUDA stands for Compute Unified Device Architecture. In simple terms, it is a set of programming tools that enables programmers to better utilize NVIDIA GPUs.

If we say that NVIDIA's GPU is the “computer mainframe” of the AI era, then CUDA is the “Windows system” of the AI era. Think about it, why has technically superior and free Linux never been able to shake Windows' dominance in the personal desktop market for decades? The answer is not the system itself, but the ecosystem.

Because the massive application ecosystem on Windows is too powerful. From Microsoft's Office to Adobe, and various industry-specific software, they are all closely intertwined with the entire ecosystem. Just imagine, a company that needs to use a lot of specialized software, when faced with the cost of purchasing a Windows license and retraining employees, what choice would they make? The answer is evident.

CUDA, that's how it is, it has an incredibly vast application ecosystem. For many companies and individuals, this is a must-have option. Statistics show that currently, there are over 4.5 million developers worldwide using CUDA for development. And in 2020, this number was still 1.8 million. The monthly download volume of the CUDA toolkit reaches hundreds of thousands.

20 years of high-stakes gambling from being overlooked to becoming irreplaceable

In 2006, when CUDA was introduced, no one paid attention to it, and both Silicon Valley and Wall Street were not optimistic. By 2008, affected by the financial crisis, NVIDIA's stock price had once plummeted by more than 80%, with a market capitalization of around 4 billion. Even within NVIDIA, there were differences regarding the future of CUDA. At the same time, the cost of developing CUDA was also very high. NVIDIA's first GPU that supported CUDA was the G80. To develop this chip, NVIDIA spent a full 4 years, with costs reaching 475 million USD, which accounted for one-third of the total R&D budget for those 4 years.

At that time, it was truly a matter of life and death. What to do? Jensen Huang came up with a solution: throw money at it. The focus was to invest heavily in schools and research institutions. He facilitated the entry of CUDA into universities through donations and equipment, aiming to cultivate users in the education and research fields. In addition, various CUDA R&D centers, teaching centers, and educational courses were established globally. At that time, the annual R&D cost for CUDA was as high as 500 million dollars.

Despite the immense effort and resources invested, CUDA was not favored for a long time. In early 2013, many investment analysts believed that only by abandoning CUDA and returning to the core PC gaming business could NVIDIA's stock price rise. Some even questioned whether CEO Jensen Huang could continue in his role. Looking back now, NVIDIA's CUDA was indeed a gamble. Moreover, he made the right bet.

Why has CUDA transformed from being overlooked to a hot commodity? Because graduates who have learned CUDA are entering tech companies, and the community resources and code libraries for CUDA are becoming increasingly rich. By 2015, there were already 800 universities around the world offering CUDA courses. Over time, the use cases for CUDA have expanded from universities to more fields such as healthcare and commercial applications. As for its “hand-in-hand” connection with the field of artificial intelligence, it can be said that it is purely a “coincidence.”

In 2012, at a global AI image recognition competition initiated by Stanford University, a three-member team from the University of Toronto presented an AI neural network called AlexNet, winning first place. Moreover, their accuracy was 41% higher than that of the second place. How did they achieve this? The team stated that they used 2 NVIDIA GTX 580 GPUs, and they were the only team in the entire competition that trained their neural network using CUDA.

At that time, Google also took notice of this team. They discovered that AlexNet, which used only two GPU graphics cards, achieved results almost identical to what Google obtained using 16,000 CPUs for training. Soon, the entire industry realized that GPUs are the best hardware to support AI. And NVIDIA is likely to become the key to AI development.

The Struggles of Rivals: AMD Open Source, Intel Segmentation, Google Self-Development

Once you understand CUDA and then look at NVIDIA's competitors, you'll find that every step they take is tightly restricted by NVIDIA. For example, AMD chose to go open source. They developed an open-source platform called ROCm, aimed at replacing CUDA. However, it is like “Linux of the AI era”: free, open source, with technical potential, and cheaper. But for users, the migration cost is too high.

For example, Intel has chosen to segment its market. Intel is smart to directly acknowledge that NVIDIA is far ahead in the high-end AI market. Therefore, Intel positions its Gaudi series chips in the niche markets of enterprise-level inference and training of small to medium-sized models. But this also means that Intel has given up the most profitable part of the AI chip market.

For example, Google chose to confront it head-on. Investopedia reported that NVIDIA's gross margin from selling GPUs is about 80%, which is referred to in the industry as the “NVIDIA tax.” To avoid paying high premiums, Google has been developing its own technology since 2015, using a computing ecosystem internally called TPU. In contrast, TPU is deeply integrated with Google's internal platform and does not affect NVIDIA's position.

So you see, those giant competitors are not simply competing on performance, but are strategizing. They are all trying to find ways to bypass CUDA and NVIDIA. But at least for now, no one can shake them. All challengers are going around the mountain, and this itself is the highest tribute to this difficult mountain to climb.

Insights for Entrepreneurs: The Moat is Irreplaceable

NVIDIA has gone from being overlooked to being the center of attention today. This journey has truly been astonishing. Some people online express concerns: NVIDIA's rise is so high, it's too exaggerated, could it be a bubble? Could it be the second Cisco? During the internet bubble in 2000, Cisco was a network hardware supplier, and at its peak, its valuation exceeded 150 times its forward P/E ratio. However, after the bubble burst, it plummeted.

In fact, they have fundamental differences. Cisco is facing a “one-time” construction market. When the early internet completed the “laying of pipes”, Cisco's growth naturally stopped. On the other hand, Nvidia is facing a constantly growing market. At least for now, AI is still growing, so this “arms race” shows no signs of a conclusion. More importantly, Nvidia's clients are giants like Microsoft, Google, and Meta, who are among the wealthiest in the world. For them, purchasing Nvidia's chips is not a choice, but a necessity to survive in the AI era.

Of course, we cannot predict what will happen in the future. Perhaps one day, a new algorithm will suddenly emerge that could render GPUs unimportant and change the entire game rules. But at least for now, we can learn a very important lesson from NVIDIA. What is your moat? It’s not about getting caught up in the question, “Is my product better, faster, or cheaper than my competitors?” but rather asking, “Does my product have an ecosystem that makes customers unable to leave?”

In fact, Nvidia's $5 trillion market capitalization is the loudest answer to this question. It proves the simplest and most important core logic in the business world. The deepest moat is not built on price and performance, but on making you irreplaceable.