Nvidia's ambition: "AI native" completely subverts the data center

Image source: Generated by Unbounded AI

Source: Wall Street News

Author: Zhao Ying

Nvidia CEO Jensen Huang said so at a press conference on Tuesday. Yesterday, Nvidia released a new generation of GH200 Grace Hopper super chip platform, which is specially designed for the era of accelerated computing and generative AI.

Huang Renxun pointed out that in order to meet the growing demand of generative AI, the data center needs to have an accelerated computing platform for special needs. The new GH200 chip platform offers superior memory technology and bandwidth, improved ability to connect GPU aggregates without loss, and has a server design that can be easily deployed throughout a data center. **

It is worth mentioning that the arrival of the wave of large-scale models has spawned various AI native applications, leading to a surge in demand for computing power. The data center market dedicated to data-intensive artificial intelligence applications is rapidly emerging.

The data center ushers in new changes

Analysts note that as established cloud computing providers race to retrofit data centers with advanced chips and other upgrades to meet the demands of artificial intelligence software, some upstart builders see an opportunity to build new facilities from scratch. Chance.

A data center is similar to a large warehouse, equipped with multiple racks of servers, networks and storage equipment for storing and processing data. Compared with traditional data centers, AI data centers have more servers using high-performance chips, so the average power consumption per rack of AI data center servers can reach 50 kilowatts or more, while each rack of traditional data centers The power consumption is about 7 kW.

This means that the AI data center needs to build more infrastructure that can provide higher power. Since the additional power consumption will generate more heat, the AI data center also needs other cooling methods, such as liquid cooling systems, to Protect the device from overheating.

Manju Naglapur, senior vice president at services and consulting firm Unisys, noted:

**Purpose-built AI data centers can house servers utilizing AI chips such as Nvidia’s GPUs, allowing multiple computations to run concurrently as AI applications sift through vast data stores. ** These data centers are also equipped with fiber optic networks and more efficient storage devices to support large-scale artificial intelligence models.

AI data centers are highly specialized buildings that require a large investment of money and time. According to data from research firm Data Bridge Market Research, by 2029, spending on the global artificial intelligence infrastructure market is expected to reach US$422.55 billion, with a compound annual growth rate of 44% over the next six years.

DataBank chief executive Raul Martynek said the pace of AI deployment is likely to lead to a shortage of data center capacity in the next 12 to 24 months.

AI Computing Rookie Receives $2.3 Billion in Financing

At present, various giants are betting on AI data centers, and the “real estate benchmark” Blackstone sells houses and switches to AI data centers. Meta has also said it will build a new AI data center.

As mentioned in the previous article, CoreWeave, a rookie in AI computing power, took a mortgage loan from Nvidia H100 and obtained debt financing of 2.3 billion US dollars (about 16.5 billion yuan).

**CoreWeave said that the funds will be used to accelerate the construction of artificial intelligence data centers. This is another financing after the company received US$221 million in April this year and US$200 million in May. Founded six years ago, CoreWeave already has seven AI data centers online and expects to double that by the end of this year.

CoreWeave is working with Nvidia and Inflection AI to build a super-large AI server cluster, with the goal of running 22,000 Nvidia H100s. **If completed, it will become the world’s largest AI server cluster. **

It is worth mentioning that, according to CoreWeave’s official website, their services are 80% cheaper than traditional cloud computing vendors. Nvidia’s latest HGX H100 server, which contains 8 H100s with 80G video memory and 1T memory, starts at only $2.23 per hour (16 RMB). **

Compared with the previous generation platform, the dual-chip configuration of the new GH200 Grace Hopper platform increases the memory capacity by 3.5 times and the bandwidth by three times. One server has 144 Arm Neoverse high-performance cores, 8 petaflops of AI performance and the latest HBM3e of 282GB memory technology.

No wonder that in this era of LLM explosion, Huang Renxun still boldly said “the more you buy, the more you save”!