Nvidia Unveils Advanced AI Chip: A Game-Changer for Large Language Models

Nvidia, a leading player in the AI chip market, has introduced its latest chip, the GH200. This announcement comes as the company aims to maintain its dominance in the AI hardware sector, amidst rising competition from giants like AMD, Google, and Amazon.

The GH200 boasts impressive specifications. While it retains the GPU from Nvidia’s top-tier H100 AI chip, it couples this with a whopping 141 gigabytes of state-of-the-art memory and a 72-core ARM central processor. Nvidia’s CEO, Jensen Huang, emphasized the chip’s potential during a conference, stating that it’s tailored for the “scale-out of the world’s data centers.” The GH200 is set to be available for sampling by year-end and will hit the market in the next year’s second quarter.

A significant aspect of AI model development involves two phases: training and inference. Training, as the name suggests, involves feeding vast amounts of data to the model, a process that can span months and demands immense GPU power. Once trained, the model undergoes inference, where it makes predictions or generates content. This step is computationally intensive and occurs frequently, unlike the periodic training phase.

The GH200 is engineered primarily for inference. Its enhanced memory capacity allows it to accommodate larger AI models within a single system. This is a marked improvement from Nvidia’s H100, which has 80GB of memory, in contrast to the GH200’s 141GB. Furthermore, Nvidia has unveiled a system that merges two GH200 chips, catering to even more extensive models.

This development is timely, especially when AMD, Nvidia’s primary GPU competitor, has launched its AI-centric MI300X chip. This chip supports 192GB of memory and is touted for its AI inference capabilities. Other tech behemoths like Google and Amazon are also venturing into crafting bespoke AI chips for inference.