SEMICONDUCTOR INDUSTRY HBM chip shortages put data center growth plans at risk

Related Vendors

dSPACE GmbH

Koki Deutschland Niederlassung KOKI Europe A/S

Schunk Sonosystems GmbH

The chip supply chain faces a new bottleneck as high-bandwidth memory (HBM) chips, crucial for AI processing in data centers, are in short supply due to soaring demand, with manufacturers like SK Hynix and Micron sold out until 2025.

Just as it began to look like the chip supply chain was recovering from COVID-induced shortages, a new bottleneck popped up that could impact the supply of data center GPUs and restrict the expansion plans of data center developers.

This problem came to light in May when South Korean memory manufacturer SK Hynix announced that its supply of high-bandwidth memory (HBM) chips was sold out for 2024 and most of 2025. Micron, one of SK’s competitors, issued a similar statement in March.

A surge in AI applications since late 2022 is thought to be the driving force behind HBM shortages. These applications have significantly increased the demand for HBM chips, and this demand is expected to grow by 200 % this year and potentially double again by 2025. The issue is that HBM chips are difficult to manufacture due to their complexity, and this makes it challenging for manufacturers to quickly ramp up production to meet soaring demand.

AI

The impact of artificial intelligence on the semiconductor industry

Why are HBM chips important?

High-bandwidth memory is used in the actual GPU package itself, with the chips physically sitting next to the GPU silicon. This is different from standard DRAM which is mounted on DIMM sticks and sits next to the CPU. The design of HBM is the key to the performance of AI processing because it provides much greater speed and reduced latency.

And unlike during the COVID-19 pandemic when automakers got around the chip shortage by simply building cars without chips, GPU vendors don’t have this option. An absence of high-bandwidth memory means that GPUs cannot be assembled because HBM must be added to the GPU package at the manufacturing stage. It cannot be retrofitted at a later date.

Potential growth issues for data centers

The shortage of HBM chips is creating a bottleneck in the data center supply chain, which could hinder expansion plans. This is particularly concerning as data centers are critical infrastructure for supporting cloud services and AI applications.

The shortage may also affect the ability of data centers to expand and upgrade their facilities to handle increased AI workloads. This could slow down the deployment of new technologies and services that rely on high-performance computing. At the moment, however, the industry is hanging on and data center construction is continuing at the moment.

In response to the shortage, major DRAM suppliers are expanding their manufacturing capacities. For instance, Samsung and SK Hynix are investing in new facilities and transitioning existing ones to produce more HBM chips.

According to Jon Peddie Research, a tech hardware research firm, data centers that are currently under construction will be built, but going forward, GPU and memory supply issues will be the least of their problems because data center operators are struggling with other issues such as obtaining real estate and establishing sufficient power and cooling.

“The installed base will get to a certain point, it’ll start to approach demand and then customers will shrug it off because they will say we don't need any more boards right now we’ve got enough,” said Peddie. “And not only do we have enough, we have no place to put new ones.”

Power Electronics in the Energy Transition

The parameters for energy transition and climate protection solutions span education, research, industry, and society. In the new episode of "Sound On. Power On.", Frank Osterwald of the Society for Energy and Climate Protection Schleswig‐Holstein talks about the holistic guidance his organization can provide.

Listen now!

(ID:50153476)