AI Boom sparks Critical Shortage of Memory Chips
SAN FRANCISCO,oct 26 - The surging demand for artificial intelligence is triggering a supply crunch in high-bandwidth memory (HBM) chips,essential components for AI processing,threatening to slow the rollout of next-generation AI technologies. Leading manufacturers are struggling to meet orders as AI developers race to build and deploy increasingly powerful systems.
The bottleneck centers on HBM, a premium type of memory stacked vertically to deliver significantly faster data transfer speeds than traditional chips. This makes it crucial for training and running large language models (LLMs) like those powering ChatGPT and other AI applications. The current imbalance between supply and demand is expected to persist well into 2024, potentially impacting the availability and cost of AI services and hardware.
Currently, South Korean chipmakers SK Hynix and Samsung Electronics, along with Taiwan’s Micron Technology, dominate the HBM market. SK Hynix currently holds the largest market share, shipping HBM chips to major AI companies like Nvidia.Demand has skyrocketed, with Nvidia alone forecasting a fifteen-fold increase in HBM requirements over the next four years.
“We are seeing unprecedented demand for HBM,” saeid a source familiar with the situation at one of the major memory chip manufacturers, speaking on condition of anonymity. “The entire supply chain is stretched to its limits.”
The shortage is driving up prices. HBM3, the latest generation, now costs around $300 per chip, a considerable increase from earlier this year. Analysts predict further price hikes are unavoidable. This escalating cost impacts not only AI companies but also consumers, as the expense is highly likely to be passed on through higher prices for AI-powered products and services.
The crisis is prompting important investment in expanding HBM production capacity. SK Hynix plans to invest $3.75 billion to build a new HBM fabrication plant in South Korea. Samsung is also reportedly accelerating its HBM production plans. Though, building new fabrication facilities is a lengthy and complex process, meaning substantial increases in supply won’t materialize for at least 18-24 months.
Beyond HBM, the AI boom is also straining the supply of other memory types, including High Bandwidth Memory 2e (HBM2e) and Graphics double Data Rate 6 (GDDR6). This broader shortage underscores the basic challenge of scaling the infrastructure needed to support the rapidly expanding AI landscape.
The situation highlights the critical role of memory chips in the future of AI and the potential for supply chain vulnerabilities to hinder innovation. As AI continues to permeate more aspects of daily life, securing a stable and sufficient supply of these essential components will be paramount.