SemiconductorsFEATURE

Why AI GPUs Are Scarce and Expensive: Unpacking the HBM and CoWoS Bottlenecks

林紀旭 James LinEditor-in-Chief
Published · Updated
AI GPUs stay scarce and expensive because production runs through several constrained stages at once, not one single choke point. HBM supply is concentrated among SK Hynix, Samsung Electronics, and Micron; CoWoS advanced packaging capacity sits with TSMC (台積電); and vertical integration across chip, memory, packaging, and software gives suppliers pricing power. Any one tight stage caps shipments regardless of raw chip fabrication capacity.

What makes AI GPUs scarce across multiple stages of production?

AI GPUs such as NVIDIA's data center accelerators are expensive and hard to obtain because several scarce stages stack on top of each other rather than a single causeCITE:E1. Advanced process node capacity, High Bandwidth Memory (HBM) supply, and CoWoS advanced packaging each act as an independent constraint, and a bottleneck at any one of them limits how many finished GPUs can shipCITE:E1.

Why is HBM supply concentrated among just three makers?

HBM supply for AI GPUs is concentrated among SK Hynix, Samsung Electronics, and Micron, with SK Hynix holding a 58% share of the HBM supplier market in the first quarter of 2026CITE:E2. AI GPUs require large volumes of HBM to keep their compute cores fed with data, and evidence describes overall HBM capacity across these three suppliers as tightCITE:E2.

Why does CoWoS packaging capacity cap GPU shipments?

TSMC's (台積電) CoWoS advanced packaging integrates a GPU die with multiple HBM stacks, and CoWoS capacity functions as the practical ceiling on how many AI chips can shipCITE:E3. Expansion of CoWoS capacity has not kept pace with demand, which evidence identifies as a persistent constraint on AI chip outputCITE:E3.

Why has demand for AI GPUs moved faster than supply can follow?

Data center revenue at AI chip makers and capital expenditure at major cloud providers have moved upward together, with large buyers competing for the same limited production capacityCITE:E4. This simultaneous increase on the demand side pushes prices and delivery times higher on top of the supply-side constraints from HBM and CoWoSCITE:E4.

Why do GPU suppliers retain such strong pricing power?

AI GPU suppliers bundle the chip, HBM, advanced packaging, and the CUDA software ecosystem into one integrated system, and this vertical integration gives them strong pricing powerCITE:E5. Evidence attributes very high gross margins on AI accelerators to this structural bundling, which is described as a core reason prices remain elevated rather than fallingCITE:E5.

What does a "GPU shortage" actually mean in practice?

A reported "GPU shortage" is usually not a shortage of the compute die itself but a shortage of HBM supply or CoWoS packaging capacityCITE:E6. Identifying which of these two stages is the actual constraint at a given time is what determines when shipment shortages can be expected to easeCITE:E6.

What this means

Taken together, the evidence traces AI GPU scarcity to three layers that move independently: HBM supply concentrated among three makers with SK Hynix at 58% shareCITE:E2, TSMC's CoWoS packaging capacity lagging demandCITE:E3, and data center revenue and cloud capex rising in tandem on the demand sideCITE:E4. Because the chip, HBM, packaging, and CUDA ecosystem are sold as one bundle, the same suppliers that face these capacity constraints also hold the pricing power that keeps margins highCITE:E5. That structure is why a "GPU shortage" headline usually points to an HBM or CoWoS constraint rather than a shortage of the processor die itselfCITE:E6.

📊 Evidence

FAQ

What makes AI GPUs scarce across multiple stages of production?

AI GPUs such as NVIDIA's data center accelerators are expensive and hard to obtain because several scarce stages stack on top of each other rather than a single…

Why is HBM supply concentrated among just three makers?

HBM supply for AI GPUs is concentrated among SK Hynix, Samsung Electronics, and Micron, with SK Hynix holding a 58% share of the HBM supplier market in the firs…

Why does CoWoS packaging capacity cap GPU shipments?

TSMC's (台積電) CoWoS advanced packaging integrates a GPU die with multiple HBM stacks, and CoWoS capacity functions as the practical ceiling on how many AI chips …

Why has demand for AI GPUs moved faster than supply can follow?

Data center revenue at AI chip makers and capital expenditure at major cloud providers have moved upward together, with large buyers competing for the same limi…

📎 Sources

  1. en.wikipedia.org
  2. effectstory.com
  3. effectstory.com
  4. effectstory.com
  5. effectstory.com
  6. en.wikipedia.org

Related data

Author's Take林紀旭 James Lin

The number worth tracking is SK Hynix's 58% share of HBM supply, because it means the AI GPU bottleneck is currently concentrated in a market with only three suppliers rather than spread across dozens of chipmakers. That concentration, combined with TSMC's CoWoS packaging capacity lagging demand, explains why GPU pricing power sits with the suppliers rather than the buyers bidding up data center and cloud capex together. The single indicator to watch next is whether SK Hynix's share moves materially away from 58% or CoWoS capacity additions start closing the gap with demand — either shift would be the first sign that the current bottleneck structure is loosening, rather than just shifting from one constrained stage to another.

林紀旭 James LinEditor-in-Chief

Related

FEATURE

NVIDIA Q2 FY2027 Results: Revenue Hits $96.2B, Up 106% as Data Center Tops $89B

NVIDIA reported Q2 FY2027 revenue of $96.2 billion, up 106% year-over-year and above the roughly $92 billion consensus, with Data Center revenue of $89.0 billion, up 117% and about 92% of total sales. Gross margin held at 75.0% and non-GAAP EPS of $2.22 beat the $2.10 estimate. Q3 guidance of $108.0 billion excludes any China Data Center compute revenue.

林紀旭 James Lin ·
BRIEF

SpaceX and NVIDIA Plan to Launch Orbital AI Data Centers Starting Next Year, Musk Says

SpaceX plans to launch its first NVIDIA-chip-powered AI satellites, Starmind AI1, in the fourth quarter of 2027, reaching large-scale deployment by 2028 — at least a year ahead of the original schedule. The company has also filed with the FCC for a network of up to 1 million satellites, while Taiwanese suppliers Unitech and Sesoda report rising satellite-related revenue and order shares tied to the buildout.

林紀旭 James Lin ·
BRIEF

NVIDIA Launches Jetson Orin Nano 2 Robotics Computer, Targets Entry-Level Edge AI

NVIDIA announced the Jetson Orin Nano 2 robotics computer on August 25, 2026, packing 78 TOPS of AI compute, 8GB of memory, and an 8-core Arm CPU while doubling inference performance and cutting power draw 40% versus its predecessor. Modules and developer kits ship in the first half of 2027, targeting a robotics developer base NVIDIA says already tops 3 million.

Nathan ·