HBM3E, the fifth generation of High Bandwidth Memory, stacks DRAM dies vertically using through-silicon vias (TSV) to meet AI accelerators' bandwidth needs. SK hynix mass-produced the first 12-layer, 36GB HBM3E in September 2024 by thinning each DRAM chip 40%, while Micron and Samsung Semiconductor ship competing designs rated above 1.2 TB/s and up to 1,180GB/s. NVIDIA's H200 GPU pairs 141GB of HBM3e with 4.8TB/s of bandwidth, showing how the technology underpins current AI hardware.
What Is HBM3E and What Advances Does It Bring Over Prior HBM Generations?
Samsung Semiconductor identifies HBM3E as the fifth generation of HBM, engineered for high speed, strong power efficiency, and superior thermal resistanceCITE:E1. In its own product description, Samsung Semiconductor states that "as the fifth generation of HBM, HBM3E achieves exceptional speeds, outstanding power efficiency, and superior thermal resistance"CITE:E1. The company does not attach a numeric figure to these claims in this description, framing them instead as generational characteristics relative to earlier HBM productsCITE:E1.
How Does Through-Silicon Via (TSV) Vertical Stacking Enable High-Density HBM3E?
SK hynix builds HBM3E by stacking DRAM dies vertically with through-silicon via (TSV) technology, after thinning each DRAM chip 40% compared with the prior generationCITE:E2. Describing the process on September 26, 2024, the company states it made each DRAM chip "40% thinner than before and stacked vertically using TSV technology"CITE:E2. This thinning step is what allows more DRAM layers to fit within the same package height, directly enabling the higher-layer-count products described belowCITE:E2.
How Do SK hynix, Micron, and Samsung Semiconductor's HBM3E Specs Compare on Layers, Capacity, and Bandwidth?
SK hynix, Micron, and Samsung Semiconductor each report different HBM3E stack heights, capacities, and bandwidth figuresCITE:E3CITE:E4CITE:E5CITE:E7.
| Vendor | Stack | Capacity | Bandwidth |
|---|
| SK hynix | 12-layer | 36GB | — |
| Micron | 8-high | 24GB per placement | >1.2 TB/s per placement |
| Micron | 12-high | — | >1.2 TB/s per placement |
| Samsung Semiconductor | 12-layer | — | up to 1,180GB/s at 9.2Gbps |
On September 26, 2024, SK hynix began mass production of what it calls the world's first 12-layer HBM3E product, with 36GB of capacity — the largest capacity of any HBM to dateCITE:E3. Micron states that its HBM3E 8-high and 12-high solutions deliver "industry-leading bandwidth of more than 1.2 TB/s per placement"CITE:E4, and that its HBM3E 8-high separately provides 24GB of capacity per placementCITE:E5. Samsung Semiconductor's 12-layer HBM3E stack offers up to 1,180GB/s of bandwidth at a 9.2Gbps data rateCITE:E7.
What Role Does HBM3E Play in AI Accelerators Such as NVIDIA's H200 GPU?
NVIDIA's H200 GPU is the first GPU to combine 141GB of HBM3e memory with 4.8TB/s of memory bandwidthCITE:E6. NVIDIA states directly that "the NVIDIA H200 is the first GPU to offer 141 gigabytes (GB) of HBM3e memory at 4.8 terabytes per second (TB/s)"CITE:E6, positioning the memory subsystem as a defining feature of the accelerator.
What does this mean?
Set side by side, the vendor figures show why HBM3E matters for AI hardware: SK hynix's 36GB single-stack capacityCITE:E3 and Micron's 24GB per-placement ratingCITE:E5 are each smaller than the 141GB NVIDIA packages into the H200CITE:E6, while Samsung Semiconductor's 1,180GB/s per-stack bandwidthCITE:E7 and Micron's 1.2 TB/s per-placement ratingCITE:E4 are each a fraction of the H200's aggregate 4.8TB/sCITE:E6. This gap indicates that an accelerator like the H200 reaches its published totals by combining multiple HBM3E stacks, rather than relying on any single vendor's per-die specification.
Author's Take・EffectStory 編輯部
The specifications reported here point to a straightforward engineering story: HBM3E's gains come from packing more DRAM dies per stack, not from a new memory architecture. SK hynix's move to a 12-layer, 36GB stack by thinning each die 40% shows the industry pushing TSV stacking density as the primary lever, while Micron and Samsung Semiconductor compete on bandwidth per stack rather than capacity alone. NVIDIA's H200 figure of 141GB at 4.8TB/s sits well above any single vendor's per-stack numbers cited above, which means it is assembled from multiple HBM3E stacks rather than one. The metric worth watching next is whether any vendor pushes past SK hynix's 12-layer, 36GB benchmark, since that would signal a further step in TSV stacking density rather than incremental bandwidth tuning alone.