The AI wave continues to sweep across the global semiconductor industry. As the core cornerstone of AI computing power, the technological development and breakthroughs of HBM directly concern the future AI competitive landscape. At the recently held Hot Chips 2026 conference, the three major memory giants—Samsung, SK Hynix, and Micron—appeared on the same stage, each unveiling their latest insights and technological directions in the HBM field.
01. Samsung Unveils zHBM: DRAM Stacked Directly on GPU
Samsung disclosed a three-stage evolution roadmap for HBM at the conference, progressing from "freeing up space" to "expanding functions" and then to "3D stacking," culminating in zHBM. The goal is to upgrade the HBM Base Die from a passive data transit station to an active co-processor with computing capabilities.
Phase 1: Freeing Up XPU Area (cHBM4)
Samsung moves some functions, such as memory controllers, originally placed inside AI chips like GPUs/TPUs, to the Base Die of the HBM, thereby freeing up 5%-10% of the XPU chip area and bringing a 10%-20% performance improvement. Meanwhile, Samsung introduces D2D PHY in HBM4 to replace the traditional HBM PHY, shortening the data transmission path and reducing the area and power consumption of the interface circuitry. Furthermore, to address the local hot spot issues caused by advanced process nodes, Samsung simultaneously launches HPB technology, which can reduce the peak temperature by over 35%.
Phase 2: Expanding Functions and Capacity (cHBM4E/HBM5)
Samsung integrates memory expansion controllers and PHYs in the idle space of the Base Die to directly expand memory capacity, meeting the growing demand for KV Cache in AI large models. At the same time, it integrates advanced RAS sensors, real-time telemetry, and other intelligent functions. Some computing tasks can be offloaded to the Base Die, reducing the power and bandwidth consumption of data movement.
Phase 3: True 3D Integration—zHBM
Image source: Wccftech
The ultimate form, zHBM, stacks the HBM directly vertically above the XPU, eliminating the interposer required for 2.5D Advanced Packaging. It adopts a distributed I/O architecture to shorten the data transmission distance and removes SerDes through architectural optimization. Compared to standard HBM4E, zHBM can achieve a 70% reduction in power consumption and a 230% increase in DRAM bandwidth, saving 100W of power per DRAM module, while freeing up 8.3% of additional power space for the GPU. To realize zHBM, Samsung is developing Advanced Packaging technologies such as WoW (Wafer-on-Wafer) and HCB (Hybrid Cube Bonding).
02. SK Hynix's Diversified Layout: Advancing on Three Fronts with MR-MUF, Hybrid Bonding, and EMIB
The HBM blueprint disclosed by SK Hynix covers a comprehensive upgrade from packaging process iteration, local thermal optimization, and diversified Advanced Packaging technology layout to the long-term goal of 3D integration, setting clear performance targets for HBM4.
SK Hynix is currently fully applying advanced MR-MUF technology in its 16-layer HBM3E solution and has successfully introduced new processes such as warpage control, fine-pitch interconnection, and narrow-gap filling. The advantages of this technology lie in its high production efficiency and low thermal resistance. However, the existing technology is facing the challenge of a 2.2x thermal burden and the continuous increase in TSV area. To this end, SK Hynix is accelerating the exploration of hybrid bonding technology, aiming to break through the 16-layer stacking limit and move towards 20 layers or more. The improvements brought by hybrid bonding are highly significant: the core die thickness can be increased by 24%, and the TSV pitch can be reduced to below 18 microns, reducing thermal resistance by about 35% while increasing the number of stacking layers.
SK Hynix has introduced iHBM technology to optimize the issue of the highest heat concentration in the D2D PHY area inside the HBM. This technology directly embeds integrated cooling elements and builds dedicated heat dissipation paths within the HBM package, which can reduce thermal resistance by an additional 30% or more compared to traditional solutions. This technology is planned to be applied starting from the 8th generation HBM, namely HBM5.
In this roadmap, SK Hynix places Intel's EMIB on a par with TSMC's (Taiwan Semiconductor Manufacturing Company) CoWoS technology. EMIB technology achieves high-density interconnection between chips by embedding micro silicon bridges within the packaging substrate, offering advantages such as design flexibility, cost reduction, and yield improvement. The background of this move is that more and more customers intend to choose Intel's foundry services, and SK Hynix hopes to meet the diverse needs of future customers by supporting EMIB.
SK Hynix's long-term goal is to stack HBM directly above AI accelerators to form a true 3D integration architecture, thereby significantly shortening the distance between memory and computing units, and further improving bandwidth and energy efficiency. To achieve this goal, SK Hynix has planned multiple technical paths, including doubling the number of TSVs to increase I/O density, combining logic process integration to boost I/O speed, and optimizing the power delivery network to address power consumption challenges.
In addition, SK Hynix also revealed the product planning and performance specifications for HBM4 at the conference. HBM4 adopts a 24Gb DRAM density, supports 2048 I/Os, and has a maximum I/O speed of up to 8Gbps. The bandwidth per stack can exceed 2TB/s, and the power efficiency is improved by over 40% compared to HBM3E, with thermal resistance improved by more than 14%. In terms of package dimensions, the height of HBM4 is approximately 775 microns, with a size of 12.8×11 mm, integrating over 20,000 TSVs and more than 16,000 bottom microbumps. Currently, the 12-layer product has entered mass production, while the 16-layer product is still in the customer qualification phase, with the latter capable of reaching a maximum capacity of 48GB.
03. Micron Discusses HBM Challenges: Thermal Management, Area, and Reliability
Raghu Sreeramaneni, a researcher in HBM design architecture at Micron, revealed the challenges faced by HBM in pursuing high performance at the Hot Chips 2026 conference, mainly including area and cost, reliability and faults, and thermal management.
In a typical GPU system, the area of HBM memory chips accounts for about 90% of the total silicon area. Moreover, to produce the same capacity, HBM consumes approximately 3 times the number of wafers compared to DDR5.
Memory faults have become a "stumbling block" for AI training. Citing data from Meta, Micron pointed out that during the training of the Llama 3 large model, up to 17% of unexpected interruptions were caused by HBM faults.
As data transmission speeds accelerate, thermal management has become the primary consideration in design. In the multi-layer stacking structure of HBM, the bottom layer generates the most heat, while cooling is applied at the top, meaning heat must penetrate the entire stack to dissipate.
Sreeramaneni pointed out that HBM needs to integrate all design, process, and Advanced Packaging technologies into a chip no larger than a postage stamp. Further optimization of HBM requires memory manufacturers, GPU design companies, foundries, and packaging houses to engage in co-design from the very beginning.
Micron is developing a new generation of Advanced Packaging solutions and exploring memory-optimized SerDes and Die-to-Die PHY for faster I/O, larger System-in-Package (SiP), and Advanced Packaging technologies such as glass substrates. These technologies aim to solve the challenges of power density and thermal management through disruptive process and packaging technologies.
04. Advanced Packaging: The "Second Battlefield" for HBM
Since its inception, HBM technology has been deeply bound with Advanced Packaging.
A standard HBM module consists of multiple layers of DRAM Core Dies vertically stacked through Through-Silicon Vias (TSVs), with a Base Die at the bottom responsible for controlling the transmission of data, power, and signals between the DRAM and the GPU. HBM and GPU are two independent chips, mounted together on the same silicon interposer through 2.5D Advanced Packaging. Each HBM module provides 1024 I/Os and 16 channels through the silicon interposer, connected to the XPU (accelerator processor) via PHY.
This 2.5D Advanced Packaging architecture is precisely the key to HBM's ability to achieve ultra-high bandwidth. Given the explosive growth in AI computing power demand, this mature architecture is facing multiple challenges. The bandwidth nearly doubles every two generations, bringing a huge thermal burden to existing packaging technologies; the number of TSVs and microbumps continues to rise (HBM4 has integrated over 20,000 TSVs and more than 16,000 microbumps); and the number of stacking layers is approaching the physical limit of 16 layers. Relying solely on increasing the number of stacking layers or improving I/O speed is no longer sufficient to support the continuous expansion of HBM.
For this reason, major memory manufacturers have reached a key consensus: GPU, memory, wafer foundry, and packaging must be designed as a complete system. Advanced Packaging is gradually becoming the second battlefield for HBM competition.
Samsung is advancing on two fronts with HCB and zHBM, attempting a comprehensive reconstruction from the bonding process to the packaging architecture; SK Hynix is evolving towards hybrid bonding on the mature foundation of MR-MUF, while embracing the diversified routes of EMIB and CoWoS; Micron is validating the feasibility of 16-layer stacking in the mass production of HBM4, while also exploring more long-term packaging directions such as glass substrates.
The emergence of HBM may be changing the underlying logic of the memory industry. Memory chips are gradually transforming from temporary data storage carriers into a critical link in the AI computing power chain, and the industry's pricing power, competitive landscape, and technological direction may change accordingly. Driven by the demand for AI computing power, technological breakthroughs are gradually becoming an important variable in the next stage of competition. This race surrounding HBM is far from over.
#HBM #Samsung Electronics #Micron #SK Hynix