SPEAKER_1: Alright, so last episode the big insight was that TSMC is the physical gatekeeper—Nvidia designs the chips, but TSMC manufactures them, and that manufacturing layer became its own binding constraint. Now let's delve deeper into the technological advancements and future potential of HBM, such as HBM4 and beyond, and their implications on AI system performance. SPEAKER_2: Right, and that's where memory enters the picture. Specifically high-bandwidth memory—HBM. The key idea is that modern AI accelerators don't just need raw compute. They need data delivered fast enough to keep the compute busy. If memory can't keep up, the GPU sits idle waiting. SPEAKER_1: So it's not just about storage capacity; it's about the speed and efficiency of data delivery. SPEAKER_2: Nvidia's H100 requires around 3.35 terabytes per second of memory bandwidth. The H200 pushes that to roughly 4.8 terabytes per second. The Blackwell B200 reaches about 8 terabytes per second. Standard DDR5 can't come close to those numbers. SPEAKER_1: So what makes HBM physically different from conventional memory? SPEAKER_2: It stacks multiple DRAM dies vertically and connects them with through-silicon vias—essentially tiny vertical wires running through the chips. That 3D architecture is what delivers bandwidth that conventional flat modules simply can't match. Industry comparisons put HBM3E at roughly 18 to 25 times more bandwidth per stack than DDR5. SPEAKER_1: Mm-hmm. And that premium shows up in the price. SPEAKER_2: Significantly. On a per-gigabyte basis, HBM average selling prices run roughly four to five times higher than conventional DRAM. And HBM costs around six times more per gigabyte than DDR5 specifically. So the performance gap and the price gap are both enormous. SPEAKER_1: Wait—so who actually makes this? Because if HBM is this critical, the supplier list matters a lot. SPEAKER_2: It's a tight oligopoly. Samsung, SK Hynix, and Micron together control roughly 90 to 95 percent of global DRAM output—and that same triopoly controls nearly all HBM supply. There's no fourth option. SK Hynix has been a key partner in advancing HBM technology, holding a significant share of the market and driving innovation in memory solutions. SPEAKER_1: So not just a concentrated market—a market where one supplier has a first-mover partnership with the dominant chip designer. SPEAKER_2: [emphasis] That's the structural fact that makes this investable. SK Hynix's CEO indicated that HBM would reach a double-digit share of the company's DRAM revenue in 2024. Sector analysts project HBM rising to roughly 15 percent of total DRAM sales industry-wide. That's a product-mix shift, not just a volume story. SPEAKER_1: Now, here's the counterintuitive part I want to press on. Memory has historically been one of the most cyclical, commoditized businesses in semiconductors. Boom, bust, repeat. So why does HBM deserve a different valuation framework? SPEAKER_2: Because the demand signal is structurally different. Long-term supply agreements with AI chipmakers and cloud providers turn what used to be spot-price volatility into more predictable, AI-linked cash flows. Higher ASPs, more complex qualification processes, and customer commitments that can't be easily unwound. HBM revenues behave more like specialty infrastructure than commodity memory. SPEAKER_1: And there's a supply-side mechanic that tightens the whole market, not just HBM. SPEAKER_2: Right—this is the part that surprises most people. HBM manufacturing requires significantly more wafer area per gigabyte compared to conventional DRAM, highlighting the unique challenges in production. So every wafer shifted toward HBM removes a multiple of potential DDR4 or DDR5 output from the market. That reallocation quietly pushes conventional memory pricing higher too. SPEAKER_1: So the HBM shortage tightens the broader DRAM market as a side effect. SPEAKER_2: [short pause] Exactly. And the capacity constraint is durable. HBM is effectively sold out for at least 2025 through 2026 among major suppliers. You can't solve an AI capacity problem by simply ordering more GPUs if the memory to feed those GPUs isn't available. The global HBM market is poised for substantial growth, driven by increasing demand for high-performance memory in AI applications. SPEAKER_1: How long does it actually take to expand HBM capacity? Because that's the pressure test from our framework—time-to-supply. SPEAKER_2: Years, not quarters. The stacking process, TSV interconnects, and advanced packaging are all capital-intensive and technically complex. HBM4, the next standard, doubles interface width to 2,048 bits and targets roughly 1.5 to 2 terabytes per second per stack. Building toward that requires entirely new process qualifications. You can't retrofit a commodity DRAM fab overnight. SPEAKER_1: And the demand side keeps escalating. Blackwell platforms are already pushing toward around 288 gigabytes of HBM per GPU with HBM4. SPEAKER_2: Which means memory suppliers capture a disproportionate share of AI system economics. HBM already accounts for more than half the bill of materials cost in Nvidia's Blackwell B200 and B300 accelerators. For someone tracking where value accrues in the AI supply chain, that's a significant data point. The chip gets the headline; the memory captures the margin. SPEAKER_1: So what could actually reverse this? What breaks the memory rerating? SPEAKER_2: Four risks worth naming. Oversupply if all three suppliers expand aggressively at the same time. A slowdown in AI server demand that delays GPU deployments. Customer concentration—Nvidia is a dominant buyer, so any shift in its roadmap matters. And technology substitution, though nothing on the near-term horizon replaces HBM's bandwidth advantage at scale. SPEAKER_1: The takeaway for someone tracking this chain: memory isn't a footnote to the AI compute story. It's a co-equal constraint running in parallel with the chip shortage. SPEAKER_2: And it expands the bottleneck map from individual components to full data-center systems. Once the GPU is fabricated, packaged, and fed with HBM, the entire system still has to run somewhere—and that somewhere needs enormous amounts of power. That's where the rerating chain moves next: the physical infrastructure keeping these data centers alive. SPEAKER_1: So the consolidation story in DRAM is actually older than the AI boom. This isn't a market that got concentrated because of AI—it was already concentrated. SPEAKER_2: Right, and that history matters. Micron's acquisition of Elpida was one of the moves that reduced the field to three major producers. That oligopolistic structure means when all three restrict supply simultaneously, pricing recovers fast. The AI cycle didn't create the triopoly—it just handed the triopoly enormous leverage. SPEAKER_1: Mm-hmm. And the leverage shows up in how they're allocating capacity. They're not just building more—they're redirecting what they already have. SPEAKER_2: That's the key idea. The current DRAM upcycle is being driven by product-mix shifts, not massive greenfield expansion. Suppliers are redirecting capital toward advanced nodes and HBM instead of adding commodity DRAM output. Think of it as the same factory floor producing a more valuable product rather than more units. SPEAKER_1: And that reallocation has a side effect on the broader market. SPEAKER_2: A significant one. HBM consumes roughly three times the wafer area per gigabyte versus conventional DRAM. So every wafer shifted toward HBM quietly removes a multiple of potential DDR4 or DDR5 supply. That's why conventional memory pricing has stayed elevated even outside the AI sector—the HBM pivot is tightening the whole market as a byproduct. SPEAKER_1: Wait—so someone buying a laptop with DDR5 is indirectly feeling the AI memory shortage? SPEAKER_2: [short pause] In pricing terms, yes. The reallocation toward HBM and advanced server DRAM sustains higher pricing across the broader DRAM market. It's not a direct link, but the supply pool for PC and smartphone memory shrinks when fabs pivot toward AI-grade products. SPEAKER_1: Now, for someone tracking this as an investment signal—what's the metric that confirms memory has shifted from commodity cycle to specialty bottleneck? Like, what are they actually watching? SPEAKER_2: Several things moving together. HBM's share of total DRAM revenue at leading suppliers—SK Hynix's CEO indicated HBM would reach a double-digit share of the company's DRAM revenue in 2024. Sector analysts project HBM rising to roughly 15 percent of total DRAM sales industry-wide. Gross margin expansion at the memory suppliers. And long-term supply agreements replacing spot pricing. When those move in the same direction, that's the signature. SPEAKER_1: So not one signal—a cluster. Same confirmation logic we established in the opening episode. SPEAKER_2: Exactly. And the profit dynamics are structurally different from commodity DRAM. Higher average selling prices, more complex qualification processes, AI-linked demand that doesn't swing with the PC cycle. HBM revenues behave more like specialty infrastructure than the boom-bust memory business most investors remember. SPEAKER_1: But here's the pressure test. What actually breaks this? Because every rerating has a reversal scenario. SPEAKER_2: Four risks worth naming. Oversupply if all three suppliers expand aggressively at the same time—that's the classic memory bust pattern. A slowdown in AI server deployments that delays GPU rollouts. Customer concentration, because Nvidia is a dominant buyer and any shift in its roadmap ripples through memory demand. And technology substitution, though nothing on the near-term horizon matches HBM's bandwidth advantage at scale. SPEAKER_1: The customer concentration point is interesting. For example, if Nvidia's next-generation platform shifts its memory architecture, SK Hynix feels that immediately. SPEAKER_2: [emphasis] That's the double edge of a first-mover partnership. SK Hynix is projected to hold roughly half or more of global HBM market share through the mid-2020s, and it secured a majority share of HBM4 supply for Nvidia's upcoming Rubin-generation platforms. That's enormous upside—but it also means the relationship is load-bearing. The partnership that creates the advantage also creates the concentration risk. SPEAKER_1: And the demand side keeps escalating regardless. Blackwell platforms are already pushing toward around 288 gigabytes of HBM per GPU with HBM4. SPEAKER_2: Which shifts economic leverage further toward memory suppliers as AI models demand larger context windows and parameter counts. The global HBM market was valued at roughly 2.9 to 3.9 billion dollars in 2024 and is projected to grow well above 20 percent compounded through 2030. The memory suppliers capture a disproportionate share of AI system economics—the chip gets the headline, the memory captures the margin. SPEAKER_1: The takeaway for someone tracking this chain: memory isn't a footnote to the AI compute story. It's a co-equal constraint running in parallel with the chip and foundry shortages. SPEAKER_2: And it expands the bottleneck map from individual components to full data-center systems. Once the GPU is fabricated, packaged, and fed with HBM, the entire system still has to run somewhere—and that somewhere needs enormous amounts of power. That's where the rerating chain moves next: the physical infrastructure keeping these data centers alive. Vertiv, Eaton, GE Vernova—those names start appearing in the same conversation as Nvidia and SK Hynix because the constraint migrated again.