16 min • 3 lectures
This course examines the shift from brute-force GPU scaling to architectural efficiency in machine learning hardware and algorithms. Lecture 1 identifies the Compute Trap—the belief that adding more H100 GPUs solves performance limits—and shows how energy and interconnect bottlenecks force a move to Hardware-Software Co-Design. It covers NVIDIA Blackwell’s second-generation Transformer Engine and FP4 precision, contrasts these with Groq’s LPU Deterministic Scheduling, and states the Rule of 10x: any new chip must deliver tenfold efficiency gains, not just higher throughput. Lecture 2 addresses the Density Tax, the high cost of floating-point matrix multiplication. It presents BitNet b1.58’s ternary weights (-1, 0, 1) that replace multiplication with addition and subtraction, and the MatMul-free LLM paper that removes matrix multiplications entirely. The lecture ends with the Quantization First rule: algorithms that cannot be compressed cannot scale. Lecture 3 covers the Context Crunch caused by Transformer quadratic attention costs. It introduces the Infinite Window principle through Mamba’s Selective State Space Models that scale linearly, cites the Jamba hybrid model, and closes with the Linearity Mandate: future architectures must keep compute growth linear with data size. The course therefore moves from physical silicon limits to algorithmic compression to new model structures that together define the next scaling wave.
