OpenClaw: Advanced Memory Architectures
Lecture 2

The Great Allocation War: Heap vs. Pool

OpenClaw: Advanced Memory Architectures

Transcript

SPEAKER_1: Alright, so last time we established that OpenClaw's whole foundation is about virtualizing the 1997 memory environment — replacing DirectDraw with SDL2 so the old engine logic doesn't know the ground has changed. That was the prerequisite. Now I want to get into what happens above that layer — specifically how OpenClaw actually manages memory at runtime. SPEAKER_2: Right, and that's where things get genuinely interesting. Once you've severed the OS-specific dependency, you're free to make real architectural choices. And the first big one is: when an object needs memory — a projectile, an enemy, a particle — how does the engine hand that out? That's the heap-versus-pool question. SPEAKER_1: So for someone like Sergey who's comfortable with systems programming, the instinct might be — just use malloc, it's flexible. What's the actual problem with that in a game loop? SPEAKER_2: The problem is time and fragmentation. A standard malloc call isn't O(1) — it has to search a free list or a buddy system for a block that fits, which is typically O(log n) or worse under pressure. In practice that's 10 to 100 microseconds. A pool allocator, by contrast, just pops a pre-reserved slot — that's closer to sub-microsecond. tcmalloc and similar pool-based systems hit under one microsecond consistently. SPEAKER_1: Wait, why does malloc need to search at all? Can't it just... give you the next free address? SPEAKER_2: It could if memory were never freed in the middle. But in a game loop, objects are constantly being born and dying — projectiles, enemies, effects. After a while, the heap looks like Swiss cheese: allocated blocks with gaps between them. That's external fragmentation. A large contiguous allocation can fail even when total free memory is sufficient, because the free chunks aren't adjacent. SPEAKER_1: And how bad does that get? Is this a theoretical concern or does it actually bite you? SPEAKER_2: It bites hard. Long-running heap-based applications can see 50% or more of their memory effectively wasted due to fragmentation — without any compaction step. Heaps do try to fight this: the buddy system splits memory into power-of-two blocks to make merging easier, and deallocation has to coalesce adjacent free blocks. But that coalescing is overhead on every free call. SPEAKER_1: So pool allocators sidestep all of that. How exactly? SPEAKER_2: A pool pre-reserves a fixed block of memory upfront — say, 512 slots for projectile objects. Every slot is the same size. When you need a projectile, you grab the next available slot — no search, no metadata per allocation beyond a simple index or pointer. When you free it, the slot goes back to the pool immediately. No fragmentation because the slots never move and never vary in size. SPEAKER_1: In OpenClaw specifically, what object types actually benefit from this? Our listener might be wondering whether this is a niche optimization or something that touches most of the game's runtime. SPEAKER_2: It touches the hot path directly. The main candidates are projectiles — Claw fires constantly — enemies, which spawn and despawn per room, particles and visual effects, and collision event objects. Those four categories cover the vast majority of runtime allocations in a 2D action platformer. They're all fixed-size, high-frequency, and short-lived — exactly the profile a pool is designed for. SPEAKER_1: There's also a cache angle here, right? It's not just about allocation speed. SPEAKER_2: Exactly — and this is underappreciated. When all projectile objects live in a contiguous pool, iterating over them to run physics or collision checks means sequential memory access. The CPU prefetcher loves that. With heap allocation, those objects could be scattered anywhere, so each access is potentially a cache miss. At 120 frames per second, that adds up fast. SPEAKER_1: What about arena allocation? I've seen that mentioned as even faster than pools — why isn't that the obvious answer? SPEAKER_2: Arenas are fast because they're essentially a bump pointer — you just increment an offset for every allocation, no tracking at all. The risk is that you can't free individual objects. You free the entire arena at once. That works beautifully for a level-load phase where you allocate everything and then tear it all down. But for mid-game object churn — enemies dying one at a time — you'd be leaking memory until the next full reset. It's a sharp tool. SPEAKER_1: Here's something counterintuitive our listener might trip over: if pool allocators are faster, does that mean they use less memory? Because that's not obvious to me. SPEAKER_2: It's actually the opposite, and that's the real trade-off. A pool pre-reserves all its slots whether they're used or not. If you allocate 512 projectile slots but only 20 are ever active simultaneously, you've committed memory for 492 idle slots. Heap allocation only uses what's actively allocated. So pools trade memory efficiency for speed and predictability — which is the right trade in a real-time game, but it's not free. SPEAKER_1: So the heap isn't useless — it's just the wrong tool for the high-frequency objects. SPEAKER_2: Precisely. Heap allocation is still the right choice for variable-size, infrequent allocations — level geometry, asset data, configuration structures. The rule of thumb is: measure your allocation patterns first. If an object type is allocated and freed hundreds of times per second and has a fixed size, pool it. Everything else, the heap handles fine. SPEAKER_1: One more thing — thread safety. Does pooling help there too? SPEAKER_2: It can, yes. Memory pools can be made thread-local, which eliminates synchronization overhead entirely. With a shared heap, every malloc and free needs a lock or atomic operation to stay consistent across threads. A thread-local pool sidesteps that completely — each thread manages its own slots. That's part of why Linux's kernel uses kmem_cache, which is essentially a slab allocator — a pool with per-slab metadata — for hot kernel objects. SPEAKER_1: So for our listener taking all of this in — what's the one thing they should hold onto from this lecture? SPEAKER_2: Pool allocators reduce fragmentation and allocation overhead by pre-reserving fixed-size blocks for specific object types — projectiles, enemies, particles, collision events. In OpenClaw's high-refresh-rate loop, that predictability isn't a micro-optimization. It's what keeps the physics deterministic and the frame budget intact. The heap is flexible; the pool is fast. Knowing which one to reach for, and when, is the actual skill.