Capping the allocator's cache: how 0.36.0 grew a small program's memory, and how 0.37.0 takes it back

Every Xo release is measured against the one before it and against Go, on the same programs. Xo 0.36.0’s comparison flagged one result we did not expect: coll putget, a small program that fills and reads a map, peaked at 1.18 times its 0.35.0 memory on macOS and 1.32 times on Linux. Its speed had not changed. This post covers how we found where the memory went, what we changed for 0.37.0 (the next release), and how Go handles the same problem. Every figure links to the committed file it comes from.

What the release benchmark saw

coll is bench/perf/xo, a set of collection workloads; putget makes 3 million puts over 50,000 map keys and reads them back, and push appends 3 million integers to a list and sums them. Against 0.35.0, the release run measured (release-vs-go-v0.36.0.md, lines 33 to 38):

• macOS putget: peak memory 1.182 times 0.35.0’s (7,280 to 8,608 KB), wall time within noise.
• macOS push: 1.101 times (50,928 to 56,064 KB).
• Linux putget: 1.317 times (10,352 to 13,634 KB), wall time within noise.

A program that does the same work at the same speed but holds more memory at its peak is holding memory it is not using.

Where it went

Two bisects over the 0.36.0 commits named two changes (decision 0448, lines 1062 to 1063):

1. A new list growth curve (c78bc379d, decision 0452). Lists now grow the way Go’s slices do: doubling up to 256 elements, then about 1.25 times. That saves memory in large programs, but a list that keeps growing reallocates more often, and each reallocation frees the old block.
2. Page and large blocks (5523fb8dd, decision 0425). Blocks of 16 KiB and over got their own memory, and a freed large block’s mapping is kept in a cache (up to 16 mappings and 64 MB) so the next large allocation can reuse it instead of asking the kernel again.

Together they explain the rise. Every reallocation of a growing list frees a large block, and the cache keeps it. A kept mapping’s pages stay resident, and resident pages count toward peak memory whether or not the program uses them. At exit, putget held 14 cached mappings (8 MB) for a heap that peaked at 6.4 MB (decision 0448, lines 1064 to 1067). The extra memory was not live data. It was freed memory kept for reuse.

Two fixes

Grow with room to spare. A list or string that grows past the heap’s size classes now reserves room for four times its size, not twice (8238745f4). It reallocates less often, so it frees fewer blocks to cache.

Cap the cache. Removing the cache outright fixed the memory: putget’s peak fell to 0.624 of the build with the cache on macOS and 0.683 on Linux. But the compiler itself needs the cache. Without it, Linux builds of coll ran 1.039 times as long, a concurrency benchmark’s build 1.072 times, and the compiler’s build of itself 1.018 times (decision 0448, lines 1069 to 1071). So 0.37.0, the next release, keeps the cache but bounds it by the program’s size: a freed mapping is kept only while the cache stays under a quarter of the bytes the program has in use. A small program caches little. The compiler, with tens of megabytes to gigabytes in use, keeps all of its reuse.

The results

Peak memory (maximum resident set size, KB, median of 7 runs) of the same coll binary built at each step, and of the same workloads written in Go (bench/perf/go), on macOS; the 0.37.0 row is b6421a8c5, the build that ships as 0.37.0, the next release (alloc-cache-cap-2026-11-03.md, lines 44 to 56 and 104 to 116):

On Linux (arm64, transparent huge pages on, the default huge page mode), the same builds; 0.37.0 also ships decision 0469 (see below) (alloc-cache-cap-2026-11-03.md, lines 79 to 102 and 104 to 116):

0.36.0’s putget is bimodal on Linux (about 12,900 or 14,400 KB a run), which is why its median here differs from the release run’s 13,634 KB. Go’s push varies the most: from 66,236 to 88,380 KB across these 7 runs, and from 55,992 to 103,240 KB in the job that measured 0.37.0. macOS has no huge page advice, so decision 0469 changes nothing there.

The compiler keeps its speed. With the cap, the build of coll without a binary cache ran at 0.991 of the uncapped time on macOS, the compiler’s build of itself at 1.000 on macOS and 1.003 on Linux, and the Linux edit builds within noise (decision 0448, lines 1077 to 1087).

How Go bounds it

Go solves the same problem differently, and the comparison is worth making exactly, so the figures below come from Go 1.26.1’s runtime source.

Go does not free an object when the program stops using it. The heap grows until it reaches a heap goal, then a collection runs. With the default GOGC=100 the goal is the heap that survived the last collection plus 100% of it (plus stacks and globals): about twice the live heap (mgcpacer.go, lines 1292 to 1304), and never under 4 MiB (mgcpacer.go, lines 58 to 60).

Pages that a collection frees are returned to the kernel by the scavenger. Its goal is to keep the memory Go retains at about 10% over what the heap goal implies (retainExtraPercent = 10), and its background worker uses about 1% of the program’s CPU time (scavengePercent = 1). It also skips chunks of the heap that were densely used in the last cycle, since they are likely to be reused and returning them would break up huge pages (mgcscavenge.go, lines 19 to 26, 70 to 78 and 103 to 126).

So Go also keeps freed memory for reuse, bounded relative to the heap, and returns it gradually in the background. Xo frees memory as soon as a value’s last reference goes, so its live bytes track the program’s real use closely. Its bound on large blocks is simpler: a cap on the freed blocks it keeps, checked when a block is freed. The quarter is our choice for 0.37.0, not a figure derived from Go’s.

Huge pages, and what is left

On Linux, part of the peak is huge pages: the heap asks the kernel for 2 MiB pages once a thread allocates densely (decision 0456), and until now the main thread’s first 4 MiB chunk asked for them from the start, so even a small program’s first allocations could fault in a whole 2 MiB page. Decision 0469 drops that exception for 0.37.0: the main thread asks for huge pages by the same rule as every other thread. With the cache capped, putget peaked at 10,704 KB in the default mode; without the exception it peaks at 8,696 KB, and with huge pages turned off (XO_THP=off) at 5,604 KB, 0.54 of 0.35.0 (alloc-cache-cap-2026-11-03.md, lines 58 to 77).

That last gap, about 3 MB for a program this small, is what huge pages still cost putget, in exchange for fewer page faults and TLB misses in programs that allocate a lot. Whether a small program should wait longer before asking for them is the next question for the heap.

As always, the results files and decisions linked above have the full numbers, including the runs that did not go our way.