Cleaning the Heap - Helping the Go Compiler
Stacks of Pancakes and Heaps of Banana Bread
The Go language was created by one of the inventors of the C language. Despite this, unlike C and other low-level languages like C++ or Rust, Go is not a manually memory-managed language. For it's memory management it relies on a background garbage collector.
To run its garbage collector efficiently, every Go program distributes its data across two distinct regions that decides how long the data (or variable) will live in memory and who will clean it up. These regions are called the stack and the heap.
The stack holds local variables on calls that are already running, once per goroutine. When a function is called, Go reserves a block of memory for it called a frame, which is sized to hold exactly the function call's local variables.
I like to think of the stack as a freshly cooked stack of pancakes (because who doesn't love pancakes?). This pancake "stack" is temporal in nature and eaten one at a time from top to bottom. The stack holds local variables on calls that are already running, one per goroutine.
Every time the chef drops a fresh pancake on top of the stack, that specific pancake holds all the temporary variables needed for that function to work. The moment the function finishes its job, that top pancake is completely eaten and disappears with it. This ordering is what makes the stack so fast. There is no list of objects to scan nor memory to find and reclaim. So a returning function that uses the stack cleans up after itself. This is why the stack is the default region of memory in Go code.
It's a lot of willpower to not immediately garbage collect this with my face
If the stack is the pancake on my plate; hot, stacked, ready for me to eat from top to bottom and gone the moment I finish eating it, then the heap is the Tupperware of banana bread my wife made yesterday. It outlives the original meal it was made for and it sits in our fridge because one of us might want a slice of it later throughout the week.
The heap is a longer-lived pool of values that must outlive the call that made them. If I return a pointer to a local, because the frame is gone the instant the function returns and the pointer is still in use, then the value of the pointer goes on the heap instead. Unlike the stack, nothing is automatically freed up. Although Go's garbage collector is extremely well engineered, the nature of cleaning the heap still costs real CPU resources. This is the price the heap pays for outliving its original call.
In other languages (like C or C++) this would actually be a bug but in Go this works. The compiler reads the code, notices that &t outlives the function and quietly puts t on the heap. This stack vs heap call is known as Escape Analysis and the Go compiler runs it on every value that is in the code.
Helping the Go compiler
In most cases, when writing idiomatic Go, we shouldn't care about how this works. The Go compiler and Go Garbage-collector are so well engineered we can forget about how memory usage is managed and leave it to clean itself up. However, there are situations where needlessly returning pointers for your Go functions will affect performance. Adding memory to the heap that doesn't need to be there versus it being an ephemeral frame in the stack can be costly.
In my example below, I create a small program to check the network performance statistics of a VPN session.
The code itself is not very interesting, as both goodStats() and badStats() function calls return the same value of 1024 bytes sent.
What is interesting is when I compile the code with a flag that displays detailed memory escape analysis logs:
go build -gcflags '-m'
Because badStats() hands the caller a pointer to the VPNStats struct, but the stack frame dies the moment that function returns, the value cannot live there anymore, which is why it goes (escapes) to the heap. Notice that we don't actually get a reference to goodStats(), outside of argument does not escape. goodStats() returns the value itself, so a copy is made into the function caller's frame before the original dies cleanly. No escape to the heap and no needless use of the garbage collector.
Helping the Go compiler doesn't mean that we should never use pointers. In many cases we will genuinely need a value to outlive our function call, but it's useful to be cognisant of the memory costs associated with flinging tupperwared banana bread into the fridge, to be cleaned at some point in the future. In my VPN stats example, if badStats() allocates a fresh VPNStats on the heap every time it's called, and if its called once a second over an 8-hour session, that's 28,800 heap allocations for a struct that's just a uint64 and a float64 value.
This all adds up and not being aware of how much strain we're putting on Go's garbage collector can waste CPU resources that should be spent on the actual program itself. "Helping the compiler" isn't to do with micro-managing or fighting it and banning all pointer usage, it just means that we should know when something is being flung onto the heap. If you're running
go build -gcflags="-m" and you see escapes to heap on a hot path, you probably should be asking whether it actually needs to be on the heap.
As always, thanks for reading,
Mehmet