C++ Module 20 – Systems programming with C++

Custom allocators and std::pmr memory resources

Give C++ containers memory from a buffer you choose with std::pmr, build arenas for each request, and count and time allocations on two compilers.

Advanced
30 minutes
Examples run with Clang 21.0.0, GCC 16.2.0 and Python 3.14.8
By MySmartCoPilot

Free: How std::pmr fits together · The standard memory resources · A request handler that never touches the heap · Check yourself. With a pass: 3 in-depth parts and 3 interview answers, about 21 minutes.

What you will learn

Use polymorphic memory resources with the standard containers
Build per-request and per-frame arenas from a monotonic buffer
Measure allocation counts and times instead of assuming that an allocator is faster
Implement a std::pmr::memory_resource that reports its own misuse under AddressSanitizer

Before you start

What you’ll learn · Before you start

What you will learn

Use polymorphic memory resources with the standard containers
Build per-request and per-frame arenas from a monotonic buffer
Measure allocation counts and times instead of assuming that an allocator is faster
Implement a std::pmr::memory_resource that reports its own misuse under AddressSanitizer

Before you start

On this page

An allocator decides where a container’s memory comes from. By default every std::vector, std::string and std::map gets it from the global operator new, which uses the general-purpose heap: right for any size and any lifetime, but each call has a cost, and a program that builds many short-lived containers can spend much of its time there. Custom allocators replace that source with one that suits the program, such as a buffer that is thrown away in one piece when a request ends. The std::pmr memory resources of C++17 make this practical. A container from the std::pmr namespace takes a pointer to a std::pmr::memory_resource chosen at run time, and every allocation of the container, and of the strings inside it, goes to that resource.

How std::pmr fits together

Custom allocators existed before C++17, but each one was a template parameter, so a vector with another allocator was another type, and every function that took it had to be a template. The std::pmr memory resources move the choice to run time. std::pmr::vector<T> is std::vector<T, std::pmr::polymorphic_allocator<T>>, and the polymorphic_allocator holds only a pointer to a memory_resource, a class with three virtual functions: do_allocate, do_deallocate and do_is_equal. All containers that use one resource share one type, whatever the resource is.

Where a std::pmr container's memory comes from

Text description of the diagram

The diagram has two parts, from top to bottom.

1. A std::pmr::vector of std::pmr::string. Its polymorphic_allocator holds a pointer to a memory_resource, and every string element is built with the same resource, so the characters come from the same place. 2. An arrow labelled "allocate(bytes, alignment)" leads to that resource, a monotonic_buffer_resource. Inside it: a 4 KiB std::array on the stack, which the resource hands out from front to back, and below it, joined by an arrow labelled "when full", the upstream resource, null_memory_resource(). The upstream is asked only when the buffer is used up, and this one throws std::bad_alloc instead of allocating.

Nested containers pass the resource on. When a std::pmr::vector<std::pmr::string> builds an element, it hands its allocator to the string’s constructor, so the characters come from the same place as the vector’s array ([allocator.uses.construction]). Two costs come with the design. A std::pmr::vector<int> is a different type from a std::vector<int>, so it cannot be passed to a function that takes const std::vector<int>&; such functions should take a std::span instead. And every pmr container carries the extra pointer: on the build machine a std::pmr::string is 8 bytes larger than a std::string.

The standard memory resources

Resource	What it does	Gives memory back	Thread-safe
`new_delete_resource()`	Forwards every request to the global `operator new`; it is the default resource	At once, on each deallocate	Yes
`null_memory_resource()`	Throws `std::bad_alloc` for every request: an upstream that proves nothing reached the heap	Not applicable	Yes
`monotonic_buffer_resource`	Hands out a buffer from front to back, then asks its upstream for more	Only all at once, with `release()` or its destructor	No
`unsynchronized_pool_resource`	Keeps pools of blocks of a few sizes and reuses freed blocks	Freed blocks are reused; the pools on `release()`	No
`synchronized_pool_resource`	The same pools, safe to use from several threads without a lock of your own	As the unsynchronized pool	Yes

A monotonic buffer is the arena of this lesson: allocating is a pointer bump, deallocating does nothing, and everything goes at once. A pool reuses memory, so it suits containers that grow and shrink for a long time.

A request handler that never touches the heap

This program parses one request’s query string into key-value pairs twice: with std containers, and with std::pmr containers on a 4 KiB buffer on the stack. count_new.hpp replaces the global allocation functions with versions that count, so the program can report every heap allocation it makes:

count_new.hpp C++ header · count_new.hpp

// count_new.hpp: replaces the global allocation functions ([replacement.functions]) with versions that count, so a
// program can report how many times it used the heap and for how many bytes. The aligned forms are replaced too,
// because a library may call them (libstdc++'s new_delete_resource does). Include it in exactly one .cpp file.
#pragma once
#include <cstddef>
#include <cstdlib>
#include <new>

namespace heap {
inline std::size_t allocations = 0;
inline std::size_t bytes = 0;
struct Snapshot {
    std::size_t allocations, bytes;
};
inline Snapshot now() { return {allocations, bytes}; }
}  // namespace heap

void* operator new(std::size_t n) {
    ++heap::allocations;
    heap::bytes += n;
    if (void* p = std::malloc(n ? n : 1)) return p;
    throw std::bad_alloc();
}
void* operator new(std::size_t n, std::align_val_t al) {
    ++heap::allocations;
    heap::bytes += n;
    void* p = nullptr;
    const std::size_t a = static_cast<std::size_t>(al) < sizeof(void*) ? sizeof(void*) : static_cast<std::size_t>(al);
    if (posix_memalign(&p, a, n ? n : 1) == 0) return p;
    throw std::bad_alloc();
}
void operator delete(void* p) noexcept { std::free(p); }
void operator delete(void* p, std::size_t) noexcept { std::free(p); }
void operator delete(void* p, std::align_val_t) noexcept { std::free(p); }
void operator delete(void* p, std::size_t, std::align_val_t) noexcept { std::free(p); }

handler.cpp C++ · handler.cpp

// Parsing one request's query string into key-value pairs, twice: with std containers, which take every byte from the
// heap, and with std::pmr containers that take it from a 4 KiB buffer on the stack.
#include <array>
#include <cstddef>
#include <cstdio>
#include <memory_resource>
#include <string>
#include <string_view>
#include <utility>
#include <vector>

#include "count_new.hpp"

constexpr std::string_view request = "user=asha&city=lisbon&lang=pt&page=3&sort=price&q=noise+cancelling+headphones+for+long+flights";

// Calls fn(key, value) for each "key=value" field of a query string.
template <class F>
void each_field(std::string_view s, F fn) {
    while (!s.empty()) {
        const std::size_t amp = s.find('&');
        const std::string_view field = s.substr(0, amp);
        const std::size_t eq = field.find('=');
        fn(field.substr(0, eq), eq == std::string_view::npos ? std::string_view{} : field.substr(eq + 1));
        s = amp == std::string_view::npos ? std::string_view{} : s.substr(amp + 1);
    }
}

std::size_t with_std(std::string_view query) {
    std::vector<std::pair<std::string, std::string>> fields;
    each_field(query, [&](std::string_view k, std::string_view v) { fields.emplace_back(k, v); });
    return fields.size();
}

std::size_t with_pmr(std::string_view query) {
    std::array<std::byte, 4096> buffer;  // on the stack: gone when the function returns
    std::pmr::monotonic_buffer_resource arena(buffer.data(), buffer.size(), std::pmr::null_memory_resource());
    std::pmr::vector<std::pair<std::pmr::string, std::pmr::string>> fields(&arena);
    each_field(query, [&](std::string_view k, std::string_view v) { fields.emplace_back(k, v); });
    return fields.size();  // the vector, then the arena, are destroyed: nothing to free one by one
}

int main() {
    const heap::Snapshot a = heap::now();
    const std::size_t n1 = with_std(request);
    const heap::Snapshot b = heap::now();
    const std::size_t n2 = with_pmr(request);
    const heap::Snapshot c = heap::now();
    std::printf("sizeof(std::string) %zu, sizeof(std::pmr::string) %zu\n", sizeof(std::string), sizeof(std::pmr::string));
    std::printf("std containers:            %zu fields, %zu heap allocations, %zu bytes\n", n1, b.allocations - a.allocations, b.bytes - a.bytes);
    std::printf("std::pmr, 4 KiB on stack:  %zu fields, %zu heap allocations, %zu bytes\n", n2, c.allocations - b.allocations, c.bytes - b.bytes);
}

One request, std containers against an arena, two standard libraries Python · handler_session.py

from session import run

# Apple clang's libc++ and GCC's libstdc++ make std::string objects of different sizes, so the same request takes a
# different number of bytes from the heap with each library.
run("""
clang++ -std=c++23 -O2 handler.cpp -o handler && ./handler
g++-16 -std=c++23 -O2 handler.cpp -o handler && ./handler
""")

Output

$ clang++ -std=c++23 -O2 handler.cpp -o handler && ./handler
sizeof(std::string) 24, sizeof(std::pmr::string) 32
std containers:            6 fields, 5 heap allocations, 768 bytes
std::pmr, 4 KiB on stack:  6 fields, 0 heap allocations, 0 bytes
$ g++-16 -std=c++23 -O2 handler.cpp -o handler && ./handler
sizeof(std::string) 32, sizeof(std::pmr::string) 40
std containers:            6 fields, 5 heap allocations, 1005 bytes
std::pmr, 4 KiB on stack:  6 fields, 0 heap allocations, 0 bytes

Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 handler_session.py

The std version made 5 heap allocations: four as the vector grew to 1, 2, 4 and 8 pairs, and one for the 44-character search phrase, too long to fit inside a string object. The bytes differ between the libraries, 768 against 1005, because a std::string is 24 bytes in libc++ and 32 in libstdc++. The std::pmr version made none, and this is proved rather than hoped: its upstream is null_memory_resource(), so a single byte past the buffer would have thrown. When with_pmr returns, the vector and the arena are destroyed and the stack frame is gone; nothing is freed piece by piece. An arena per request, per frame of a game or per parsed file follows the same pattern: choose a buffer that covers the usual case, and let the upstream catch the rest.

Data Storage Converter Convert bytes to KiB and MiB when you size an arena for a request or a frame.

Check yourself

Heap, arena and pool: 100,000 small vectors timed, and an arena that only grows Pro: opens with a Pro, Premium or Ultimate pass

Whether an arena or a pool is faster than the heap depends on what the program does with the memory. This part times five ways to build small vectors on two standard libraries, reproduces the arena that grows without limit, and sets out which resource fits which lifetime.

In this part:

100,000 small vectors five ways, timed on two compilers
An arena that only grows: the failure reproduced

6 runnable examples (C++, C++ header, Python) · about 3 minutes

Opening this part with your pass…

This part opens with any MySmartCoPilot pass.

Pro pass · 1 week₹79$2.99
30 days₹179$5.99
1 year₹1,699$45

One payment; a pass never renews. The price shown is the total you pay: no fees or charges are added at checkout.

Choose a pass I have a licence key

It also opens every other in-depth part in Learn and every Pro tool. Sales are final, except where the law gives you a right to withdraw, for example in the EU and the UK (refund policy).

Lifetime traps reproduced: copies that leave the arena, a buffer that dies first Pro: opens with a Pro, Premium or Ultimate pass

A memory resource must outlive every container that uses it, and the rules for copies and moves decide where new elements live. This part reproduces the three mistakes that most often turn an arena into a bug.

In this part:

A copy that moves to the heap without a word
A container that outlives its buffer, caught by AddressSanitizer
Building objects in storage you own

6 runnable examples (C++, Python) · about 3 minutes

Opening this part with your pass…

This part opens with any MySmartCoPilot pass.

Pro pass · 1 week₹79$2.99
30 days₹179$5.99
1 year₹1,699$45

One payment; a pass never renews. The price shown is the total you pay: no fees or charges are added at checkout.

Choose a pass I have a licence key

It also opens every other in-depth part in Learn and every Pro tool. Sales are final, except where the law gives you a right to withdraw, for example in the EU and the UK (refund policy).

Your turn: an arena memory_resource checked by AddressSanitizer Pro: opens with a Pro, Premium or Ultimate pass

The best way to understand memory_resource is to write one. This part gives you an arena to finish, public tests for alignment, overlap, capacity, reset and containers, a test that AddressSanitizer catches misuse, and one solution.

In this part:

The interface you implement
The public tests, and how the starting arena fares
One solution, passing on both compilers
Taking the arena further

4 runnable examples (C++, C++ header, Python) · about 3 minutes

Opening this part with your pass…

This part opens with any MySmartCoPilot pass.

Pro pass · 1 week₹79$2.99
30 days₹179$5.99
1 year₹1,699$45

One payment; a pass never renews. The price shown is the total you pay: no fees or charges are added at checkout.

Choose a pass I have a licence key

It also opens every other in-depth part in Learn and every Pro tool. Sales are final, except where the law gives you a right to withdraw, for example in the EU and the UK (refund policy).

Interview questions

Mid to senior · Warm-up

What is the polymorphic_allocator of std::pmr, and how does it differ from passing a custom allocator as a template argument?

It is the allocator of the std::pmr containers. Instead of encoding a memory strategy in its type, it holds a pointer to a std::pmr::memory_resource, an abstract class with the virtual functions do_allocate, do_deallocate and do_is_equal, and forwards every request to it. So std::pmr::vector<int> is one type whatever resource it uses, a monotonic buffer, a pool or the heap, chosen at run time. With a classic allocator template argument, every strategy makes a different container type, and every function that takes the container must be a template or name that allocator. The costs of the pmr design are an indirect call per allocation, one more pointer in every container, and the rule that the resource must outlive every container that uses it. Nested pmr containers pass their resource on to the elements they build.
Mid to senior

A pull request moves a hot path's containers to std::pmr with a monotonic buffer and claims it is faster. What evidence do you ask for, which results would not surprise you, and when would you reject the change?

Model answer, Pro. Model answers open with any pass; prices are in the notice on this page. First step and follow-ups
Mid to senior

After request handling moved to one long-lived monotonic buffer resource from std::pmr, a server's memory grows steadily for days until it is restarted, yet leak checkers report nothing. Explain the cause and the fixes.

Model answer, Pro. First step and follow-ups
Senior to staff

Code builds a std::pmr::vector on a per-request arena and copies it into a result object. Heap profiles show the copies on the heap, and one code path crashes after the request ends. Explain both and fix the design.

Model answer, Pro. First step and follow-ups

Key takeaways

A std::pmr container takes its memory from a memory_resource chosen at run time, and passes it to its elements.
A monotonic buffer allocates by bumping a pointer and frees everything at once; release it at the end of each unit of work.
Measure before you switch: reserve alone can beat a custom allocator.

References

Working Draft, Programming Languages: C++ (N4950): [mem.res] memory resources, [allocator.uses.construction], [specialized.construct], [replacement.functions] (ISO/IEC JTC1/SC22/WG21)
P0220R1: Adopt Library Fundamentals V1 TS Components for C++17 (memory resources) (ISO/IEC JTC1/SC22/WG21)
AddressSanitizer (Clang documentation): stack-use-after-return and manual poisoning (LLVM Project)
compiler-rt: include/sanitizer/asan_interface.h (ASAN_POISON_MEMORY_REGION) (LLVM Project)
Program Instrumentation Options (GCC): -fsanitize=address (GNU Project)
The GNU C++ Library manual: Memory (allocators) (GNU Project)

Report a problem with this lesson

In this module: Systems programming with C++

Calling C from C++ (and C++ from C) Free lesson, Pro answers: 25 minutes
POSIX in C++: files, pipes and child processes Free core, Pro depth: 9 of 25 minutes free
TCP sockets in C++: client and server on localhost Free core, Pro depth: 11 of 30 minutes free
Event loops in C++: poll, kqueue and epoll Free core, Pro depth: 11 of 30 minutes free
Memory-mapped files in C++: mmap and binary formats Free core, Pro depth: 11 of 30 minutes free
Custom allocators and std::pmr memory resources Free core, Pro depth: 9 of 30 minutes free

Your country

Custom allocators and std::pmr memory resources

What you will learn

Before you start

What you will learn

Before you start

How std::pmr fits together

The standard memory resources

A request handler that never touches the heap

Check yourself

Check yourself

Heap, arena and pool: 100,000 small vectors timed, and an arena that only grows Pro: opens with a Pro, Premium or Ultimate pass

Lifetime traps reproduced: copies that leave the arena, a buffer that dies first Pro: opens with a Pro, Premium or Ultimate pass

Your turn: an arena memory_resource checked by AddressSanitizer Pro: opens with a Pro, Premium or Ultimate pass

Interview questions

What is the polymorphic_allocator of std::pmr, and how does it differ from passing a custom allocator as a template argument?

A pull request moves a hot path's containers to std::pmr with a monotonic buffer and claims it is faster. What evidence do you ask for, which results would not surprise you, and when would you reject the change?

After request handling moved to one long-lived monotonic buffer resource from std::pmr, a server's memory grows steadily for days until it is restarted, yet leak checkers report nothing. Explain the cause and the fixes.

Code builds a std::pmr::vector on a per-request arena and copies it into a result object. Heap profiles show the copies on the heap, and one code path crashes after the request ends. Explain both and fix the design.

Key takeaways

References

Related tools