C++ Module 15 – Build systems, testing and tooling
Sanitizer and hardened C++ build presets
Build C++ in four CMake presets (ASan with UBSan, TSan, hardened and release), see which checks catch which bugs, and what each check costs.
What you will learn
- Choose the build that finds each family of bugs, and state what each build costs
- Distinguish the sanitizers that can share one build from those that need a build of their own
- Read a bug matrix and tell sanitizer reports from library precondition checks
- Write sanitizer and hardened presets in CMakePresets.json and run them as workflows
- Pick a hardening mode for production from what it catches and what it costs
Before you start
What you’ll learn · Before you start
What you will learn
- Choose the build that finds each family of bugs, and state what each build costs
- Distinguish the sanitizers that can share one build from those that need a build of their own
- Read a bug matrix and tell sanitizer reports from library precondition checks
- Write sanitizer and hardened presets in CMakePresets.json and run them as workflows
- Pick a hardening mode for production from what it catches and what it costs
Before you start
On this page
zoo.cpp holds five bugs, and its release build ran all five to the end, printing “finished” every time. A
sanitizer: A compiler option, such as -fsanitize=address or -fsanitize=undefined, that builds extra checks into a program, so that memory errors and undefined behaviour are reported, with their source line, while it runs. More in “Finding memory bugs with AddressSanitizer and UBSan” build compiles the program with run-time checks for one family of bugs, and a
hardened build keeps only the checks that are cheap enough to ship. Sanitizer and hardened C++ build presets give
each of those builds a name in CMakePresets.json, so that everyone and the continuous-integration server build them
the same way. The usual set has four: asan-ubsan for memory errors and undefined behaviour: What a program does when it breaks a rule for which the C++ standard sets no requirements, such as reading past the end of an array. The compiler may assume it never happens, so the result can change with the compiler and the optimisation level. More in “Object lifetime and undefined behaviour in C++”, tsan for data
races, hardened for the standard library’s precondition checks, and release for timing and shipping. ASan
and UBSan can share one build; ThreadSanitizer needs a build of its own, because it cannot be combined with
AddressSanitizer.
Each build finds one family of bugs, at its own cost
| Preset | Flags with Clang or GCC | Finds | Cost |
|---|---|---|---|
| asan-ubsan | -O1 -g -fsanitize=address,undefined -fno-sanitize-recover=all -fno-omit-frame-pointer |
Memory errors, signed overflow, bad shifts and indexes | About 2× the time, says Clang’s ASan documentation |
| tsan | -O1 -g -fsanitize=thread |
Data races between threads | 5× to 15× the time and 5× to 10× the memory, for TSan |
| hardened | libc++: -D_LIBCPP_HARDENING_MODE=_LIBCPP_HARDENING_MODE_FAST; GCC on Linux: -fhardened |
Misuse of the standard library, such as an index past the end | Small, by design |
| release | -O2 |
Nothing | None |
-fno-sanitize-recover=all matters in a test preset: without it most UBSan checks
recovery (sanitizers): What most UndefinedBehaviorSanitizer checks do by default after a report: they print it and let the program go on, so a test can pass with errors in its output. -fno-sanitize-recover=all makes the first report end the program., so a test can pass with errors in its output. The two
sanitizer presets are for tests and CI only. The hardened preset is the one meant for production: libc++ recommends
its fast mode for most projects, and -fhardened is GCC’s bundle of security options for release builds. A common
split runs asan-ubsan and tsan on every change, the hardened preset in the release pipeline and in production, and the
plain release preset only for benchmarks.
Checkpoint
Check: Four builds and what each one finds
AddressSanitizer and ThreadSanitizer never share a build
- AddressSanitizer and UndefinedBehaviorSanitizer work together, and on Linux ASan also brings LeakSanitizer.
- ThreadSanitizer cannot be combined with AddressSanitizer or LeakSanitizer, so it gets its own build.
- MemorySanitizer, Clang’s check for reads of uninitialised memory on Linux, needs a build of its own too.
- hardened standard library: A standard library that checks the preconditions of calls such as operator[] and front() while the program runs, and stops it when one fails, as libc++'s hardening modes and libstdc++ with _GLIBCXX_ASSERTIONS do. C++26 makes such hardened implementations part of the standard. More in “std::vector: the container to reach for first” is ordinary code in the library headers, so it combines with any of them.
The diagram shows the three sanitizer builds this leaves, and the one kind of check that fits into all of them:
Which checks can share one build
Text description of the diagram
At the top, a box for standard library hardening, which is code in the library headers. An arrow labelled "fits into each" leads from it to a group of three sanitizer builds that are never merged.
- The asan-ubsan build holds AddressSanitizer and UndefinedBehaviorSanitizer together, and on Linux AddressSanitizer brings LeakSanitizer too.
- The tsan build holds ThreadSanitizer alone: it cannot be combined with AddressSanitizer or LeakSanitizer.
- The msan build holds MemorySanitizer alone, Clang's check for reads of uninitialised memory on Linux.
Hardening can be added to any of the three builds, and to a release build, because it is ordinary code in the headers of the standard library.
Checkpoint
Check: Sanitizers that can share a build
Five bugs in four builds, and no build catches them all
zoo.cpp holds five bugs, and the command line chooses one:
// Five bugs, chosen by name on the command line; each check catches only some of them.
#include <climits>
#include <optional>
#include <print>
#include <string>
#include <string_view>
#include <thread>
volatile int sink; // reads go here, so the optimiser keeps them
void view_past_end() {
std::string_view code = "IN-KA";
sink = code[5]; // one past the last character
}
void empty_optional() {
std::optional<int> port; // nothing was stored
sink = *port;
}
void array_past_end(int i) {
int scores[4] = {70, 82, 91, 64};
sink = scores[i]; // called with i == 4
}
void signed_overflow(int n) {
sink = n + 1; // called with n == INT_MAX
}
void data_race() {
int hits = 0;
auto count = [&hits] { for (int k = 0; k < 100000; ++k) ++hits; };
std::thread a(count), b(count); // both threads write hits
a.join();
b.join();
sink = hits;
}
int main(int argc, char** argv) {
const std::string bug = argc > 1 ? argv[1] : "";
if (bug == "view") view_past_end();
else if (bug == "optional") empty_optional();
else if (bug == "array") array_past_end(argc + 2);
else if (bug == "overflow") signed_overflow(INT_MAX - 2 + argc);
else if (bug == "race") data_race();
else return 2;
std::println("{}: finished", bug);
} Here is each bug in each build, compiled with the flags of the presets above by Apple clang on a Mac:
"""Builds zoo.cpp with the flags of each preset of this lesson, runs each of its five bugs in each build, and prints
one line per run: the first line of the report that stopped it, or that it finished with nothing reported."""
import signal
import subprocess
from session import run, tool_env
PRESETS = {
"release": "-O2",
"asan-ubsan": "-O1 -g -fsanitize=address,undefined -fno-sanitize-recover=all -fno-omit-frame-pointer",
"tsan": "-O1 -g -fsanitize=thread",
"hardened": "-O2 -D_LIBCPP_HARDENING_MODE=_LIBCPP_HARDENING_MODE_FAST",
}
BUGS = ["view", "optional", "array", "overflow", "race"]
MARKERS = ["runtime error:", "ERROR: AddressSanitizer:", "WARNING: ThreadSanitizer:"]
def outcome(result: subprocess.CompletedProcess) -> str:
"""What stopped a run, in a few words."""
if result.returncode == 0:
return "finished, nothing reported"
for line in result.stdout.splitlines():
for marker in MARKERS:
if marker in line:
return line[line.index(marker):].split(" (pid=")[0]
if result.returncode < 0:
return f"stopped by {signal.Signals(-result.returncode).name}, no message"
return f"exit status {result.returncode}"
run("\n".join(f"clang++ -std=c++23 {flags} zoo.cpp -o zoo-{name}" for name, flags in PRESETS.items()))
print()
for name in PRESETS:
for bug in BUGS:
result = subprocess.run([f"./zoo-{name}", bug], env=tool_env(), stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)
print(f"{name:<11} {bug:<9} {outcome(result)}") Output
$ clang++ -std=c++23 -O2 zoo.cpp -o zoo-release $ clang++ -std=c++23 -O1 -g -fsanitize=address,undefined -fno-sanitize-recover=all -fno-omit-frame-pointer zoo.cpp -o zoo-asan-ubsan $ clang++ -std=c++23 -O1 -g -fsanitize=thread zoo.cpp -o zoo-tsan $ clang++ -std=c++23 -O2 -D_LIBCPP_HARDENING_MODE=_LIBCPP_HARDENING_MODE_FAST zoo.cpp -o zoo-hardened release view finished, nothing reported release optional finished, nothing reported release array finished, nothing reported release overflow finished, nothing reported release race finished, nothing reported asan-ubsan view finished, nothing reported asan-ubsan optional finished, nothing reported asan-ubsan array runtime error: index 4 out of bounds for type 'int[4]' asan-ubsan overflow runtime error: signed integer overflow: 2147483647 + 1 cannot be represented in type 'int' asan-ubsan race finished, nothing reported tsan view finished, nothing reported tsan optional finished, nothing reported tsan array finished, nothing reported tsan overflow finished, nothing reported tsan race WARNING: ThreadSanitizer: data race hardened view stopped by SIGTRAP, no message hardened optional stopped by SIGTRAP, no message hardened array finished, nothing reported hardened overflow finished, nothing reported hardened race finished, nothing reported
Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 matrix_session.py
No build caught everything. The release build caught nothing: all five runs printed “finished”. asan-ubsan stopped
the two bugs that are undefined behaviour of the language, the array index and the signed overflow; UBSan reported
first because the array’s bound is known at compile time. tsan found only the
Two threads reach the same memory at the same time, at least one of them writing, with nothing that orders the two accesses. In C++ that is undefined behaviour, and ThreadSanitizer reports it..
hardened stopped the string_view read and the empty optional, which both sanitizers missed: the memory they
read was valid, so nothing was wrong at the level of bytes, only at the level of the library’s preconditions. That is
why a project needs both kinds of build, and why the matrix runs every test in each of them.
Here are the twenty runs again, as a table that rules out builds by the job they do:
Answer the questions above: the table then marks what fits your case and says why the rest does not.
| Build | string_view read past its end | Empty optional read | C array index past its end | Signed integer overflow | Data race between threads |
|---|---|---|---|---|---|
| release Checks: nothing: -O2 only | Poor finished, nothing reported | Poor finished, nothing reported | Poor finished, nothing reported | Poor finished, nothing reported | Poor finished, nothing reported |
| asan-ubsan Checks: memory errors and undefined behaviour | Poor finished: the byte it read was valid memory | Poor finished: an empty optional still holds its storage | Good runtime error: index 4 out of bounds for type 'int[4]' | Good runtime error: signed integer overflow: 2147483647 + 1 | Poor finished, nothing reported |
| tsan Checks: data races between threads | Poor finished, nothing reported | Poor finished, nothing reported | Poor finished, nothing reported | Poor finished, nothing reported | Good WARNING: ThreadSanitizer: data race |
| hardened Checks: the standard library's preconditions | Good stopped by SIGTRAP, no message | Good stopped by SIGTRAP, no message | Poor finished: a C array has no library check | Poor finished: arithmetic is not a library call | Poor finished, nothing reported |
How the answers decide
- “What is the build for?” Shipping to production: rules out asan-ubsan and tsan. Sanitizer builds are for tests and CI. Clang's documentation puts ASan at about 2× the time and TSan at 5× to 15× the time and 5× to 10× the memory.
- “What is the build for?” Shipping to production: adds a caveat to release. A release build ships, but it stopped none of the five bugs. The hardened build is the one meant for production.
- “What is the build for?” Running the tests on every change: adds a caveat to release. The release build stopped nothing in the zoo. Keep it for timing and benchmarks, and run the tests in the other builds.
- “Does the code under test start threads?” Yes, it uses threads: adds a caveat to asan-ubsan and hardened. Only tsan stopped the race. Code that starts threads runs its tests in a tsan build as well.
- “Does the code under test start threads?” No, one thread only: adds a caveat to tsan. With one thread there is no race for tsan to find; asan-ubsan and hardened together stopped the other four bugs.
Key idea
Sanitizers find bugs in bytes and in the rules of the language; hardening finds misuse of the library.
Checkpoint
Check: Bytes against library preconditions
A presets file names each build once, for every machine
- Configure presets set the generator, the build folder and cache variables such as
CMAKE_CXX_FLAGS. - A hidden preset holds shared settings; other presets copy them with
inherits. - Build and test presets name a configure preset; test presets add options and environment variables.
- A workflow preset (CMake): A preset in CMakePresets.json that runs other presets in order, such as configure, build and test, so that one command, cmake --workflow --preset name, does all of them. runs configure, build and test in one command:
cmake --workflow --preset asan-ubsan. - A condition limits a preset to one system, for example
-fhardenedpresets to Linux.
Step through what that one command does with the presets it names:
The drawing shows the whole scene; its 5 steps are listed below.
Every step as text
- One command.
cmake --workflow --preset asan-ubsanreadsCMakePresets.jsonand finds the workflow preset named asan-ubsan. It lists three steps, configure, build and test, each by the name of another preset. - Configure. The configure preset copies the generator and the build folder from a hidden preset with
inherits, and setsCMAKE_CXX_FLAGSto the asan-ubsan flags of the first table,-fno-sanitize-recover=allincluded. - Build. The build preset names the configure preset it builds, so the compiler gets exactly those flags. Nobody types them again, on a laptop or on the CI server.
- Test. The test preset runs the tests of that build and adds its own options and environment variables. With
-fno-sanitize-recover=allin the flags, a UBSan report ends the program and fails its test. - Only where it works. A condition limits a preset to one system. A preset with GCC's
-fhardenedcan be limited to Linux, so that the same file also works on a Mac.
Because the flags live in the file, the same command gives a laptop and a CI server the same build. With hardened C++ build presets beside the sanitizer ones, a release can keep cheap checks while the tests run every expensive one.
Checkpoint
Check: What a presets file holds
Check yourself
Eleven questions on the four builds, the checks that share one, the bug zoo and the presets file, weakest idea first, with a reason for every choice.
Practice that adapts to you
Practise this lesson
11 questions about 7 ideas of this lesson. Each answer is explained at once, and what you get right is kept in this browser.
A CMakePresets.json for the four builds, run end to end Pro: opens with a Pro, Premium or Ultimate pass
Writing the four builds as presets turns a list of flags into one command per build. This part walks through a complete presets file and runs every workflow on Linux, with the test results each one produced.
In this part:
- The presets file, part by part
- Every workflow on Linux, with its test results
- Wiring the presets into CI
2 runnable examples (CMakeLists.txt, CMakePresets.json, Text) · about 4 minutes
Opening this part with your pass…
This part opens with any MySmartCoPilot pass.
- Pro pass · 1 week₹79$2.99
- 30 days₹179$5.99
- 1 year₹1,699$45
One payment; a pass never renews. The price shown is the total you pay: no fees or charges are added at checkout.
Passes can’t be bought right now, so this part can’t be opened at the moment. The free part above stands on its own. If you have a pass, enter your licence key.
It also opens every other in-depth part in Learn and every Pro tool. Sales are final, except where the law gives you a right to withdraw, for example in the EU and the UK (refund policy).
Hardening for production: what each mode catches and costs Pro: opens with a Pro, Premium or Ultimate pass
Production hardening is a trade: every check costs a little time and turns a silent bug into a stopped program. This part measures both sides on one program, compares libc++’s modes with GCC’s options, and shows where C++26 takes it.
In this part:
- What fast, extensive and debug mode each catch, reproduced
- The cost of each check, measured
- GCC’s -fhardened, MSVC and the C++26 hardened implementation
6 runnable examples (C++, Python, Text) · about 6 minutes
Opening this part with your pass…Pro part. It opens with the same pass as the part above.
Interview questions
-
Fresher to mid · Warm-up
Why do AddressSanitizer and ThreadSanitizer need separate builds?
Both instrument every memory access and bring their own run-time library, which replaces parts of the normal run-time, such as the memory allocator, with versions that do their bookkeeping. The two designs cannot share one process, so the compilers refuse the combination outright: GCC says that -fsanitize=thread is incompatible with -fsanitize=address, and Clang rejects it too. A project therefore keeps two sanitizer builds: one with ASan and UBSan, which do combine, for memory errors and undefined behaviour, and one with TSan for data races, run on the tests that use threads. MemorySanitizer, Clang's check for uninitialised reads, also needs its own build.
-
Mid to senior
Your team wants to ship a C++ service with standard library hardening, but the latency budget allows at most 3 % more CPU. A benchmark shows libc++'s fast mode costing 12 % in one hot loop and nothing measurable elsewhere. What do you ship, how do you measure it, and what do you change?
-
Mid to senior
After an out-of-bounds crash in production, a teammate proposes shipping the AddressSanitizer build of the service from now on, "so that the next bug is caught". Evaluate the idea and propose what you would ship instead.
-
Fresher to mid
After the team moves to GCC 15, a debug build that worked for years aborts in a test with "Assertion '__n < this->size()' failed" inside std::vector. The release build still passes. Is the new compiler wrong, and what do you do?
Key takeaways
- Give each build a preset: asan-ubsan and tsan for tests, hardened for production, release for timing.
- ASan and UBSan share a build; TSan and MSan need builds of their own.
- Sanitizers find bugs in bytes; hardening finds misuse of the library; a release build finds neither.
- Add
-fno-sanitize-recover=allto test builds, so that every report fails the run.
Terms in this lesson
- recovery (sanitizers)
- What most UndefinedBehaviorSanitizer checks do by default after a report: they print it and let the program go on, so a test can pass with errors in its output. -fno-sanitize-recover=all makes the first report end the program.
- workflow preset (CMake)
- A preset in CMakePresets.json that runs other presets in order, such as configure, build and test, so that one command, cmake --workflow --preset name, does all of them.
References
- Hardening Modes (libc++ documentation) (LLVM Project)
- Macros (libstdc++ manual) (GNU Project)
- Program Instrumentation Options (GCC) (GNU Project)
- P3471R4: Standard library hardening (ISO/IEC JTC1/SC22/WG21)
- cmake-presets(7) (Kitware)
- AddressSanitizer (Clang documentation) (LLVM Project)
- ThreadSanitizer (Clang documentation) (LLVM Project)
- UndefinedBehaviorSanitizer (Clang documentation) (LLVM Project)
- MemorySanitizer (Clang documentation) (LLVM Project)
- GCC 15 Release Series, Changes, New Features, and Fixes (GNU Project)
- Working Draft, Programming Languages: C++ ([intro.compliance.general]) (ISO C++ committee working draft)
- STL Hardening (Microsoft STL wiki) (Microsoft)
Related tools
Report a problem with this lesson
Kept only in this browser. Your Learn progress