Hardening in compiler output: canaries, PIE, NX

The same source, built twice, behaved two different ways: one build copied 36 bytes into a 16-byte buffer and then died on the way out of the damaged function, the other stopped itself cleanly with status 134. Nothing in the C changed. One compiler option did.

Hardened builds are the reason a disassembly rarely matches the source instruction for instruction: the compiler has added a guard word, swapped a library call for a checked one, and asked the linker for an image the loader may place anywhere. This page reads those additions in generated code on three instruction sets, runs them to see what each one catches, and ends with the fields you can check on a binary you built yourself.

A guard word catches an overflow at the return

The copy below has the oldest bug in C: it copies until the terminating zero byte and never compares the length with the size of the buffer.

/* A name is copied into a 16-byte buffer by hand, with no length check: the bug this page is about. */
#include <stdio.h>

__attribute__((noinline)) static void copy_loop(const char *src) {
    char name[16];
    int i = 0;
    while (src[i] != '\0') {      /* nothing compares i with sizeof name */
        name[i] = src[i];
        i++;
    }
    name[i] = '\0';
    printf("copied %d bytes: %s\n", i, name);
}

int main(void) {
    setvbuf(stdout, NULL, _IONBF, 0);   /* print each line as it happens, so the output survives an abort */
    copy_loop("Aarav");
    copy_loop("a name far longer than sixteen bytes");
    printf("returned normally\n");
    return 0;
}

Build it twice, with and without the stack protector, and run both:

import pathlib
import sys

sys.path.insert(0, str(pathlib.Path(__file__).parent / "sessions"))
import session

session.run("""
# Build the unchecked copy twice: once with the stack protector the compiler offers, once without it
clang -O1 -fstack-protector-strong -o copy_guarded copy_loop.c
clang -O1 -fno-stack-protector -o copy_bare copy_loop.c
# Without the protector, 36 bytes land in a 16-byte buffer and nothing checks them before the return
{ ./copy_bare; } 2>/dev/null; echo "exit=$?"
# With it, the same overflow is caught where the damaged function returns
{ ./copy_guarded; } 2>/dev/null; echo "exit=$?"
""")

session.run("""
# The guard word and its comparison are the only difference in the generated code
clang -O1 -fstack-protector-strong -S -o guarded.s copy_loop.c
clang -O1 -fno-stack-protector -S -o bare.s copy_loop.c
grep -c 'stack_chk' guarded.s bare.s
grep -n -A1 'stack_chk_guard@GOTPAGE' guarded.s | head -8
grep -n -B2 -A1 'stack_chk_fail' guarded.s | tail -8
""")

# Build the unchecked copy twice: once with the stack protector the compiler offers, once without it
$ clang -O1 -fstack-protector-strong -o copy_guarded copy_loop.c
$ clang -O1 -fno-stack-protector -o copy_bare copy_loop.c
# Without the protector, 36 bytes land in a 16-byte buffer and nothing checks them before the return
$ { ./copy_bare; } 2>/dev/null; echo "exit=$?"
copied 5 bytes: Aarav
copied 36 bytes: a name far longer than sixteen bytes
exit=138
# With it, the same overflow is caught where the damaged function returns
$ { ./copy_guarded; } 2>/dev/null; echo "exit=$?"
copied 5 bytes: Aarav
copied 36 bytes: a name far longer than sixteen bytes
exit=134
# The guard word and its comparison are the only difference in the generated code
$ clang -O1 -fstack-protector-strong -S -o guarded.s copy_loop.c
$ clang -O1 -fno-stack-protector -S -o bare.s copy_loop.c
$ grep -c 'stack_chk' guarded.s bare.s
guarded.s:5
bare.s:0
$ grep -n -A1 'stack_chk_guard@GOTPAGE' guarded.s | head -8
58:	adrp	x8, ___stack_chk_guard@GOTPAGE
59-Lloh10:
60:	ldr	x8, [x8, ___stack_chk_guard@GOTPAGEOFF]
61-Lloh11:
--
91:	adrp	x9, ___stack_chk_guard@GOTPAGE
92-Lloh15:
93:	ldr	x9, [x9, ___stack_chk_guard@GOTPAGEOFF]
$ grep -n -B2 -A1 'stack_chk_fail' guarded.s | tail -8
101-	ret
102-LBB1_7:
103:	bl	___stack_chk_fail
104-	.loh AdrpLdrGotLdr	Lloh9, Lloh10, Lloh11

Both builds copied 36 bytes into name[16] and printed them, so both overflowed. What differs is the ending. The build with -fstack-protector-strong ended with status 134, which a shell reports as 128 + 6, a process ended by signal 6: the guarded function noticed the damage and called the failing routine. The build with -fno-stack-protector reached its own return instruction with a return address made of the overflowing name, jumped there and ended with status 138. That second ending is undefined behaviour, not a defence; it happened to fault here, and on another day, with other bytes, it could land somewhere that runs.

The guarded listing has five stack_chk lines: an adrp and ldr pair on entry, the same pair before the return, and the call to the failing routine. Step through the frame they protect:

The stack canarystack canary: A value that a protected function copies into its stack frame on entry and compares with the original before it returns. If an overflow of a local buffer changed the copy, the program stops instead of returning through the damaged frame. More in “Reading compiler output: -O0 versus -O2” is the copy in the frame; the program-wide value it is compared with is named __stack_chk_guard in C, and a Mach-O listing puts one more underscore in front of every C name, which is why it reads ___stack_chk_guard. On AArch64 the listing reaches it through the table of addresses the loader fills in, with adrp and two ldr instructions; cmp and b.ne then route a mismatch to ___stack_chk_fail instead of the return. GCC’s documentation sets out which functions get this treatment: the plain form protects functions that call alloca and functions with buffers of eight bytes or more, and the strong form adds any function with a local array or with a reference to a local frame address. A function of pure arithmetic gets nothing, which is why counting the stack_chk lines is a quick way to see what a build chose.

Where the value lives differs by target, and the difference shows in one instruction:

import pathlib
import sys

sys.path.insert(0, str(pathlib.Path(__file__).parent / "sessions"))
import session

session.run("""
# Where the guard word comes from differs by target. x86-64 reads it from thread-local storage at %fs:40
clang --target=x86_64-linux-gnu -O2 -fstack-protector-strong -S -o - copy_name_portable.c | grep -E 'fs:40|stack_chk|cmpq|jne'
""")

session.run("""
# AArch64 and RISC-V read a global, __stack_chk_guard, through the table of addresses the loader fills in
clang --target=aarch64-linux-gnu -O2 -fstack-protector-strong -S -o - copy_name_portable.c | grep -E 'stack_chk|cmp|b.ne'
clang-23 --target=riscv64-linux-gnu -march=rv64gc -O2 -fstack-protector-strong -S -o - copy_name_portable.c | grep -E 'stack_chk|bne|call'
""")

# Where the guard word comes from differs by target. x86-64 reads it from thread-local storage at %fs:40
$ clang --target=x86_64-linux-gnu -O2 -fstack-protector-strong -S -o - copy_name_portable.c | grep -E 'fs:40|stack_chk|cmpq|jne'
	movq	%fs:40, %rax
	movq	%fs:40, %rax
	cmpq	24(%rsp), %rax
	jne	.LBB0_2
	callq	__stack_chk_fail@PLT
	.addrsig_sym __stack_chk_fail
# AArch64 and RISC-V read a global, __stack_chk_guard, through the table of addresses the loader fills in
$ clang --target=aarch64-linux-gnu -O2 -fstack-protector-strong -S -o - copy_name_portable.c | grep -E 'stack_chk|cmp|b.ne'
	adrp	x8, :got:__stack_chk_guard
	ldr	x8, [x8, :got_lo12:__stack_chk_guard]
	adrp	x8, :got:__stack_chk_guard
	ldr	x8, [x8, :got_lo12:__stack_chk_guard]
	cmp	x8, x9
	b.ne	.LBB0_2
	bl	__stack_chk_fail
$ clang-23 --target=riscv64-linux-gnu -march=rv64gc -O2 -fstack-protector-strong -S -o - copy_name_portable.c | grep -E 'stack_chk|bne|call'
	auipc	a0, %got_pcrel_hi(__stack_chk_guard)
	call	strcpy
	call	puts
	bne	a0, a1, .LBB0_2
	call	__stack_chk_fail

x86-64 keeps it in the running thread’s own storage block and reads it with movq %fs:40, %rax, a single load with no address computation. AArch64 and RISC-V read the global, one through adrp and :got:__stack_chk_guard, the other through auipc and %got_pcrel_hi(__stack_chk_guard). The shape is identical on all three: load on entry, compare before the return, branch to a failing routine.

Checked library calls stop the program at the copy

The guard word acts late, at the return. A second addition acts at the moment of the copy, and it only appears when the compiler can see how much room the destination has.

/* One function with a 16-byte buffer and a library call, so the compiler shows both of its checks. */
#include <stdio.h>
#include <string.h>

void copy_name(const char *src) {
    char name[16];
    strcpy(name, src);
    printf("%s\n", name);
}

import pathlib
import sys

sys.path.insert(0, str(pathlib.Path(__file__).parent / "sessions"))
import session

session.run("""
# One function with a 16-byte buffer and a strcpy: the compiler adds two different checks
clang -O2 -fstack-protector-strong -S -o - copy_name.c | grep -E 'stack_chk|strcpy|cmp|b.ne|sub.sp'
""")

session.run("""
# A library call whose destination size the compiler knows becomes the checked form, and it stops the program at the copy
clang -O2 -fstack-protector-strong -o fortify_main fortify_main.c
{ ./fortify_main; } 2>/dev/null; echo "exit=$?"
# The version that compares the length itself accepts the short name and refuses the long one
clang -O1 -fstack-protector-strong -o copy_checked copy_checked.c
./copy_checked; echo "exit=$?"
""")

session.run("""
# No flag asked for the checked calls: the macro that switches them on already has a value in Apple's headers
echo '#include <string.h>' | clang -O2 -dM -E -x c - | grep 'define _FORTIFY_SOURCE'
""")

# One function with a 16-byte buffer and a strcpy: the compiler adds two different checks
$ clang -O2 -fstack-protector-strong -S -o - copy_name.c | grep -E 'stack_chk|strcpy|cmp|b.ne|sub.sp'
	sub	sp, sp, #48
	adrp	x8, ___stack_chk_guard@GOTPAGE
	ldr	x8, [x8, ___stack_chk_guard@GOTPAGEOFF]
	bl	___strcpy_chk
	adrp	x9, ___stack_chk_guard@GOTPAGE
	ldr	x9, [x9, ___stack_chk_guard@GOTPAGEOFF]
	cmp	x9, x8
	b.ne	LBB0_2
	bl	___stack_chk_fail
# A library call whose destination size the compiler knows becomes the checked form, and it stops the program at the copy
$ clang -O2 -fstack-protector-strong -o fortify_main fortify_main.c
$ { ./fortify_main; } 2>/dev/null; echo "exit=$?"
Aarav
exit=133
# The version that compares the length itself accepts the short name and refuses the long one
$ clang -O1 -fstack-protector-strong -o copy_checked copy_checked.c
$ ./copy_checked; echo "exit=$?"
copied: Aarav
rejected: input longer than the buffer
returned normally
exit=0
# No flag asked for the checked calls: the macro that switches them on already has a value in Apple's headers
$ echo '#include <string.h>' | clang -O2 -dM -E -x c - | grep 'define _FORTIFY_SOURCE'
#define _FORTIFY_SOURCE 2

The listing shows both checks in one function: the guard word around the frame, and bl ___strcpy_chk where the source said strcpy. The checked form takes the destination size as a third argument, and the compiler can supply it because name is a 16-byte array in this frame. Hardened library checkshardened library checks: Checked forms of library functions that a compiler substitutes when it can see the size of the destination, such as a copy routine that also takes that size and ends the process when the data does not fit. are what the substitution is called, and the macro _FORTIFY_SOURCE switches them on. The session’s last command shows that Apple’s headers already give it the value 2, which is why no flag asked for it; with the GNU C library you define it yourself, and the OpenSSF hardening guide recommends -D_FORTIFY_SOURCE=3 with optimisation at -O2 or above.

The run shows the order the two checks fire in. The short name printed. The long one did not: the process ended with status 133, which is 128 + 5, inside the checked copy, long before the guarded function would have reached its return. Apple’s published C library source shows what happened in there: its __strcpy_chk copies first, compares the length it copied with the 16 it was given, and ends the process on a mismatch. The bytes did land, but nothing after the copy ever ran. Step through the path from the source to that stop:

Both additions are narrow on purpose, and the same session shows the fix that is not narrow. The length-checked version compares the length with sizeof name first, returns a failure for input that does not fit, and prints rejected: input longer than the buffer before finishing with status 0. No mitigationmitigation: A defence that changes what a remaining bug can do, rather than removing the bug: it may detect the damage, deny a step an attack needs, or make a result unpredictable to anyone outside the process. was involved in that ending: the overflow simply never happened.

PIE, a non-executable stack and RELRO are facts about the file

The next three defences are not instructions at all. They are recorded in the executable’s headers, decided at link time, and applied by the loader, so you can read them from a file without running it.

import pathlib
import sys

sys.path.insert(0, str(pathlib.Path(__file__).parent / "sessions"))
import session

session.run("""
# On a Mach-O file the loader reads the flags in the header: PIE says the image may be placed anywhere
clang -O1 -fstack-protector-strong -o copy_checked copy_checked.c
otool -hv copy_checked | tail -2
""")

session.run("""
# The same questions on an ELF file are answered by the program headers, so link one object four ways
clang --target=aarch64-linux-gnu -ffreestanding -fPIE -c -o report_target.o report_target.c
ld.lld -pie -e _start -o report_pie report_target.o
ld.lld -pie -z relro -z now -e _start -o report_relro report_target.o
ld.lld -pie -z execstack -e _start -o report_execstack report_target.o
ld.lld -e _start -o report_fixed report_target.o
# DYN means the image may be placed anywhere; EXEC means it insists on its link-time addresses
llvm-readelf -h report_pie report_fixed | grep -E 'File:|Type:'
""")

session.run("""
# The stack's permissions and the read-only relocation window are program headers, one line each
llvm-readelf -l report_pie report_execstack | grep -E 'File:|GNU_STACK|GNU_RELRO'
llvm-readelf -l report_relro | grep -E 'GNU_STACK|GNU_RELRO'
llvm-readelf -d report_relro | grep -E 'FLAGS'
""")

# On a Mach-O file the loader reads the flags in the header: PIE says the image may be placed anywhere
$ clang -O1 -fstack-protector-strong -o copy_checked copy_checked.c
$ otool -hv copy_checked | tail -2
      magic  cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
MH_MAGIC_64    ARM64        ALL  0x00     EXECUTE    17       1056   NOUNDEFS DYLDLINK TWOLEVEL PIE
# The same questions on an ELF file are answered by the program headers, so link one object four ways
$ clang --target=aarch64-linux-gnu -ffreestanding -fPIE -c -o report_target.o report_target.c
$ ld.lld -pie -e _start -o report_pie report_target.o
$ ld.lld -pie -z relro -z now -e _start -o report_relro report_target.o
$ ld.lld -pie -z execstack -e _start -o report_execstack report_target.o
$ ld.lld -e _start -o report_fixed report_target.o
# DYN means the image may be placed anywhere; EXEC means it insists on its link-time addresses
$ llvm-readelf -h report_pie report_fixed | grep -E 'File:|Type:'
File: report_pie
  Type:                              DYN (Shared object file)
File: report_fixed
  Type:                              EXEC (Executable file)
# The stack's permissions and the read-only relocation window are program headers, one line each
$ llvm-readelf -l report_pie report_execstack | grep -E 'File:|GNU_STACK|GNU_RELRO'
File: report_pie
  GNU_RELRO      0x000270 0x0000000000020270 0x0000000000020270 0x000090 0x000d90 R   0x1
  GNU_STACK      0x000000 0x0000000000000000 0x0000000000000000 0x000000 0x000000 RW  0x0
File: report_execstack
  GNU_RELRO      0x000270 0x0000000000020270 0x0000000000020270 0x000090 0x000d90 R   0x1
  GNU_STACK      0x000000 0x0000000000000000 0x0000000000000000 0x000000 0x000000 RWE 0x0
$ llvm-readelf -l report_relro | grep -E 'GNU_STACK|GNU_RELRO'
  GNU_RELRO      0x000270 0x0000000000020270 0x0000000000020270 0x0000a0 0x000d90 R   0x1
  GNU_STACK      0x000000 0x0000000000000000 0x0000000000000000 0x000000 0x000000 RW  0x0
$ llvm-readelf -d report_relro | grep -E 'FLAGS'
  0x000000000000001e (FLAGS)    BIND_NOW
  0x000000006ffffffb (FLAGS_1)  NOW PIE

The Mach-O header’s flags line ends with PIE: this image is a position-independent executableposition-independent executable: An executable whose code reaches other code and data through PC-relative addressing instead of fixed addresses, so it runs correctly at any load address. This is what lets the loader randomise its placement under ASLR. More in “Memory and a process's address space”, so the loader may place it at a different address on every run. The same question on an ELF file is answered by the file type, and the recorded links show both answers from one object: DYN for the position-independent link, EXEC for the link that insists on its own addresses.

Two program headers carry the other two fields. GNU_STACK holds the permissions the kernel gives the stack mapping: RW in the default link and the RELRO link, and RWE in the one linked with -z execstack. A non-executable stacknon-executable stack: A stack mapping without execute permission, so instructions cannot be fetched from bytes stored there. On ELF the GNU_STACK program header carries the permissions. is the RW case, and what it stops is narrow and exact: instructions cannot be fetched from bytes stored on the stack. It does not stop a single write. GNU_RELRO names a window of memory the loader write-protects once it has applied the relocations in it. ld.lld writes one unless it is told -z norelro, so all three position-independent links show it; only the link with -z relro -z now also reported BIND_NOW and NOW PIE in its dynamic flags: resolve everything up front, then make that window read-only. Read-only relocationsread-only relocations: A link setting that has the loader apply the relocations in one window of memory and then write-protect that window, so the pointer tables in it cannot be changed later. Full protection also needs the relocations resolved before the program starts. are the name for the window, and the OpenSSF guide calls the pair with -z now full RELRO: without it, the table that lazy binding fills in later has to stay writable, so it cannot be inside the window.

Reading a hardening report of your own binary

Put those fields together and you have a report you can run on any binary you built. Four questions, four places to look, and one honest limit on each answer:

The last column is the part a report usually leaves out. A binary can pass every field above and still overflow a buffer on every run. The guarded build of the first section is a small proof: its guard-word code was in place, and it still wrote 36 bytes into a 16-byte buffer each time. The protector changed how the bug ended, from an unpredictable jump to an immediate stop, and that is all it changed. A report is a list of defences that are switched on, never a statement that the program is memory-safe.

Two practical cautions follow. First, a field read from a header survives stripping, while a name may not: in a static link __stack_chk_fail is one more name in the symbol table and goes with it, so a stripped binary without that name has to be checked in its disassembly before it is called unprotected. Second, these are build settings on code you own: when a field comes back missing, the answer is a rebuild with the flag, which the compilers’ documentation and the OpenSSF guide list by name.

Practise

These run on an Apple silicon Mac; on a Linux machine, follow setting up the toolchain first. Work on copies of this page’s own files.

1. Build copy_loop.c with -fstack-protector instead of -fstack-protector-strong and count the stack_chk lines in the listing. The buffer is 16 bytes, so both forms protect this function; now shrink name to 4 bytes and count again, and the plain form drops it while the strong form keeps it.
2. Add a second function to copy_loop.c that only multiplies two integers, build with the strong form and look for stack_chk inside it. There is none: no array, no address of a local, nothing to guard.
3. Run the report session’s four links again with -z noexecstack added to the execstack link and compare the GNU_STACK line. Then link report_target.o without -pie and check which of the four report fields change.
4. Change copy_checked.c to use a buffer of 64 bytes and run it with the same long name: the rejection line disappears, because the 36 bytes and the terminating zero now fit. The fix depends on the size comparison, not on the buffer being small.

Interview questions

A warm-up for freshers: a build has the stack protector on, so is the program safe from buffer overflows?

No. The protector does not prevent an overflow; it detects one particular consequence of one afterwards. A guard word is copied into the frame on entry and compared before the return, so an overflow that runs from a local buffer towards the saved return address changes the copy and the comparison fails, ending the process. The overflow still happened: every byte was written. An overflow that goes the other way, one into a heap buffer, one that stops short of the guard word, or one in a function the protector did not cover, is not noticed at all. In this lesson’s recorded run, the guarded build wrote 36 bytes into a 16-byte buffer and printed them before the check fired. The fix is the length comparison in the code; the protector decides only how a remaining bug fails.

Three deeper questions follow from this page: choosing flags for a build you own, reading a report on a stripped binary, and explaining to a team what a mitigation does not do.

Key takeaways

• A hardened build adds a guard word to functions with arrays or escaping locals, loaded from __stack_chk_guard (or from %fs:40 on x86-64), and compares it just before the return.
• A library call whose destination size the compiler can see becomes a checked call, which ends the process at the copy instead of at the return; a hand-written loop never becomes one.
• PIE, the stack’s permissions and the read-only relocation window are header fields the linker wrote, readable from a file you never run.
• Every field in a hardening report says a defence is switched on, and none of them says the program has no memory bugs: the length comparison in the source is the only thing that removes the bug.