Reverse engineering binaries you built

You built a small tool, shipped the release build, and lost the source. The binary still runs, but nm on the stripped copy names no function of your own, only the two it imports, printf and strlen, and its one function sits at a bare offset 0x460, 1120 bytes into the file. Recovering what that function computes, exactly, is reverse engineeringreverse engineering: Recovering how a compiled program works from the program itself when its source is not available: its format, where it starts, what it calls and what its functions compute. Whether you may do it depends on who owns the program and on its licence. a binary you built: reading the compiled file to recover its behaviour once the source is gone.

This page takes apart one stripped program in a fixed order that reads the cheap clues before the dear ones: check that you may study it, identify the file, find where it starts, read its imports and strings, and only then read instructions.

Read only what you may, then work from the outside in

The first question is not a tool but permission. A binary you compiled, or one built for you to study, is yours to take apart; a third party’s program is not, unless its licence and the law allow it, and such allowances are narrow. United States law, for one, lets a lawful user get around an access control only to identify what an independently written program needs to work with it, and only when that was not already available. Settle that first.

The order matters because each step narrows the next: the file’s type picks the tools, the entry point says where the code begins, and the imports and strings name the landmarks to read the instructions around.

Identify the file, its processor and where it starts

file and otool -hv answer what the thing is before an instruction is read; the header’s last word, PIE, marks a position-independent executableposition-independent executable: An executable whose code reaches other code and data through PC-relative addressing instead of fixed addresses, so it runs correctly at any load address. This is what lets the loader randomise its placement under ASLR. More in “Memory and a process's address space”, whose load address changes from run to run. otool -l then prints the LC_MAIN load command, whose entryoff is the offset of the first instruction to run; an ELFELF: The Executable and Linkable Format of the generic System V ABI, used by Linux and most Unix systems. It has a section header table for the linker and a program header table for the loader. More in “From source to executable: objects and linking” file names its entry as an address, e_entry, which readelf -h prints.

/* The program this page takes apart: it prints a 32-bit checksum of a word, or of "kaksha" when given none. The page
   builds it, strips it, and then works only from the stripped file, as if this source were lost. */
#include <stdio.h>
#include <string.h>

static unsigned int mix(unsigned int h, unsigned char c) {
    h ^= c;
    h *= 16777619u;
    return h;
}

static unsigned int checksum(const char *s) {
    unsigned int h = 2166136261u;
    for (size_t i = 0; i < strlen(s); i++) {
        h = mix(h, (unsigned char)s[i]);
    }
    return h;
}

int main(int argc, char **argv) {
    const char *word = (argc > 1) ? argv[1] : "kaksha";
    printf("%s -> %08x\n", word, checksum(word));
    return 0;
}

import pathlib
import sys

sys.path.insert(0, str(pathlib.Path(__file__).parent / "sessions"))
import session

session.run("""
# Build the program and keep a stripped copy; from here on, only the stripped file is read
clang -O2 -o checksum checksum.c
strip -o checksum_stripped checksum
# What kind of file it is, for which processor, and the header flags the loader reads
file checksum_stripped
otool -hv checksum_stripped | tail -1
# Where it starts: LC_MAIN gives main's offset in the file, which is where the reading begins
otool -l checksum_stripped | grep -A3 LC_MAIN
# What stripping took: the name main is gone, while the two imports stay because the loader binds them by name
nm checksum
nm checksum_stripped
""")

# Build the program and keep a stripped copy; from here on, only the stripped file is read
$ clang -O2 -o checksum checksum.c
$ strip -o checksum_stripped checksum
# What kind of file it is, for which processor, and the header flags the loader reads
$ file checksum_stripped
checksum_stripped: Mach-O 64-bit executable arm64
$ otool -hv checksum_stripped | tail -1
MH_MAGIC_64    ARM64        ALL  0x00     EXECUTE    17       1056   NOUNDEFS DYLDLINK TWOLEVEL PIE
# Where it starts: LC_MAIN gives main's offset in the file, which is where the reading begins
$ otool -l checksum_stripped | grep -A3 LC_MAIN
       cmd LC_MAIN
   cmdsize 24
  entryoff 1120
 stacksize 0
# What stripping took: the name main is gone, while the two imports stay because the loader binds them by name
$ nm checksum
0000000100000000 T __mh_execute_header
0000000100000460 T _main
                 U _printf
                 U _strlen
$ nm checksum_stripped
0000000100000000 T __mh_execute_header
                 U _printf
                 U _strlen

Stripping removed the name _main from the symbol table, so nm on the stripped file lists only the header symbol and the two undefined imports. The entry offset 1120, which is 0x460 in hex, is where the reading begins, and the header already names the file: an arm64 Mach-O executable whose code starts there. Nothing so far has read an instruction.

Imports and strings are the clues a stripped file keeps

A stripped file still names what it imports, because the loader binds each import by name when the program starts; the code reaches each one through a symbol stubsymbol stub: A few instructions the Mach-O linker writes into the __TEXT,__stubs section for each imported function. On arm64 a stub loads the function's address from its __got slot into x16 and branches there. More in “Calling C library functions from assembly”. otool -Iv lists them, and strings prints the readable byte sequences, by default runs of at least four printable characters.

import pathlib
import sys

sys.path.insert(0, str(pathlib.Path(__file__).parent / "sessions"))
import session

session.run("""
# The same stripped build as before
clang -O2 -o checksum checksum.c
strip -o checksum_stripped checksum
# The imports: every library call goes through a stub, and the stripped file still names each one
otool -Iv checksum_stripped
# The strings: a word, and a format that prints a word, an arrow and eight hex digits
strings checksum_stripped
# Run it on inputs you choose: no word, two words one letter apart, and the empty word
./checksum_stripped
./checksum_stripped hello
./checksum_stripped hellp
./checksum_stripped ''
""")

# The same stripped build as before
$ clang -O2 -o checksum checksum.c
$ strip -o checksum_stripped checksum
# The imports: every library call goes through a stub, and the stripped file still names each one
$ otool -Iv checksum_stripped
checksum_stripped:
Indirect symbols for (__TEXT,__stubs) 2 entries
address            index name
0x00000001000004e0     2 _printf
0x00000001000004ec     3 _strlen
Indirect symbols for (__DATA_CONST,__got) 2 entries
address            index name
0x0000000100004000     2 _printf
0x0000000100004008     3 _strlen
# The strings: a word, and a format that prints a word, an arrow and eight hex digits
$ strings checksum_stripped
kaksha
%s -> %08x
# Run it on inputs you choose: no word, two words one letter apart, and the empty word
$ ./checksum_stripped
kaksha -> ca60f7d4
$ ./checksum_stripped hello
hello -> 4f9f2cab
$ ./checksum_stripped hellp
hellp -> 5c9f4122
$ ./checksum_stripped ''
 -> 811c9dc5

Two imports, printf and strlen, and two strings, a default word and a format %s -> %08x, already sketch the program: it measures a string, folds it into a 32-bit value and prints that as eight hex digits. Running it confirms the shape: the empty word gives a fixed starting value 811c9dc5, and hello and hellp, one letter apart, differ in six of their eight hex digits, as a hash’s results do. The code between those calls comes next, read from the listing and proved by output. A listing shows every instruction exactly, a decompilerdecompiler: A program that reads a binary's machine code and rebuilds C-like source from it, so that a function can be read as statements and loops rather than as instructions. Ghidra is one widely used example. rebuilds C-like source from the bytes, and a debugger shows real values as the program runs.

Practise

Work on your own copies of checksum.c and the sessions above; for a machine other than macOS on Apple silicon, see setting up the toolchain.

1. Build and strip checksum.c, then read the entry offset from otool -l and predict which section 0x460 falls in before you disassemble.
2. Run the stripped program on a word and on the same word with one letter changed, and count how many of the eight hex digits differ.
3. Add a third string to checksum.c, rebuild and strip, and find it with strings; note that _main stays gone.
4. Run otool -Iv on the stripped file and match each stub address to the import it binds.

Recovering the checksum's loop from the listing, and proving it by output Pro: opens with a Pro, Premium or Ultimate pass

What stripping leaves behind, and which way of reading fits the question Pro: opens with a Pro, Premium or Ultimate pass

Interview questions

A warm-up for freshers: you are handed a stripped binary your team built and asked what one function does. What do you look at first, before any disassembly?

Before any disassembly, read the file from the outside in, cheapest clue first. file and otool -hv, or readelf -h for an ELF file, give the format, the processor and whether the code is position independent, and the entry load command gives the offset where the code begins. Then otool -Iv lists the library functions it imports, which a stripped file keeps because the loader binds them by name, and strings prints its readable text. Those landmarks usually sketch what the function does before you read a single instruction, and they tell you which instructions are worth reading closely.

Three harder questions follow: recovering a lost function exactly, the scope of work on someone else’s binary, and what stripping a release build really buys.

Key takeaways

• Reverse engineering a binary you built means reading the compiled file to recover its behaviour, and it starts with whether you are allowed to read it at all.
• Work from the outside in: identify the file, find its entry, then read its imports and strings, so you know the landmarks before you read any instructions.
• A stripped file still names the library functions it imports, because the loader binds those by name, so stripping hides a program’s own names but not what it calls.
• Running the program on inputs you choose, such as the empty word and two words one letter apart, confirms what the imports and strings suggest before any instruction is read.