Calls, returns and the link register

greet_broken calls puts twice in seven AArch64 instructions, prints both of its lines, and then never comes back: the test driver’s alarm has to end the program after 1 second, with exit status 3. The working version stores x30, the register that holds the address it has to return to, before its first call, and loads it back before it returns.

A call does two jobs. It records where the program must come back to, and then it jumps. x86-64 records that address on the stack, AArch64 and RISC-V in a register, and that difference decides which functions must save something before they call. You will follow the return address through a call on each ISA, save it where it has to be saved, pick out leaf, non-leaf and tail calls in compiler output, and see what the linker does with a call whose target lies too far away.

A call writes down where to come back, then jumps

The leaf here below is two instructions long, and it returns the one thing it knows that its caller does not: the address it is about to return to. gap_bl notes the address of its own bl, calls here, and subtracts. The difference says how far past the call the return address points.

On AArch64, bl writes the address of the instruction after itself into x30, the link registerlink register: The register that holds the return address a call left behind: x30 on AArch64 and ra (x1) on RISC-V. A leaf function can return through it without using the stack; x86-64 instead keeps the return address on the stack. More in “Registers on x86-64, AArch64 and RISC-V”, and then branches. That address is the return addressreturn address: The address a call records so that the callee can come back: the address of the instruction right after the call. x86-64's call pushes it onto the stack; AArch64's bl and blr and RISC-V's jal and call write it to x30 or ra., and ret branches to whatever x30 holds. Arm’s instruction set reference puts it in two lines: BL branches and sets X30 to PC+4, and RET branches to a register that defaults to X30. blr sets X30 the same way but takes its target from a register, which is the shape of a call through a function pointer. gap_bl calls here itself, so it parks its own return address on the stack first, for the reason the next section shows.

// Where does a call leave its return address, and what is it? Each gap function notes the address of
// its own call, calls the leaf here, and subtracts: the answer is how far past the call the return goes.
    .text
    .p2align 2
_here:                          // long here(void): a leaf, it calls nothing
    mov x0, x30                 // bl and blr left the return address in x30, the link register
    ret                         // branch to the address in x30

    .globl _gap_bl
    .p2align 2
_gap_bl:                        // long gap_bl(void)
    str x30, [sp, #-16]!        // park our own return address: the bl below overwrites x30
    adr x1, Lcall_bl            // x1 = the address of the bl
Lcall_bl:
    bl _here                    // x30 = the address of the next instruction, then jump
    sub x0, x0, x1              // here's answer minus the address of the bl
    ldr x30, [sp], #16          // take our return address back
    ret

    .globl _gap_blr
    .p2align 2
_gap_blr:                       // long gap_blr(void): the same call through a register
    str x30, [sp, #-16]!
    adr x8, _here               // x8 = a pointer to here
    adr x1, Lcall_blr
Lcall_blr:
    blr x8                      // call the address in x8: x30 = the next instruction
    sub x0, x0, x1
    ldr x30, [sp], #16
    ret

# The same two gap functions for x86-64 Linux (System V), in Intel operand order. call pushes the
# return address on the stack, so a caller's own return address is never overwritten by a call it makes.
.intel_syntax noprefix
.text
here:                           # long here(void): a leaf
    mov rax, [rsp]              # the call pushed the return address: it is the top word of the stack
    ret                         # pop that word into rip

.globl gap_call
gap_call:                       # long gap_call(void)
    lea rcx, [rip + .Lcall]     # rcx = the address of the call
.Lcall:
    call here                   # push the address of the next instruction, then jump
    sub rax, rcx                # here's answer minus the address of the call
    ret

.globl gap_call_reg
gap_call_reg:                   # long gap_call_reg(void): the same call through a register
    lea rdx, [rip + here]       # rdx = a pointer to here
    lea rcx, [rip + .Lcall_reg]
.Lcall_reg:
    call rdx                    # call the address in rdx
    sub rax, rcx
    ret

.section .note.GNU-stack,"",@progbits

# The same gap functions for 64-bit RISC-V Linux (LP64D). jal, jalr and call write the return address
# to ra (x1), as bl does to x30, so a function that calls must park its own ra first.
.text
here:                           # long here(void): a leaf
    mv a0, ra                   # the call left the return address in ra
    ret                         # jalr zero, 0(ra): jump to ra and keep no new link

.globl gap_jal
gap_jal:                        # long gap_jal(void)
    addi sp, sp, -16
    sd ra, 8(sp)                # park our own return address
    lla a1, .Lcall_jal          # a1 = the address of the jal
.Lcall_jal:
    jal ra, here                # ra = the address of the next instruction, then jump
    sub a0, a0, a1              # here's answer minus the address of the jal
    ld ra, 8(sp)
    addi sp, sp, 16
    ret

.globl gap_call
gap_call:                       # long gap_call(void): the call pseudo-instruction
    addi sp, sp, -16
    sd ra, 8(sp)
    lla a1, .Lcall
.Lcall:
    call here                   # auipc and jalr: two instructions
    sub a0, a0, a1
    ld ra, 8(sp)
    addi sp, sp, 16
    ret

.globl gap_jalr
gap_jalr:                       # long gap_jalr(void): the same call through a register
    addi sp, sp, -16
    sd ra, 8(sp)
    lla a5, here                # a5 = a pointer to here
    lla a1, .Lcall_jalr
.Lcall_jalr:
    jalr ra, 0(a5)              # call the address in a5: ra = the next instruction
    sub a0, a0, a1
    ld ra, 8(sp)
    addi sp, sp, 16
    ret

.section .note.GNU-stack,"",@progbits

A C driver calls both AArch64 gap functions and prints what they return:

/* Calls the two AArch64 gap functions and prints how far past each call its return address points. */
#include <stdio.h>

long gap_bl(void);
long gap_blr(void);

int main(void) {
    printf("bl:  the return address is the address of the call + %ld bytes\n", gap_bl());
    printf("blr: the return address is the address of the call + %ld bytes\n", gap_blr());
    return 0;
}

The session runs the AArch64 version and lists the bytes of the other two:

"""Runs the AArch64 gap functions, then assembles the x86-64 and RISC-V versions and lists their bytes:
how long each call instruction is, and so how far past it the return address points."""

import pathlib
import sys

sys.path.insert(0, str(pathlib.Path(__file__).parent / "sessions"))
import session  # noqa: E402

session.run("""
# AArch64: build the C driver with gap_aarch64.s and run it
clang -o gap gap_main.c gap_aarch64.s
./gap
# x86-64: assemble gap_x86_64.s into an ELF object and list it, bytes and instructions
clang --target=x86_64-linux-gnu -c -o gap_x86_64.o gap_x86_64.s
llvm-objdump -d -M intel gap_x86_64.o
# RISC-V: assemble gap_riscv64.s and list it with the real instruction names
llvm-mc --triple=riscv64-linux-gnu -filetype=obj -o gap_riscv64.o gap_riscv64.s
llvm-objdump -d -M no-aliases gap_riscv64.o
""", max_lines=60)

# AArch64: build the C driver with gap_aarch64.s and run it
$ clang -o gap gap_main.c gap_aarch64.s
$ ./gap
bl:  the return address is the address of the call + 4 bytes
blr: the return address is the address of the call + 4 bytes
# x86-64: assemble gap_x86_64.s into an ELF object and list it, bytes and instructions
$ clang --target=x86_64-linux-gnu -c -o gap_x86_64.o gap_x86_64.s
$ llvm-objdump -d -M intel gap_x86_64.o

gap_x86_64.o:	file format elf64-x86-64

Disassembly of section .text:

0000000000000000 <here>:
       0: 48 8b 04 24                  	mov	rax, qword ptr [rsp]
       4: c3                           	ret

0000000000000005 <gap_call>:
       5: 48 8d 0d 00 00 00 00         	lea	rcx, [rip]              # 0xc <gap_call+0x7>
       c: e8 ef ff ff ff               	call	0x0 <here>
      11: 48 29 c8                     	sub	rax, rcx
      14: c3                           	ret

0000000000000015 <gap_call_reg>:
      15: 48 8d 15 e4 ff ff ff         	lea	rdx, [rip - 0x1c]       # 0x0 <here>
      1c: 48 8d 0d 00 00 00 00         	lea	rcx, [rip]              # 0x23 <gap_call_reg+0xe>
      23: ff d2                        	call	rdx
      25: 48 29 c8                     	sub	rax, rcx
      28: c3                           	ret
# RISC-V: assemble gap_riscv64.s and list it with the real instruction names
$ llvm-mc --triple=riscv64-linux-gnu -filetype=obj -o gap_riscv64.o gap_riscv64.s
$ llvm-objdump -d -M no-aliases gap_riscv64.o

gap_riscv64.o:	file format elf64-littleriscv

Disassembly of section .text:

0000000000000000 <here>:
       0: 00008513     	addi	a0, ra, 0x0
       4: 00008067     	jalr	zero, 0x0(ra)

0000000000000008 <gap_jal>:
       8: ff010113     	addi	sp, sp, -0x10
       c: 00113423     	sd	ra, 0x8(sp)
      10: 00000597     	auipc	a1, 0x0
      14: 00858593     	addi	a1, a1, 0x8
      18: fe9ff0ef     	jal	ra, 0x0 <here>
      1c: 40b50533     	sub	a0, a0, a1
      20: 00813083     	ld	ra, 0x8(sp)
      24: 01010113     	addi	sp, sp, 0x10
      28: 00008067     	jalr	zero, 0x0(ra)

000000000000002c <gap_call>:
      2c: ff010113     	addi	sp, sp, -0x10
      30: 00113423     	sd	ra, 0x8(sp)
      34: 00000597     	auipc	a1, 0x0
      38: 00858593     	addi	a1, a1, 0x8
      3c: 00000097     	auipc	ra, 0x0
      40: fc4080e7     	jalr	ra, -0x3c(ra) <here>
      44: 40b50533     	sub	a0, a0, a1
      48: 00813083     	ld	ra, 0x8(sp)
      4c: 01010113     	addi	sp, sp, 0x10
      50: 00008067     	jalr	zero, 0x0(ra)

0000000000000054 <gap_jalr>:
      54: ff010113     	addi	sp, sp, -0x10
      58: 00113423     	sd	ra, 0x8(sp)
      5c: 00000797     	auipc	a5, 0x0
      60: fa478793     	addi	a5, a5, -0x5c
      64: 00000597     	auipc	a1, 0x0
      68: 00858593     	addi	a1, a1, 0x8
      6c: 000780e7     	jalr	ra, 0x0(a5)
      70: 40b50533     	sub	a0, a0, a1
      74: 00813083     	ld	ra, 0x8(sp)
      78: 01010113     	addi	sp, sp, 0x10
      7c: 00008067     	jalr	zero, 0x0(ra)

Both AArch64 calls printed 4, the length of a bl and of a blr. The x86-64 listing shows a different rule behind the same idea. call here is e8 and a 4-byte offset, 5 bytes in all, and call rdx is ff d2, 2 bytes. Intel’s manual says a near call pushes the address of the instruction that follows it onto the stack, and ret, c3, pops that address into rip. x86-64 has no link register, so its here finds the return address with mov rax, [rsp]: the System V ABI says that at a function’s first instruction rsp points to the return address.

RISC-V puts the address in ra, which is x1, as AArch64 puts it in x30. jal ra, here is 4 bytes, and call here is a pseudo-instructionpseudo-instruction: A name an assembler accepts where an instruction goes although the instruction set has no instruction of that name: RISC-V li and la become one or more real instructions. The NASM manual also uses the word for its data lines, such as db, equ and times. More in “Assembler syntax: NASM, GNU as, Intel and AT&T” for two, auipc and jalr, so ra ends up 8 bytes past the start of the call. Its ret is jalr zero, 0(ra): a jump to ra whose own link goes to the zero register, which drops it. Side by side, the three ISAs keep and use the return address like this:

Step through the AArch64 call once, with x30 as the third column:

The result came back in x0 on AArch64 and would come back in rax and a0 on the other two; which registers carry arguments and results is the subject of calling conventions.

A function that calls another must save its link register first

greet_broken and greet_fixed both call puts twice. The broken one never saves x30; the fixed one stores it before its first call and loads it back before ret. The other two tabs hold a working greet for x86-64 and RISC-V, to compare with later:

// greet calls puts twice. greet_broken never saves x30; greet_fixed saves it before its first call.
    .text
    .globl _greet_broken
    .p2align 2
_greet_broken:
    adrp x0, Lhello@PAGE           // x0 = the address of "hello"
    add x0, x0, Lhello@PAGEOFF
    bl _puts                       // x30 = the address of the next instruction: main's is gone
    adrp x0, Lagain@PAGE
    add x0, x0, Lagain@PAGEOFF
    bl _puts                       // x30 = the address of the ret below
    ret                            // branch to x30: to this ret, for ever

    .globl _greet_fixed
    .p2align 2
_greet_fixed:
    stp x29, x30, [sp, #-16]!      // save x30 (and x29) on the stack before any call
    mov x29, sp
    adrp x0, Lhello@PAGE
    add x0, x0, Lhello@PAGEOFF
    bl _puts
    adrp x0, Lagain@PAGE
    add x0, x0, Lagain@PAGEOFF
    bl _puts
    ldp x29, x30, [sp], #16        // x30 = main's return address again
    ret

    .section __TEXT,__cstring
Lhello: .asciz "hello"
Lagain: .asciz "again"

# greet for x86-64 Linux (System V): two calls of puts. Each call pushes its own return address below
# ours, so nothing has to be saved for ret; every byte taken from the stack must be given back first.
.intel_syntax noprefix
.text
.globl greet
greet:
    sub rsp, 8                  # puts expects rsp 16-byte aligned at the call (alignment has rules of its own)
    lea rdi, [rip + .Lhello]    # the string's address, for puts
    call puts@PLT               # push the address of the next instruction, jump to puts
    lea rdi, [rip + .Lagain]
    call puts@PLT
    add rsp, 8                  # give back the 8 bytes: our return address is on top again
    ret                         # pop it into rip

.section .rodata
.Lhello: .asciz "hello"
.Lagain: .asciz "again"
.section .note.GNU-stack,"",@progbits

# greet for 64-bit RISC-V Linux: call writes ra, as bl writes x30, so ra is saved before the first call.
.text
.globl greet
greet:
    addi sp, sp, -16            # room on the stack, in a 16-byte step
    sd ra, 8(sp)                # save our return address before any call overwrites ra
    lla a0, .Lhello             # the string's address, for puts
    call puts                   # ra = the address of the next instruction, then jump
    lla a0, .Lagain
    call puts
    ld ra, 8(sp)                # our own return address again
    addi sp, sp, 16
    ret                         # jalr zero, 0(ra)

.section .rodata
.Lhello: .asciz "hello"
.Lagain: .asciz "again"
.section .note.GNU-stack,"",@progbits

The C driver arms a 1-second alarm, then calls the version named on its command line. The session builds one program with both versions, runs each, and lists the broken one:

/* Calls greet_fixed, or greet_broken when the first argument is "broken". A function that never returns
   would hang the program, so an alarm ends it after 1 second with exit status 3. */
#include <signal.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>

void greet_broken(void);
void greet_fixed(void);

static void too_long(int sig) {
    (void)sig;
    static const char msg[] = "still inside after 1 s: the function never returned\n";
    write(1, msg, sizeof msg - 1);
    _exit(3);
}

int main(int argc, char **argv) {
    signal(SIGALRM, too_long);
    alarm(1);
    if (argc > 1 && strcmp(argv[1], "broken") == 0) greet_broken(); else greet_fixed();
    puts("back in main");
    return 0;
}

"""Builds greet_fixed and greet_broken into one AArch64 program and runs each, lists the broken function's
offsets, then assembles the x86-64 and RISC-V versions of greet."""

import pathlib
import sys

sys.path.insert(0, str(pathlib.Path(__file__).parent / "sessions"))
import session  # noqa: E402

session.run("""
# AArch64: one program, both versions of greet
clang -o greet greet_main.c greet_aarch64.s
# greet_fixed saves x30 before its first call
./greet fixed; echo "exit=$?"
# greet_broken never saves x30
./greet broken; echo "exit=$?"
# Where each instruction of greet_broken sits, counted from its first byte, and what the linker fills in (-r)
clang -c -o greet_aarch64.o greet_aarch64.s
llvm-objdump -d -r --disassemble-symbols=_greet_broken greet_aarch64.o
# x86-64 and RISC-V: the working greet, assembled into ELF objects
clang --target=x86_64-linux-gnu -c -o greet_x86_64.o greet_x86_64.s
llvm-mc --triple=riscv64-linux-gnu -filetype=obj -o greet_riscv64.o greet_riscv64.s
""")

# AArch64: one program, both versions of greet
$ clang -o greet greet_main.c greet_aarch64.s
# greet_fixed saves x30 before its first call
$ ./greet fixed; echo "exit=$?"
hello
again
back in main
exit=0
# greet_broken never saves x30
$ ./greet broken; echo "exit=$?"
hello
again
still inside after 1 s: the function never returned
exit=3
# Where each instruction of greet_broken sits, counted from its first byte, and what the linker fills in (-r)
$ clang -c -o greet_aarch64.o greet_aarch64.s
$ llvm-objdump -d -r --disassemble-symbols=_greet_broken greet_aarch64.o

greet_aarch64.o:	file format mach-o arm64

Disassembly of section __TEXT,__text:

0000000000000000 <_greet_broken>:
       0: 90000000     	adrp	x0, 0x0 <ltmp0>
		0000000000000000:  ARM64_RELOC_PAGE21	Lhello
       4: 91000000     	add	x0, x0, #0x0
		0000000000000004:  ARM64_RELOC_PAGEOFF12	Lhello
       8: 94000000     	bl	0x8 <ltmp0+0x8>
		0000000000000008:  ARM64_RELOC_BRANCH26	_puts
       c: 90000000     	adrp	x0, 0x0 <ltmp0>
		000000000000000c:  ARM64_RELOC_PAGE21	Lagain
      10: 91000000     	add	x0, x0, #0x0
		0000000000000010:  ARM64_RELOC_PAGEOFF12	Lagain
      14: 94000000     	bl	0x14 <ltmp0+0x14>
		0000000000000014:  ARM64_RELOC_BRANCH26	_puts
      18: d65f03c0     	ret
# x86-64 and RISC-V: the working greet, assembled into ELF objects
$ clang --target=x86_64-linux-gnu -c -o greet_x86_64.o greet_x86_64.s
$ llvm-mc --triple=riscv64-linux-gnu -filetype=obj -o greet_riscv64.o greet_riscv64.s

Both versions printed hello and again, so both calls of puts worked. Only greet_fixed got back to main. The listing explains the broken run with three offsets: the second bl sits at 0x14, so it writes greet_broken + 0x18 into x30, and 0x18 is the function’s own ret. When that ret runs, it branches to the address in x30, which is its own address, so it runs again, and again. The loop moves no stack and prints nothing; only the alarm stops it.

The first bl did the damage. Before it, x30 held the only copy of main’s return address; the bl replaced it with an address inside greet_broken, and nothing had kept the old value. Step through the run:

greet_fixed adds three instructions. stp x29, x30, [sp, #-16]! stores x29 and x30 in a 16-byte slot below the old stack pointer before the first call, mov x29, sp points x29 at that slot, and ldp x29, x30, [sp], #16 loads both back just before ret. The pair and writeback forms are those of the stack lesson; the slot, with x29 pointing at it, is a frame record, which callee-saved registers and frames explains. RISC-V follows the same rule with ra: its greet stores ra with sd ra, 8(sp) before the first call and reloads it with ld ra, 8(sp).

x86-64 cannot lose the return address this way. Each call pushes its own return address below the caller’s, so greet’s return address is still in its stack slot, untouched, when greet’s own ret runs. What x86-64 code must get right instead is the stack pointer: ret pops whatever word is on top, so the sub rsp, 8 has to be undone first. Leave a pushed register on the stack and ret pops that value as an address.

Compilers skip the save in a leaf and turn a last call into a jump

tail.c holds three small functions. leaf calls nothing, not_tail calls g and then adds 1 to its result, and tail adds 1 first and ends with the call. The session prints what clang writes for each at -O2 on the three ISAs, then tail once more at -O0:

/* Three small functions for compiler output: one calls nothing, one does work after its call,
   and one ends with its call. g is defined elsewhere, so the compiler cannot inline it. */
long g(long x);

long leaf(long x) { return x * 3 + 1; }
long not_tail(long x) { return g(x) + 1; }
long tail(long x) { return g(x + 1); }

"""Compiles tail.c at -O2 for AArch64 macOS, x86-64 Linux and RISC-V Linux and prints the assembly the
compiler writes, then the tail function once more at -O0."""

import pathlib
import sys

sys.path.insert(0, str(pathlib.Path(__file__).parent / "sessions"))
import session  # noqa: E402

session.run("""
# Each command prints the compiler's assembly; grep drops the directive and comment lines
clang --target=arm64-apple-macos -O2 -S -fno-asynchronous-unwind-tables -o - tail.c | grep -v -E '^[[:space:]]*([.#;/]|$)'
clang --target=x86_64-linux-gnu -O2 -S -masm=intel -fno-asynchronous-unwind-tables -o - tail.c | grep -v -E '^[[:space:]]*([.#;/]|$)'
clang-23 --target=riscv64-linux-gnu -march=rv64gc -mabi=lp64d -O2 -S -fno-asynchronous-unwind-tables -o - tail.c | grep -v -E '^[[:space:]]*([.#;/]|$)'
# AArch64 at -O0: only the tail function, from its label to its ret
clang --target=arm64-apple-macos -O0 -S -fno-asynchronous-unwind-tables -o - tail.c | grep -v -E '^[[:space:]]*([.#;/]|$)' | sed -n '/^_tail:/,/ret$/p'
""")

# Each command prints the compiler's assembly; grep drops the directive and comment lines
$ clang --target=arm64-apple-macos -O2 -S -fno-asynchronous-unwind-tables -o - tail.c | grep -v -E '^[[:space:]]*([.#;/]|$)'
_leaf:                                  ; @leaf
	add	x8, x0, x0, lsl #1
	add	x0, x8, #1
	ret
_not_tail:                              ; @not_tail
	stp	x29, x30, [sp, #-16]!           ; 16-byte Folded Spill
	mov	x29, sp
	bl	_g
	add	x0, x0, #1
	ldp	x29, x30, [sp], #16             ; 16-byte Folded Reload
	ret
_tail:                                  ; @tail
	add	x0, x0, #1
	b	_g
$ clang --target=x86_64-linux-gnu -O2 -S -masm=intel -fno-asynchronous-unwind-tables -o - tail.c | grep -v -E '^[[:space:]]*([.#;/]|$)'
leaf:                                   # @leaf
	lea	rax, [rdi + 2*rdi]
	inc	rax
	ret
not_tail:                               # @not_tail
	push	rax
	call	g@PLT
	inc	rax
	pop	rcx
	ret
tail:                                   # @tail
	inc	rdi
	jmp	g@PLT                           # TAILCALL
$ clang-23 --target=riscv64-linux-gnu -march=rv64gc -mabi=lp64d -O2 -S -fno-asynchronous-unwind-tables -o - tail.c | grep -v -E '^[[:space:]]*([.#;/]|$)'
leaf:                                   # @leaf
	slli	a1, a0, 1
	add	a0, a0, a1
	addi	a0, a0, 1
	ret
not_tail:                               # @not_tail
	addi	sp, sp, -16
	sd	ra, 8(sp)                       # 8-byte Folded Spill
	call	g
	addi	a0, a0, 1
	ld	ra, 8(sp)                       # 8-byte Folded Reload
	addi	sp, sp, 16
	ret
tail:                                   # @tail
	addi	a0, a0, 1
	tail	g
# AArch64 at -O0: only the tail function, from its label to its ret
$ clang --target=arm64-apple-macos -O0 -S -fno-asynchronous-unwind-tables -o - tail.c | grep -v -E '^[[:space:]]*([.#;/]|$)' | sed -n '/^_tail:/,/ret$/p'
_tail:                                  ; @tail
	sub	sp, sp, #32
	stp	x29, x30, [sp, #16]             ; 16-byte Folded Spill
	add	x29, sp, #16
	str	x0, [sp, #8]
	ldr	x8, [sp, #8]
	add	x0, x8, #1
	bl	_g
	ldp	x29, x30, [sp, #16]             ; 16-byte Folded Reload
	add	sp, sp, #32
	ret

The -O2 output, function by function:

leaf is a leaf functionleaf function: A function that makes no calls of its own. Nothing overwrites its link register while it runs, so it needs no save of x30 or ra, and at -O2 compilers often give it no stack space at all.. It makes no calls, so nothing overwrites x30 or ra while it runs, and the compiler gave it no stack at all on any ISA. not_tail makes a call and has work left after it, so on AArch64 and RISC-V it saves the link register exactly as greet_fixed did. On x86-64 its return address is already on the stack, and the one push rax saves nothing that matters: it moves rsp by 8 so that rsp is 16-byte aligned at the call, which the System V ABI requires (alignment has rules of its own). pop rcx takes the 8 bytes back without touching the result in rax.

tail ends with its call, so the compiler replaced the call and the return with a plain jump: b, jmp or RISC-V’s tail pseudo-instruction, which the assembly manual defines as auipc with jalr zero, a jump that writes no link. clang even marks the x86-64 one # TAILCALL. The link register, or the top word of the x86-64 stack, still holds the return address of whoever called tail, so g returns straight there. This is a tail calltail call: A call that is the last thing a function does, compiled as a plain jump (b, jmp or RISC-V's tail) instead of a call and a return. The caller's return address stays in place, so the callee returns straight to the caller's caller.. Step through it with main as the caller:

At -O0 the same tail keeps bl _g inside a 32-byte frame that saves x29 and x30, so the jump is an optimisation, not a rule of the language. GCC documents its switch for it, -foptimize-sibling-calls, as enabled at -O2, -O3 and -Os, and clang accepts the same flag.

A call reaches only so far, and the linker bridges the rest

A direct call does not hold its target’s address. It holds how far away the target is, counted from the call, and the number of bits it has for that count sets its reachbranch range: How far from itself a PC-relative branch can reach, set by the width of its offset field: 4 KiB either way for a RISC-V branch, 32 KiB for AArch64 tbz, about 2 GiB for an x86-64 near jump. More in “Jumps and conditional branches”. The fields, from each ISA’s documents:

Conditional branches reach less, which jumps and conditional branches works through. For calls, the two ISAs with short reaches take opposite routes, and both run through the linkerlinker: The tool that merges one or more object files into one executable or shared library. It concatenates sections, picks final addresses, resolves symbols and patches the bytes the assembler marked with relocations. More in “From source to executable: objects and linking”, which is the first tool that knows the final addresses.

RISC-V starts long and lets the linker shorten. The assembler cannot know whether a call’s target will end up within 1 MiB, so call begins as the 8-byte pair. Assembled with relaxation allowed, each auipc carries two relocationsrelocation: An entry beside a section's bytes that says which bytes the linker must patch, with which symbol's address, under which formula. Needed whenever the assembler cannot know a final address yet. More in “From source to executable: objects and linking”: R_RISCV_CALL_PLT asks the linker to fill in the distance, and R_RISCV_RELAX allows it to rewrite the sequence. The next session links three such calls with LLD, LLVM’s linker, and lists the program it wrote; the program is linked and listed, not run:

# Three calls for the linker to shorten. Assembled with relaxation allowed, each call stays an
# auipc and a jalr with two relocations; the linker may turn the pair into one jal when it reaches.
# The program is linked and listed.
.text
.globl _start
_start:
    call twice                  # 8 bytes in the object file
    li a7, 93                   # the Linux exit system call
    ecall

.globl twice
twice:                          # long twice(long x): helper(helper(x))
    addi sp, sp, -16
    sd ra, 8(sp)
    call helper
    call helper
    ld ra, 8(sp)
    addi sp, sp, 16
    ret

helper:                         # long helper(long x): x + 1
    addi a0, a0, 1
    ret

.section .note.GNU-stack,"",@progbits

"""Assembles three RISC-V calls with relaxation allowed, links them with LLD, LLVM's linker, and lists the
object file and the linked program: each call before and after the linker shortened it."""

import pathlib
import sys

sys.path.insert(0, str(pathlib.Path(__file__).parent / "sessions"))
import session  # noqa: E402

session.run("""
# Assemble with relaxation allowed (-mattr=+relax), then list the object with its relocations (-r)
llvm-mc --triple=riscv64-linux-gnu -mattr=+relax -filetype=obj -o relax.o relax_riscv64.s
llvm-objdump -dr -M no-aliases relax.o
# Link it into a RISC-V Linux program and list that
ld.lld -o relax relax.o
llvm-objdump -d -M no-aliases relax
# Bytes of code before and after the link
llvm-size relax.o relax
""")

# Assemble with relaxation allowed (-mattr=+relax), then list the object with its relocations (-r)
$ llvm-mc --triple=riscv64-linux-gnu -mattr=+relax -filetype=obj -o relax.o relax_riscv64.s
$ llvm-objdump -dr -M no-aliases relax.o

relax.o:	file format elf64-littleriscv

Disassembly of section .text:

0000000000000000 <_start>:
       0: 00000097     	auipc	ra, 0x0
		0000000000000000:  R_RISCV_CALL_PLT	twice
		0000000000000000:  R_RISCV_RELAX	*ABS*
       4: 000080e7     	jalr	ra, 0x0(ra) <_start>
       8: 05d00893     	addi	a7, zero, 0x5d
       c: 00000073     	ecall

0000000000000010 <twice>:
      10: ff010113     	addi	sp, sp, -0x10
      14: 00113423     	sd	ra, 0x8(sp)
      18: 00000097     	auipc	ra, 0x0
		0000000000000018:  R_RISCV_CALL_PLT	helper
		0000000000000018:  R_RISCV_RELAX	*ABS*
      1c: 000080e7     	jalr	ra, 0x0(ra) <twice+0x8>
      20: 00000097     	auipc	ra, 0x0
		0000000000000020:  R_RISCV_CALL_PLT	helper
		0000000000000020:  R_RISCV_RELAX	*ABS*
      24: 000080e7     	jalr	ra, 0x0(ra) <twice+0x10>
      28: 00813083     	ld	ra, 0x8(sp)
      2c: 01010113     	addi	sp, sp, 0x10
      30: 00008067     	jalr	zero, 0x0(ra)

0000000000000034 <helper>:
      34: 00150513     	addi	a0, a0, 0x1
      38: 00008067     	jalr	zero, 0x0(ra)
# Link it into a RISC-V Linux program and list that
$ ld.lld -o relax relax.o
$ llvm-objdump -d -M no-aliases relax

relax:	file format elf64-littleriscv

Disassembly of section .text:

0000000000011120 <_start>:
   11120: 00c000ef     	jal	ra, 0x1112c <twice>
   11124: 05d00893     	addi	a7, zero, 0x5d
   11128: 00000073     	ecall

000000000001112c <twice>:
   1112c: ff010113     	addi	sp, sp, -0x10
   11130: 00113423     	sd	ra, 0x8(sp)
   11134: 014000ef     	jal	ra, 0x11148 <helper>
   11138: 010000ef     	jal	ra, 0x11148 <helper>
   1113c: 00813083     	ld	ra, 0x8(sp)
   11140: 01010113     	addi	sp, sp, 0x10
   11144: 00008067     	jalr	zero, 0x0(ra)

0000000000011148 <helper>:
   11148: 00150513     	addi	a0, a0, 0x1
   1114c: 00008067     	jalr	zero, 0x0(ra)
# Bytes of code before and after the link
$ llvm-size relax.o relax
   text	   data	    bss	    dec	    hex	filename
     60	      0	      0	     60	     3c	relax.o
     48	      0	      0	     48	     30	relax

Each of the three calls became one jal ra, and the code shrank from 60 bytes to 48. This is linker relaxationlinker relaxation: A link-time rewrite that replaces a code sequence with a shorter one once the final addresses are known, such as a RISC-V auipc and jalr pair turned into one jal when the target is within about 1 MiB. An R_RISCV_RELAX relocation marks each place where it is allowed.: the RISC-V ELF psABI lets a linker replace an auipc and jalr pair with one jal when the target is within about 1 MiB. Removing bytes moves every instruction after them, so the psABI asks compilers to keep a relocation for each address the move could change.

AArch64 starts short and lets the linker lengthen. far_aarch64.s makes one call to a function placed just behind it and one to far_fn, which the linker script far.ld puts at 0x20000000:

// Two calls: near_fn sits right behind _start, far_fn in a section the linker script far.ld places
// about 510 MiB away, beyond the 128 MiB a bl can reach. The program is linked and listed.
    .text
    .globl _start
_start:
    bl near_fn
    bl far_fn
    mov x8, #93                 // Linux exit
    svc #0

    .type near_fn, %function
near_fn:
    ret

    .section .farcode, "ax"
    .globl far_fn
    .type far_fn, %function
far_fn:
    ret

    .section .note.GNU-stack,"",%progbits

/* Puts .text at 0x210000 and .farcode at 0x20000000 (512 MiB), each in its own loadable segment,
   so the file holds the two pieces of code and not the half a gigabyte between them. */
PHDRS { near PT_LOAD; far PT_LOAD; }
SECTIONS {
  . = 0x210000;
  .text : { *(.text) } :near
  . = 0x20000000;
  .farcode : { *(.farcode) } :far
}

Linked with that script by the same LLD, and listed:

"""Assembles two AArch64 calls, one near and one about 510 MiB away, links them with LLD and the linker
script far.ld, and lists the program: what the linker put between the far bl and its target."""

import pathlib
import sys

sys.path.insert(0, str(pathlib.Path(__file__).parent / "sessions"))
import session  # noqa: E402

session.run("""
# Assemble for AArch64 Linux (ELF), then link with far.ld, which places far_fn at 0x20000000
llvm-mc --triple=aarch64-linux-gnu -filetype=obj -o far.o far_aarch64.s
ld.lld -T far.ld -o far far.o
# List the linked program
llvm-objdump -d far
""")

# Assemble for AArch64 Linux (ELF), then link with far.ld, which places far_fn at 0x20000000
$ llvm-mc --triple=aarch64-linux-gnu -filetype=obj -o far.o far_aarch64.s
$ ld.lld -T far.ld -o far far.o
# List the linked program
$ llvm-objdump -d far

far:	file format elf64-littleaarch64

Disassembly of section .text:

0000000000210000 <_start>:
  210000: 94000004     	bl	0x210010 <near_fn>
  210004: 94000005     	bl	0x210018 <__AArch64AbsLongThunk_far_fn>
  210008: d2800ba8     	mov	x8, #0x5d               // =93
  21000c: d4000001     	svc	#0

0000000000210010 <near_fn>:
  210010: d65f03c0     	ret
  210014: 00000000     	udf	#0x0

0000000000210018 <__AArch64AbsLongThunk_far_fn>:
  210018: 58000050     	ldr	x16, 0x210020 <__AArch64AbsLongThunk_far_fn+0x8>
  21001c: d61f0200     	br	x16
  210020: 00 00 00 20  	.word	0x20000000
  210024: 00 00 00 00  	.word	0x00000000

Disassembly of section .farcode:

0000000020000000 <far_fn>:
20000000: d65f03c0     	ret

The second bl sits at 0x210004, so far_fn is 0x20000000 - 0x210004 = 534,708,220 bytes away, about 510 MiB and far past what a bl can encode. LLD did not refuse the call. It aimed the bl at a 16-byte veneerveneer: A short sequence of instructions that a linker inserts so that a branch can reach a target beyond its range. On AArch64 a veneer may change the flags and the registers x16 and x17. More in “Jumps and conditional branches” of its own, __AArch64AbsLongThunk_far_fn, which loads the full 64-bit address of far_fn from the literal behind it into x16 and jumps there with br x16. The near call needed nothing.

That is why AAPCS64 hands x16 and x17, named IP0 and IP1, to the linker. A veneer may change those two registers and the flags and must keep every other register, so code may never expect a value in x16 or x17 to survive a bl or b that has a long-branch relocation. Arm’s ELF specification allows veneers only for those relocations, so a far conditional branch remains a link error until it is rewritten.

Practise

Three changes to this lesson’s files, each built with the commands of its session above; tasks 1 and 2 need macOS on Apple silicon, as the runs did. Make the prediction before each run, then let the run tell you whether it held.

1. Write apply(fn, x) in AArch64 assembly in a file apply_aarch64.s. It returns fn(fn(x)), and the driver below tests it with an increment and a doubling callback. apply makes calls, so its first instruction saves x29 and x30 in a 32-byte slot, which leaves 16 bytes for a value of your own. fn arrives in x0 and x in x1. Store fn in your slot, since fn itself may change any of x0 to x17, move x into x0, and call through the pointer with blr. The first result is already in x0, so load fn back from the slot and blr again. Restore x29 and x30 and ret. The driver should print 42 and 20.
2. Copy greet_aarch64.s, delete the stp, mov x29, sp and ldp lines of greet_fixed, and run ./greet fixed. Predict the output first, and the offset x30 holds when the copy’s ret runs.
3. Rebuild tail.c for x86-64 and RISC-V at -O0, then for AArch64 at -O2 with -fno-optimize-sibling-calls. Before you look, say which of the three functions change and what the tail call turns into.

/* A test driver for apply(fn, x), which returns fn(fn(x)). Write apply in AArch64 assembly in
   apply_aarch64.s, then build and run: clang -o apply apply_main.c apply_aarch64.s && ./apply */
#include <stdio.h>

long apply(long (*fn)(long), long x);

static long inc(long x) { return x + 1; }
static long twice(long x) { return 2 * x; }

int main(void) {
    printf("apply(inc, 40) = %ld (expected 42)\n", apply(inc, 40));
    printf("apply(twice, 5) = %ld (expected 20)\n", apply(twice, 5));
    return 0;
}

Interview questions

Where do x86-64 and AArch64 keep the return address after a call, and what does that change for a leaf function?

x86-64’s call pushes the return address onto the stack, so at a function’s first instruction it is the word at [rsp], and ret pops it into rip. AArch64’s bl writes it into x30, the link register, and ret branches to x30; RISC-V’s jal does the same with ra. For a leaf, which calls nothing, the register version means no memory traffic for the return at all: leaf() at -O2 on AArch64 and RISC-V had no stack instructions, only its arithmetic and ret. An x86-64 leaf always has its return address in memory, because the call put it there, yet it needs no save either, since nothing else will overwrite that slot while the leaf runs. The difference shows once a function makes calls: then AArch64 and RISC-V code must save x30 or ra first, while x86-64 code must leave rsp as it found it.

The next two take the same mechanics into a debugging session: a routine whose new exit path never comes back, and a crash report that seems to skip a function.

Key takeaways

• A call writes the address of the instruction after it and jumps: x86-64’s call pushes it onto the stack, AArch64’s bl and blr write it to x30, and RISC-V’s jal and call write it to ra, and ret returns through the same place.
• A function that calls another must store x30 or ra in its own stack slot before the first call and load it back before ret; greet_broken skipped that, so its second bl left x30 pointing at its own ret and it never returned.
• x86-64 calls cannot overwrite a caller’s return address, because each one pushes its own, but ret pops whatever word is on top, so every push and every sub rsp has to be undone before it.
• At -O2 the compilers gave the leaf no stack, saved the link register only in the function with work after its call, and turned the call that ended tail into a plain jump: b, jmp or tail.
• A call reaches as far as its offset field allows; LLD relaxed three near RISC-V call pairs into single jal instructions and sent a bl to a target about 510 MiB away through a veneer that uses x16.