Scalar floating-point arithmetic

A C program multiplies 0.1 by 10, subtracts 1, and expects zero. Written as fmul then fadd on Apple silicon, the recorded answer is 0; written as one fmadd, it is 5.5511151231257827e-17, ten times the amount by which the stored 0.1 exceeds a tenth. The difference is one rounding, and it depends on which scalar floating-point instruction the code uses.

Scalar floating-pointA floating-point value, held in one register, that an instruction acts on by itself; the opposite of a vector operation that works lanes at a time. arithmetic works on one value at a time. On x86-64 and AArch64 a double sits in the low lane of a vector register (SSE2’s xmm, Advanced SIMD’s v); RISC-V’s F and D extensions give it a register file of its own. Below, you add, multiply and square-root doubles on all three, pass them through three calling conventions, pick a rounding rule when a double becomes an integer, and avoid a classic trap: b.lt after fcmp is true for a NaN. The lesson builds on Floating-point numbers bit by bit, which lays out the bits of a double.

Scalar floating point lives in its own register file

Each ISA keeps doubles apart from its integer registers. x86-64 names the SSE registers xmm0..xmm15 and treats a double as the low 64 bits of one of them, with addsd, subsd, mulsd, divsd and sqrtsd acting on that single lane. AArch64 names its 128-bit vector registers v0..v31, and when you spell them d0..d31 you mean the low 64 bits for a double (or s0..s31 for a float). The scalar form of the arithmetic is fadd, fsub, fmul, fdiv and fsqrt. RISC-V has its own file f0..f31, with the LP64D calling convention naming them ft, fs and fa for temporary, callee-saved and argument slots; the D extension spells scalar doubles fadd.d, fsub.d, fmul.d, fdiv.d and fsqrt.d. SSE2, which holds addsd, belongs to the baseline level of the x86-64 psABI, AAPCS64 passes floating-point arguments in v0 to v7, and RISC-V adds its file with the F and D ISA extensionsISA extension: A named group of instructions that a processor may implement on top of its base ISA, such as POPCNT and BMI2 on x86-64, CSSC on AArch64 or Zbb on RISC-V. Code that uses one runs only on processors that have it. More in “Bit-manipulation instructions and extensions”.

The three ISAs beside each other:

A single function shows this on the host: computing the distance from the origin, sqrt(x*x + y*y), from two doubles. The AArch64 version multiplies, folds the second product in with one fmadd, then takes the root:

// Scalar floating-point helpers, assembled by Apple clang.
// Mach-O symbols carry a leading underscore. AAPCS64 hands doubles in d0..d7
// and returns a double in d0 or an integer in x0; d8..d15, the low 64 bits of
// v8..v15, are callee-saved. Every helper is a leaf, so no stack frame is needed.
        .text
        .p2align 2

// 1. Distance between (x, y) and the origin.
//    sqrt(x*x + y*y). The product x*x is rounded by fmul, then fmadd adds
//    y*y and rounds once more (not twice), then fsqrt rounds the root.
        .globl _hypot2
_hypot2:
        fmul    d0, d0, d0
        fmadd   d0, d1, d1, d0
        fsqrt   d0, d0
        ret

// 2. Fused and unfused versions of 0.1*10 - 1, to show one rounding vs two.
        .globl _unfused_mla
_unfused_mla:                       // d0 * d1 + d2 with two separate roundings
        fmul    d0, d0, d1
        fadd    d0, d0, d2
        ret
        .globl _fused_mla
_fused_mla:                         // d2 + d0*d1 with one rounding
        fmadd   d0, d0, d1, d2
        ret

// 3. The five scalar double-to-integer converters.
//    AArch64 carries the rounding rule in the opcode itself; see the
//    FCVTZS / FCVTNS / FCVTAS / FCVTMS / FCVTPS pages of DDI 0602.
        .globl _trunc_toward_zero
_trunc_toward_zero: fcvtzs x0, d0  // round toward zero (truncate)
        ret
        .globl _round_to_nearest_even
_round_to_nearest_even: fcvtns x0, d0  // round to nearest, ties to even
        ret
        .globl _round_away_from_zero
_round_away_from_zero: fcvtas x0, d0  // round to nearest, ties away
        ret
        .globl _round_toward_minus
_round_toward_minus: fcvtms x0, d0  // round toward -infinity (floor)
        ret
        .globl _round_toward_plus
_round_toward_plus: fcvtps x0, d0  // round toward +infinity (ceiling)
        ret

// 4. Celsius to Fahrenheit with ONE rounding. F = 1.8 * C + 32.
//    1.8 is not an 8-bit fmov immediate, so load the two constants from a
//    read-only pool with adrp, add and ldp (PC-relative, so position-independent).
        .globl _celsius_to_fahrenheit
_celsius_to_fahrenheit:             // d0 = celsius; returns fahrenheit in d0
        adrp    x9, _ctof_consts@PAGE
        add     x9, x9, _ctof_consts@PAGEOFF
        ldp     d1, d2, [x9]        // d1 = 1.8, d2 = 32.0
        fmadd   d0, d0, d1, d2      // d0 = d0*1.8 + 32, one rounding
        ret

        .section __TEXT,__const
        .p2align 3
_ctof_consts:
        .quad   0x3FFCCCCCCCCCCCCD  // 1.8 (binary64)
        .quad   0x4040000000000000  // 32.0 (binary64)
        .text

// 5. Banker's rounding: round half to even into an integer.
        .globl _round_half_even
_round_half_even:                   // d0 = value; returns rounded integer in x0
        fcvtns  x0, d0              // one instruction: round to nearest, ties even
        ret

// 6. Comparisons for the NaN-safe branch demo.
//    fcmp writes NZCV. For an ordered less-than the right flag test is `mi`
//    (N == 1); the signed-integer condition `lt` (N != V) is also true when
//    NZCV = 0011 (unordered), which is a classic trap.
        .globl _less_mi
_less_mi:                            // returns 1 if d0 < d1 under "mi"
        fcmp    d0, d1
        cset    w0, mi
        ret
        .globl _less_lt
_less_lt:                            // returns 1 if the integer `lt` condition fires
        fcmp    d0, d1
        cset    w0, lt
        ret
        .globl _nzcv_bits
_nzcv_bits:                          // returns the four NZCV bits (bits 3..0) after fcmp
        fcmp    d0, d1
        mrs     x0, nzcv
        lsr     x0, x0, #28
        ret

// C driver for fp.s: calls each helper on fixed inputs and prints one small
// table per helper, so every result can be compared with the rule behind it.
#include <math.h>
#include <stdint.h>
#include <stdio.h>

double hypot2(double x, double y);
double unfused_mla(double x, double y, double z);
double fused_mla(double x, double y, double z);

int64_t trunc_toward_zero(double);
int64_t round_to_nearest_even(double);
int64_t round_away_from_zero(double);
int64_t round_toward_minus(double);
int64_t round_toward_plus(double);

double celsius_to_fahrenheit(double);
int64_t round_half_even(double);

int less_mi(double, double);
int less_lt(double, double);
unsigned nzcv_bits(double, double);

static const char *nzcv_text(unsigned v) {
    static char buf[8];
    int i = 0;
    buf[i++] = (v & 8) ? 'N' : '-';
    buf[i++] = (v & 4) ? 'Z' : '-';
    buf[i++] = (v & 2) ? 'C' : '-';
    buf[i++] = (v & 1) ? 'V' : '-';
    buf[i] = 0;
    return buf;
}

int main(void) {
    puts("[hypot2]");
    printf("  hypot2(3, 4)   = %.17g\n", hypot2(3.0, 4.0));
    printf("  hypot2(1, 1)   = %.17g\n", hypot2(1.0, 1.0));

    puts("[fused vs unfused]");
    printf("  (0.1 * 10) - 1 unfused = %.17g\n", unfused_mla(0.1, 10.0, -1.0));
    printf("  (0.1 * 10) - 1 fused   = %.17g\n", fused_mla(0.1, 10.0, -1.0));

    puts("[converters: value, zs, ns, as, ms, ps]");
    double table[] = {2.5, 3.5, -2.5, -2.7, 0.5, 1e20, -1e20, NAN, INFINITY};
    for (unsigned i = 0; i < sizeof table / sizeof table[0]; i++) {
        double v = table[i];
        printf("  %7g  %20lld %20lld %20lld %20lld %20lld\n", v,
               (long long)trunc_toward_zero(v),
               (long long)round_to_nearest_even(v),
               (long long)round_away_from_zero(v),
               (long long)round_toward_minus(v),
               (long long)round_toward_plus(v));
    }

    puts("[celsius -> fahrenheit, fmadd]");
    double temps[] = {0.0, 100.0, -40.0, 37.0};
    for (unsigned i = 0; i < sizeof temps / sizeof temps[0]; i++)
        printf("  %6.1f C -> %6.1f F\n", temps[i], celsius_to_fahrenheit(temps[i]));

    puts("[round_half_even]");
    double halves[] = {0.5, 1.5, 2.5, -0.5, -1.5, -2.5, 2.3, 2.7};
    for (unsigned i = 0; i < sizeof halves / sizeof halves[0]; i++)
        printf("  round_half_even(%4.1f) = %lld\n", halves[i], (long long)round_half_even(halves[i]));

    puts("[fcmp and NaN]");
    printf("  less(1, 2)     : mi=%d  lt=%d  NZCV=%s\n", less_mi(1, 2), less_lt(1, 2), nzcv_text(nzcv_bits(1, 2)));
    printf("  less(2, 1)     : mi=%d  lt=%d  NZCV=%s\n", less_mi(2, 1), less_lt(2, 1), nzcv_text(nzcv_bits(2, 1)));
    printf("  less(1, 1)     : mi=%d  lt=%d  NZCV=%s\n", less_mi(1, 1), less_lt(1, 1), nzcv_text(nzcv_bits(1, 1)));
    printf("  less(NaN, 1)   : mi=%d  lt=%d  NZCV=%s\n", less_mi(NAN, 1), less_lt(NAN, 1), nzcv_text(nzcv_bits(NAN, 1)));
    return 0;
}

Assembled, linked with a small C driver and run on Apple silicon:

"""Builds fp.s with its C driver main.c for AArch64 and runs it: the distance
function, fused and unfused multiply-add, the five converters, Celsius to
Fahrenheit, round half to even, and the fcmp flags for ordered pairs and a NaN."""
import pathlib
import sys

sys.path.insert(0, str(pathlib.Path(__file__).parent / "sessions"))
import session

session.run("""
clang -O1 -c -o fp/fp.o fp/fp.s
clang -O1 -o fp/fp fp/main.c fp/fp.o
./fp/fp; echo "exit=$?"
""")

$ clang -O1 -c -o fp/fp.o fp/fp.s
$ clang -O1 -o fp/fp fp/main.c fp/fp.o
$ ./fp/fp; echo "exit=$?"
[hypot2]
  hypot2(3, 4)   = 5
  hypot2(1, 1)   = 1.4142135623730951
[fused vs unfused]
  (0.1 * 10) - 1 unfused = 0
  (0.1 * 10) - 1 fused   = 5.5511151231257827e-17
[converters: value, zs, ns, as, ms, ps]
      2.5                     2                    2                    3                    2                    3
      3.5                     3                    4                    4                    3                    4
     -2.5                    -2                   -2                   -3                   -3                   -2
     -2.7                    -2                   -3                   -3                   -3                   -2
      0.5                     0                    0                    1                    0                    1
    1e+20   9223372036854775807  9223372036854775807  9223372036854775807  9223372036854775807  9223372036854775807
   -1e+20  -9223372036854775808 -9223372036854775808 -9223372036854775808 -9223372036854775808 -9223372036854775808
      nan                     0                    0                    0                    0                    0
      inf   9223372036854775807  9223372036854775807  9223372036854775807  9223372036854775807  9223372036854775807
[celsius -> fahrenheit, fmadd]
     0.0 C ->   32.0 F
   100.0 C ->  212.0 F
   -40.0 C ->  -40.0 F
    37.0 C ->   98.6 F
[round_half_even]
  round_half_even( 0.5) = 0
  round_half_even( 1.5) = 2
  round_half_even( 2.5) = 2
  round_half_even(-0.5) = 0
  round_half_even(-1.5) = -2
  round_half_even(-2.5) = -2
  round_half_even( 2.3) = 2
  round_half_even( 2.7) = 3
[fcmp and NaN]
  less(1, 2)     : mi=1  lt=1  NZCV=N---
  less(2, 1)     : mi=0  lt=0  NZCV=--C-
  less(1, 1)     : mi=0  lt=0  NZCV=-ZC-
  less(NaN, 1)   : mi=0  lt=1  NZCV=--CV
exit=0

The run prints hypot2(3, 4) = 5 and hypot2(1, 1) = 1.4142135623730951, the two numbers you would expect. Assembled for Linux x86-64 and RISC-V and read back, the same function is four instructions before ret on x86-64, where SSE2 has no fused multiply-add, and three on RISC-V. These files are only assembled, never run:

"""Assembles the x86-64 (NASM, SSE2 scalar) and RISC-V D versions into Linux
ELF object files and reads the instructions back with llvm-objdump, to set
beside the AArch64 run of fp_session.py. The objects are only assembled,
never run."""
import pathlib
import sys

sys.path.insert(0, str(pathlib.Path(__file__).parent / "sessions"))
import session

session.run("""
# x86-64, SSE2 scalar floating point (Linux System V object).
nasm -f elf64 -o cross/x86.o cross/x86.asm
llvm-objdump -d -M intel --no-show-raw-insn cross/x86.o | awk '/^[0-9a-f]+ </{p=1} p'
# RISC-V D-extension (Linux LP64D object).
llvm-mc --triple=riscv64-linux-gnu -mattr=+d -filetype=obj -o cross/rv.o cross/rv.s
llvm-objdump -d --mattr=+d --no-show-raw-insn cross/rv.o | awk '/^[0-9a-f]+ </{p=1} p'
""", max_lines=80)

# x86-64, SSE2 scalar floating point (Linux System V object).
$ nasm -f elf64 -o cross/x86.o cross/x86.asm
$ llvm-objdump -d -M intel --no-show-raw-insn cross/x86.o | awk '/^[0-9a-f]+ </{p=1} p'
0000000000000000 <hypot2>:
       0:      	mulsd	xmm0, xmm0
       4:      	mulsd	xmm1, xmm1
       8:      	addsd	xmm0, xmm1
       c:      	sqrtsd	xmm0, xmm0
      10:      	ret

0000000000000011 <unfused_mla>:
      11:      	mulsd	xmm0, xmm1
      15:      	addsd	xmm0, xmm2
      19:      	ret

000000000000001a <trunc_toward_zero>:
      1a:      	cvttsd2si	rax, xmm0
      1f:      	ret

0000000000000020 <round_mxcsr>:
      20:      	cvtsd2si	rax, xmm0
      25:      	ret
# RISC-V D-extension (Linux LP64D object).
$ llvm-mc --triple=riscv64-linux-gnu -mattr=+d -filetype=obj -o cross/rv.o cross/rv.s
$ llvm-objdump -d --mattr=+d --no-show-raw-insn cross/rv.o | awk '/^[0-9a-f]+ </{p=1} p'
0000000000000000 <hypot2>:
       0:      	fmul.d	fa0, fa0, fa0
       4:      	fmadd.d	fa0, fa1, fa1, fa0
       8:      	fsqrt.d	fa0, fa0
       c:      	ret

0000000000000010 <trunc_toward_zero>:
      10:      	fcvt.l.d	a0, fa0, rtz
      14:      	ret

0000000000000018 <round_to_nearest_even>:
      18:      	fcvt.l.d	a0, fa0, rne
      1c:      	ret

0000000000000020 <less_quiet>:
      20:      	flt.d	a0, fa0, fa1
      24:      	ret

fmadd rounds once; fmul then fadd rounds twice

A fused multiply-addfused multiply-add: A floating-point instruction that computes a * b + c with a single rounding at the end, so the add sees the exact product. AArch64 names it fmadd, x86-64 FMA3 vfmadd231sd (with its 132 and 213 forms) and the RISC-V D extension fmadd.d. is one instruction that computes a * b + c with a single rounding at the end: the product a * b is held in a wider internal form and the add runs on it, so no bit is dropped until the final round. On AArch64 that instruction is fmadd, one of the base floating-point instructions. x86-64 names it vfmadd231sd (with 132 and 213 forms for the other operand orders), part of FMA3, which the x86-64-v3 micro-architecture level adds; a plain SSE2 build cannot use it. RISC-V’s D extension includes fmadd.d, so every core with D has it.

Why does the number of roundings matter? Watch the fp run again:

The input 0.1 is already a hair above a tenth, so 0.1 * 10 is really 1 + ε for a very small ε. When fmul runs first, it rounds its own result to the nearest double, which is 1.0 exactly: ε is gone. The following fadd then subtracts 1 from 1.0, and the recorded answer is 0. When fmadd runs instead, the exact product survives into the add, the add subtracts 1, and the final rounding turns the real answer, about 5.55 × 10⁻¹⁷, into the nearest double, which prints as 5.5511151231257827e-17.

Passing and returning doubles under three ABIs

An ABI says, among many things, which register carries each argument of each type, and which register the result comes back in. A floating-point registerfloating-point register: A register that holds a float or a double for the floating-point unit, separate from the integer register file. x86-64 names them xmm0..xmm15 (SSE); AArch64 names 128-bit v0..v31 with d0..d31 and s0..s31 as 64-bit and 32-bit views; RISC-V names f0..f31 as ft, fs and fa under the LP64D calling convention. plays that role for doubles. The smallest signature that pulls the three ABIs apart is double scale(int64_t n, double x, double y), one integer followed by two doubles. The compiler writes assembly for each target without running it, for the two x86-64 ABIs and for AAPCS64, the host’s own:

// A mixed prototype: one integer argument, two doubles, one double result.
// This is the smallest signature that forces every ABI to say where it puts
// an integer, where it puts a floating-point value and in which order. The
// result is simply n * x + y, computed once; the lesson looks at where each
// argument lives, not at the arithmetic.
#include <stdint.h>

double scale(int64_t n, double x, double y) {
    return (double)n * x + y;
}

"""Shows where three calling conventions place the arguments of
`double scale(int64_t n, double x, double y)`: System V (Linux x86-64),
Microsoft x64 (Windows) and AAPCS64 (AArch64, built for macOS). The compiler
only writes assembly for each target, and the register names can be read in
the first few instructions."""
import pathlib
import sys

sys.path.insert(0, str(pathlib.Path(__file__).parent / "sessions"))
import session

session.run("""
# System V (Linux x86-64): integer in rdi, first double in xmm0, second in xmm1.
clang --target=x86_64-linux-gnu -ffreestanding -O2 -masm=intel -S -o - abi/scale.c | awk '/^scale:/,/^\\.Lfunc_end0:/'
# Microsoft x64 (Windows): first four slots share rcx/xmm0, rdx/xmm1, r8/xmm2, r9/xmm3.
clang --target=x86_64-pc-windows-msvc -ffreestanding -O2 -masm=intel -S -o - abi/scale.c | awk '/^scale:/{p=1} p{print; if ($1=="ret") exit}'
# AAPCS64 (AArch64): the integer in x0, the doubles in d0 and d1, counted separately.
clang --target=arm64-apple-macos -ffreestanding -O2 -S -o - abi/scale.c | awk '/^_scale:/{p=1} p{print; if ($1=="ret") exit}'
""", max_lines=40)

# System V (Linux x86-64): integer in rdi, first double in xmm0, second in xmm1.
$ clang --target=x86_64-linux-gnu -ffreestanding -O2 -masm=intel -S -o - abi/scale.c | awk '/^scale:/,/^\.Lfunc_end0:/'
scale:                                  # @scale
# %bb.0:
	cvtsi2sd	xmm2, rdi
	mulsd	xmm0, xmm2
	addsd	xmm0, xmm1
	ret
.Lfunc_end0:
# Microsoft x64 (Windows): first four slots share rcx/xmm0, rdx/xmm1, r8/xmm2, r9/xmm3.
$ clang --target=x86_64-pc-windows-msvc -ffreestanding -O2 -masm=intel -S -o - abi/scale.c | awk '/^scale:/{p=1} p{print; if ($1=="ret") exit}'
scale:                                  # @scale
# %bb.0:
	cvtsi2sd	xmm0, rcx
	mulsd	xmm0, xmm1
	addsd	xmm0, xmm2
	ret
# AAPCS64 (AArch64): the integer in x0, the doubles in d0 and d1, counted separately.
$ clang --target=arm64-apple-macos -ffreestanding -O2 -S -o - abi/scale.c | awk '/^_scale:/{p=1} p{print; if ($1=="ret") exit}'
_scale:                                 ; @scale
; %bb.0:
	scvtf	d2, x0
	fmadd	d0, d2, d0, d1
	ret

System V picks the first integer register rdi for n, and the two doubles fill xmm0 and xmm1; the integer and floating-point register sequences are independent (so a prototype with eight doubles would still use xmm0..xmm7, whatever the integers did). The recorded three lines are cvtsi2sd xmm2, rdi (convert n to a double), mulsd xmm0, xmm2 (now x meets the converted n) and addsd xmm0, xmm1 (add y). The Windows x64 ABI gives every argument a positional slot, numbered 0..3 for the first four, with slot 0 using rcx or xmm0, slot 1 using rdx or xmm1, and so on, picking integer or floating-point by the argument’s type; so n is in rcx (slot 0), x in xmm1 (slot

1. and y in xmm2 (slot 2). The three lines are cvtsi2sd xmm0, rcx, mulsd xmm0, xmm1, addsd xmm0, xmm2. On AArch64 the listing is scvtf d2, x0, which converts n, then a single fmadd d0, d2, d0, d1 for n * x + y: AAPCS64 counts integer and floating-point registers separately, as System V does, and so does RISC-V LP64D, where n, x and y arrive in a0, fa0 and fa1.

Each ABI also fixes which floating-point registers a called function must preserve. System V has none: a double in xmm is caller-saved. Microsoft x64 keeps xmm6..xmm15 nonvolatile, so a function that reaches for them must save them first. AAPCS64 keeps the low 64 bits of v8..v15 across a call: the top 64 bits may be clobbered, which matters for SIMD code. RISC-V LP64D names fs0..fs11 as callee-saved. A double result goes back in xmm0, xmm0, d0 and fa0 on the four conventions.

Converting a double to an integer, by rule

Turning a double into an integer always rounds, and the three ISAs let you choose the rule. x86-64’s cvttsd2si truncates (ignores MXCSR.RC); its sibling cvtsd2si reads MXCSR.RC, which is round to nearest, ties to even by default. AArch64 bakes the rule into the opcode: fcvtzs truncates, fcvtns rounds to nearest and ties to even, fcvtas ties away from zero, fcvtms rounds toward minus infinity, fcvtps rounds toward plus infinity, and frintn rounds in place as a double. RISC-V names a rounding moderounding mode: The rule a floating-point operation applies when the exact result falls between two representable values. IEEE 754 names five, with round to nearest, ties to even as the default. x86-64 keeps the mode in MXCSR, AArch64 in FPCR (some conversions fix it in the opcode) and RISC-V in an instruction field or the frm CSR. field inside every fcvt.l.d instruction, with the mnemonics rtz, rne, rdn, rup, rmm and dyn picking truncation, nearest even, down, up, nearest away, and the dynamic value of the frm CSR.

Even on 2.5 the five AArch64 converters give two different answers, 2 and 3, one for each rule family:

Reading the fp run row by row, 2.5 goes to 2, 2, 3, 2, 3, 3.5 to 3, 4, 4, 3, 4, -2.5 to -2, -2, -3, -3, -2 and -2.7 to -2, -3, -3, -3, -2. A C cast (long)x uses the truncation rule, which is why (long)-2.7 is -2, not -3; rounding halves away from zero, as most people do by hand, asks for fcvtas; ties to even with fcvtns avoids the bias that rounding every half away from zero adds to a long sum. For a value outside the 64-bit integer range or for a NaN, AArch64 uses a saturating conversionsaturating conversion: A float-to-integer conversion that clamps an out-of-range input to the largest or smallest integer of the destination instead of wrapping. AArch64 and RISC-V saturate (AArch64 gives 0 for a NaN, RISC-V the largest integer); x86-64 returns its integer-indefinite value, the smallest integer, instead.: the recorded run gives 9223372036854775807 for 1e20 and +inf, -9223372036854775808 for -1e20, and 0 for NaN. The two other ISAs differ:

The AArch64 column comes from the run above, where 2^63 − 1 prints as 9223372036854775807. The Intel manual names the x86-64 result the integer indefinite value, 0x8000000000000000, which is −2^63, and the RISC-V results come from the conversion table of its F and D chapters.

Comparing doubles when NaN can appear

A branch on floating point reads flags the compare instruction wrote. x86-64’s ucomisd sets ZF, PF and CF: when the pair is unordered (one operand is a NaNNaN: Not a Number: an IEEE 754 pattern with the maximum exponent and a non-zero fraction, the result of operations such as 0.0/0.0 or infinity minus infinity. A NaN is not equal to anything, including itself, so x != x detects one. More in “Floating-point numbers bit by bit”), PF = 1 and ZF = CF = 1, the ZF of an equal pair together with the CF of a smaller one. So je, jb and jbe would all branch on a NaN, while ja and jae do not; a careful sequence tests PF first with jp (jump if parity even). AArch64’s fcmp sets NZCV, the four condition flags, and the recorded run reads them one bit per row:

AArch64 has one set of conditions, and Arm’s condition table gives each of them one meaning after an integer compare and another after fcmp. After fcmp, mi (N set) means less than, while lt (N != V) means less than or unordered:

The run shows it: less(NaN, 1) is mi = 0, lt = 1. With a NaN, N is 0 and V is 1, so lt holds and b.lt branches as if NaN were less than 1, while mi, which tests N == 1, does not. Use mi for an ordered less-than (or swap the operands and test gt), or branch on vs first to handle the unordered case by itself. RISC-V makes this concrete at the register level: flt.d, fle.d and feq.d write a 0 or 1 to an integer register and give 0 for any NaN; flt and fle raise invalid for a NaN operand, feq raises it only for a signalling NaN.

Practise

Open your own copy of fp/fp.s and fp/main.c under a scratch folder. Each task edits one small place and the C driver prints what changed; keep the originals untouched so the quiz questions below still match the recorded run.

1. One rounding or two. The _celsius_to_fahrenheit helper uses fmadd to compute F = 1.8 * C + 32 with one rounding. Add -17.7 to the temps array, print the result with %.17g instead of %6.1f, then run it once as it is and once with fmul followed by fadd. Predict which rows change: the four recorded temperatures give the same result both ways, and -17.7 does not (0.14000000000000049 fused, 0.14000000000000057 unfused).
2. Banker’s rounding to the nearest even. _round_half_even is one AArch64 instruction, fcvtns. Add two more rows to the halves array in main.c — 3.5 and 4.5 — and confirm the output is 4 and 4 (ties to even rounds up for 3.5 and keeps 4 for 4.5). Then change the opcode to fcvtas, predict which rows move, and check: only the ties whose even neighbour is nearer zero change, 0.5, 2.5, 4.5, -0.5 and -2.5.
3. A NaN-safe branch. Add _unordered to fp.s that returns 1 when the pair is unordered (one is a NaN) and 0 otherwise, by testing the vs condition on fcmp’s flags; call it from main.c with (1, 2) and (NaN, 1) and check the two results are 0 and 1.
4. A NaN you choose to report. fcvtns turns a NaN into 0, which looks like a real result. Before the fcvtns in _round_half_even, add fcmp d0, d0 and a b.vs to a path that returns INT64_MIN as an error marker (only a NaN compares unordered with itself), then add 1e20, -1e20 and NAN to halves and predict their three results.

Interview questions

After fcmp d0, d1, is b.lt a safe way to branch on an ordered less-than?

No. fcmp sets NZCV, and when either operand is a NaN the result is unordered: the recorded flags are NZCV = 0011, with C and V set and N and Z clear. The AArch64 condition lt is defined as N != V because it was written for signed integer compares; with N = 0 and V = 1, lt is true, so b.lt branches as if NaN were less than the other operand. The run prints less(NaN, 1): mi=0 lt=1 for this reason. For a NaN-safe less-than, use mi (N == 1), which means an ordered less-than after fcmp; or test vs (V == 1) first and handle unordered separately. The equal case is similar: eq is Z == 1, which is false for a NaN pair, so it is already safe. The lesson’s less_mi and less_lt helpers demonstrate the gap directly.

Two harder questions take the conversions into code a team ships: a 32-bit cast on which three builds disagree, and a rounding function that a caller’s rounding mode can change. Think each through before you open it:

Key takeaways

• Each ISA keeps doubles in its own register file: SSE registers xmm0..xmm15 on x86-64, the low 64 bits of v0..v31 named d0..d31 on AArch64, and f0..f31 on RISC-V, with scalar addsd, fadd and fadd.d spelling the same operation.
• fmadd rounds once and keeps the exact product of the multiply, so error-sensitive code uses it; fmul then fadd rounds twice and the small bits are lost (0.1 * 10 - 1 is 0 unfused, 5.55 × 10⁻¹⁷ fused).
• The ABIs disagree on where the arguments of double scale(int64_t n, double x, double y) go: System V picks rdi, xmm0, xmm1; Microsoft x64 fills positional slots rcx, xmm1, xmm2; AAPCS64 fills x0, d0, d1; RISC-V LP64D fills a0, fa0, fa1.
• Converting a double to an integer always rounds, and each ISA names the rule: cvttsd2si truncates, fcvtzs, fcvtns, fcvtas, fcvtms and fcvtps bake five rules into the AArch64 opcode, and RISC-V’s fcvt.l.d carries the rule in its instruction field.
• A compare after a NaN leaves NZCV = 0011 on AArch64, so the condition lt holds for an unordered pair; use mi for an ordered less-than, or test vs first and handle unordered as its own case.