Operating Systems Module 2 – Processes: creation, life cycle, signals and IPC

Context switches and their real cost

What a context switch saves and where, how it differs from a mode switch and a thread switch, and what it costs: a pipe ping-pong, measured and counted.

Intermediate
15 minutes
Examples run with Python 3.14.8, Pyodide 314.0.7, Clang 21.0.0, GCC 16.2.0 and Rust 1.99.0
By MySmartCoPilot

Free: the whole lesson. With a pass: the model answers to its 2 interview questions.

What you will learn

List what the kernel saves and restores in a context switch, and where it keeps each part
Distinguish a mode switch, a thread switch and a process switch by what each one changes
Measure a hand-off between two threads or processes with a pipe ping-pong, as a median with its spread
Explain the costs a switch leaves behind in the caches, the TLB and the branch predictors
Read the voluntary and involuntary switch counts of a process, and say why macOS counts them differently

Before you start

The process: address space, PCB and states

What you’ll learn · Before you start

What you will learn

List what the kernel saves and restores in a context switch, and where it keeps each part
Distinguish a mode switch, a thread switch and a process switch by what each one changes
Measure a hand-off between two threads or processes with a pipe ping-pong, as a median with its spread
Explain the costs a switch leaves behind in the caches, the TLB and the branch predictors
Read the voluntary and involuntary switch counts of a process, and say why macOS counts them differently

Before you start

The process: address space, PCB and states

On this page

Handing one byte to another thread and back took a median of 102.5 microseconds on the machine behind this lesson, while writing and reading the same byte within one thread took 0.8. Nearly all of the difference is the CPU being handed over twice on every trip: one thread goes to sleep, and the kernel wakes the other and switches to it. A context switch: The kernel stopping one process and running another: it saves the first one's registers, switches to the second one's page tables and loads its registers. More in “User mode, kernel mode and traps” is that change of thread: the kernel saves the registers of the one that was running, chooses another, and loads the registers of that one.

This lesson follows one switch through the kernel, runs a model in 14 languages of what forgetting a single register does, separates mode, thread and process switches, measures a hand-off with care, looks at what a switch costs after it is over, in the caches and the TLB, and reads the counters that Linux and macOS keep for switches.

A context switch saves every register the program can see

A program never knows when it will be stopped. The timer can interrupt it between any two instructions, in the middle of a loop, with half a result in a register. So the kernel must save everything the program could be using and put all of it back before the program runs again, or the program carries on with somebody else’s values. On 64-bit Arm Linux the work is done in two saves, one at the trap and one at the switch. Step through a switch from A to B:

Interactive · message by message One context switch on 64-bit Arm Linux Step through a timer interrupt that ends A's time slice and starts B. Watch where each part of A's state is put.

The drawing shows the whole scene; its 8 steps are listed below.

Every step as text

A runs. A runs in user mode. Its registers are in the CPU, and the MMU translates its addresses with A's page tables.
The timer fires. The timer interrupt traps into the kernel. The entry code saves A's user registers, x0 to x30, the stack pointer, the program counter and the processor state, in a frame on A's kernel stack.
The scheduler picks B. A's time slice is over, so the scheduler puts A back among the ready tasks and picks B to run next.
B's page tables. B belongs to another process, so switch_mm points the MMU at B's page tables, with B's ASID. Between two threads of one process this step is skipped.
Per-thread state. __switch_to swaps what each thread owns beyond the general registers, such as the floating-point and vector registers and the register that points at thread-local storage.
The switch itself. cpu_switch_to stores the registers the kernel's own code must keep, x19 to x29, the stack pointer and the link register, in A's task_struct, and loads B's. The kernel now runs on B's kernel stack.
Back to B. On the way out, the kernel restores B's user registers from the frame saved when B was last interrupted. B continues at the instruction where it stopped.
A's state, in two places. A's user registers wait in the frame on its kernel stack and its kernel registers in its task_struct. Both come back when the scheduler next picks A.

The first save happens at the trap, before the kernel knows whether it will switch at all: the entry code, kernel_entry, stores x0 to x29, the link register, the stack pointer, the program counter and the processor state in a frame on the thread’s kernel stack: The stack a thread uses while it runs in the kernel. Linux gives every thread its own, and the registers saved when a trap enters the kernel are kept on it.. The second save, in cpu_switch_to, covers the registers that the kernel’s own code must keep across a call, x19 to x28, the frame pointer, the stack pointer and the link register, and puts them in the thread’s task_struct. Around it, __switch_to swaps the floating-point and vector registers and the thread-local storage pointer, and switch_mm changes the page tables when the next thread belongs to another process.

What happens if one register is left out? The model below runs two programs on a CPU with three registers, pc, acc and tmp: A adds 1², 2² and 3² into acc, B multiplies 1 × 2 × 3 × 4 there, and both load each number into tmp first. The CPU switches every two instructions. The first run saves all three registers at every switch; the second saves pc and acc only, as a kernel would that assumed nothing important was left in tmp:

One program · 14 languages What a switch must save, and what forgetting one register does

Python 3.14 · main.py

from collections import deque
from dataclasses import dataclass

SLICE = 2  # instructions until the timer interrupts

PROGRAMS = {
    "A": "load 1, square, add, load 2, square, add, load 3, square, add".split(", "),
    "B": "load 1, add, load 2, mul, load 3, mul, load 4, mul".split(", "),
}


@dataclass
class Saved:
    pc: int = 0
    acc: int = 0
    tmp: int = 0


def execute(step: str, acc: int, tmp: int) -> tuple[int, int]:
    if step.startswith("load "):
        return acc, int(step.removeprefix("load "))
    if step == "square":
        return acc, tmp * tmp
    if step == "add":
        return acc + tmp, tmp
    return acc * tmp, tmp


def run(save_tmp: bool) -> None:
    print("saving pc, acc and tmp:" if save_tmp else "saving pc and acc only:")
    saved = {name: Saved() for name in PROGRAMS}
    ready = deque(PROGRAMS)
    tmp = 0
    turn = 0
    while ready:
        name = ready.popleft()
        code, regs = PROGRAMS[name], saved[name]
        pc, acc = regs.pc, regs.acc  # restore the saved registers
        if save_tmp:
            tmp = regs.tmp
        ran = code[pc : pc + SLICE]
        for step in ran:
            acc, tmp = execute(step, acc, tmp)
        pc += len(ran)
        turn += 1
        print(f"  turn {turn}: {name} {', '.join(ran)} -> acc {acc}, tmp {tmp}")
        regs.pc, regs.acc = pc, acc  # save them when the timer fires
        if save_tmp:
            regs.tmp = tmp
        if pc < len(code):
            ready.append(name)
        else:
            print(f"  {name} finished with {acc}")


run(True)
run(False)

In Python: code[pc : pc + SLICE] stops early at the last instruction, so a turn runs two instructions or fewer without any bounds check.

Recorded with CPython 3.14.8 on macOS 26 arm64. It runs in your browser with Pyodide 314.0.7 (CPython 3.14.2). To run it yourself: python3 main.py

JavaScript Node 24 · main.mjs

const SLICE = 2; // instructions until the timer interrupts

const PROGRAMS = {
  A: "load 1, square, add, load 2, square, add, load 3, square, add".split(", "),
  B: "load 1, add, load 2, mul, load 3, mul, load 4, mul".split(", "),
};

const execute = (step, acc, tmp) => {
  if (step.startsWith("load ")) return [acc, Number(step.slice(5))];
  if (step === "square") return [acc, tmp * tmp];
  if (step === "add") return [acc + tmp, tmp];
  return [acc * tmp, tmp];
};

const run = (saveTmp) => {
  console.log(saveTmp ? "saving pc, acc and tmp:" : "saving pc and acc only:");
  const saved = { A: { pc: 0, acc: 0, tmp: 0 }, B: { pc: 0, acc: 0, tmp: 0 } };
  const ready = Object.keys(PROGRAMS);
  let tmp = 0;
  let turn = 0;
  for (let name = ready.shift(); name !== undefined; name = ready.shift()) {
    const code = PROGRAMS[name];
    const regs = saved[name];
    let { pc, acc } = regs; // restore the saved registers
    if (saveTmp) tmp = regs.tmp;
    const ran = code.slice(pc, pc + SLICE);
    for (const step of ran) [acc, tmp] = execute(step, acc, tmp);
    pc += ran.length;
    turn += 1;
    console.log(`  turn ${turn}: ${name} ${ran.join(", ")} -> acc ${acc}, tmp ${tmp}`);
    Object.assign(regs, { pc, acc }); // save them when the timer fires
    if (saveTmp) regs.tmp = tmp;
    if (pc < code.length) ready.push(name);
    else console.log(`  ${name} finished with ${acc}`);
  }
};

run(true);
run(false);

In JavaScript: slice also stops early at the end of a program, and the destructuring [acc, tmp] = execute(…) updates both registers at once.

Recorded with Node.js 24.21.0 LTS (V8 13.6) on macOS 26 arm64. It runs in your browser with QuickJS (quickjs-emscripten 0.32.0). To run it yourself: node main.mjs

Java 25 LTS · Main.java

import java.util.ArrayDeque;
import java.util.List;
import java.util.Map;

public class Main {
  static final int SLICE = 2; // instructions until the timer interrupts

  static final Map<String, List<String>> PROGRAMS =
      Map.of(
          "A", List.of("load 1, square, add, load 2, square, add, load 3, square, add".split(", ")),
          "B", List.of("load 1, add, load 2, mul, load 3, mul, load 4, mul".split(", ")));

  record Cpu(int acc, int tmp) {
    Cpu execute(String step) {
      if (step.startsWith("load ")) {
        return new Cpu(acc, Integer.parseInt(step.substring(5)));
      }
      return switch (step) {
        case "square" -> new Cpu(acc, tmp * tmp);
        case "add" -> new Cpu(acc + tmp, tmp);
        default -> new Cpu(acc * tmp, tmp);
      };
    }
  }

  static final class Saved {
    int pc;
    int acc;
    int tmp;
  }

  static void run(boolean saveTmp) {
    System.out.println(saveTmp ? "saving pc, acc and tmp:" : "saving pc and acc only:");
    Map<String, Saved> saved = Map.of("A", new Saved(), "B", new Saved());
    ArrayDeque<String> ready = new ArrayDeque<>(List.of("A", "B"));
    int tmp = 0;
    for (int turn = 1; !ready.isEmpty(); turn++) {
      String name = ready.removeFirst();
      List<String> code = PROGRAMS.get(name);
      Saved regs = saved.get(name);
      Cpu cpu = new Cpu(regs.acc, saveTmp ? regs.tmp : tmp); // restore the saved registers
      List<String> ran = code.subList(regs.pc, Math.min(regs.pc + SLICE, code.size()));
      for (String step : ran) {
        cpu = cpu.execute(step);
      }
      tmp = cpu.tmp();
      String did = String.join(", ", ran);
      System.out.println(
          "  turn " + turn + ": " + name + " " + did + " -> acc " + cpu.acc() + ", tmp " + tmp);
      regs.pc += ran.size(); // save them when the timer fires
      regs.acc = cpu.acc();
      if (saveTmp) {
        regs.tmp = tmp;
      }
      if (regs.pc < code.size()) {
        ready.addLast(name);
      } else {
        System.out.println("  " + name + " finished with " + cpu.acc());
      }
    }
  }

  public static void main(String[] args) {
    run(true);
    run(false);
  }
}

In Java: subList is a view of one turn, capped with Math.min at the end of the program. Cpu is a record, so execute returns new register values.

Recorded with OpenJDK 25.0.4.1 LTS (Eclipse Temurin) on macOS 26 arm64. It runs in your browser with a TeaVM 0.16.0 build in QuickJS (quickjs-emscripten 0.32.0). To run it yourself: java Main.java

C++23 · main.cpp

#include <algorithm>
#include <cstddef>
#include <deque>
#include <map>
#include <print>
#include <string>
#include <vector>

// instructions until the timer interrupts
constexpr std::size_t SLICE = 2;

const std::map<char, std::vector<std::string>> PROGRAMS = {
    {'A',
     {"load 1", "square", "add", "load 2", "square", "add", "load 3", "square",
      "add"}},
    {'B', {"load 1", "add", "load 2", "mul", "load 3", "mul", "load 4", "mul"}},
};

struct Saved {
  std::size_t pc = 0;
  int acc = 0;
  int tmp = 0;
};

void execute(const std::string &step, int &acc, int &tmp) {
  if (step.starts_with("load "))
    tmp = std::stoi(step.substr(5));
  else if (step == "square")
    tmp *= tmp;
  else if (step == "add")
    acc += tmp;
  else
    acc *= tmp;
}

void run(bool save_tmp) {
  std::println("{}", save_tmp ? "saving pc, acc and tmp:"
                              : "saving pc and acc only:");
  std::map<char, Saved> saved = {{'A', {}}, {'B', {}}};
  std::deque<char> ready = {'A', 'B'};
  int tmp = 0;
  for (int turn = 1; !ready.empty(); turn++) {
    const char name = ready.front();
    ready.pop_front();
    const auto &code = PROGRAMS.at(name);
    Saved &regs = saved[name];
    int acc = regs.acc; // restore the saved registers
    if (save_tmp)
      tmp = regs.tmp;
    const auto end = std::min(regs.pc + SLICE, code.size());
    std::string ran;
    for (auto pc = regs.pc; pc < end; pc++) {
      execute(code[pc], acc, tmp);
      ran += (pc > regs.pc ? ", " : "") + code[pc];
    }
    std::println("  turn {}: {} {} -> acc {}, tmp {}", turn, name, ran, acc,
                 tmp);
    regs.pc = end; // save them when the timer fires
    regs.acc = acc;
    if (save_tmp)
      regs.tmp = tmp;
    if (regs.pc < code.size())
      ready.push_back(name);
    else
      std::println("  {} finished with {}", name, acc);
  }
}

int main() {
  run(true);
  run(false);
}

In C++: execute takes acc and tmp by reference and changes them in place, the way an instruction changes a CPU's registers.

Recorded with Apple clang 21.0.0 (C++23) and GCC 16.2.0 on macOS 26 arm64. It runs in your browser with Clang in WebAssembly (YoWASP) 22.0.0-git20542-10. To run it yourself: clang++ -std=c++23 -O2 -Wall -Wextra -Werror -pedantic -fsanitize=address,undefined main.cpp -o main && ./main

C# 14 · .NET 10 · app.cs

const int Slice = 2; // instructions until the timer interrupts

var programs = new Dictionary<string, string[]>
{
    ["A"] = "load 1, square, add, load 2, square, add, load 3, square, add".Split(", "),
    ["B"] = "load 1, add, load 2, mul, load 3, mul, load 4, mul".Split(", "),
};

Run(true);
Run(false);

void Run(bool saveTmp)
{
    Console.WriteLine(saveTmp ? "saving pc, acc and tmp:" : "saving pc and acc only:");
    var saved = new Dictionary<string, Saved> { ["A"] = new(), ["B"] = new() };
    var ready = new Queue<string>(["A", "B"]);
    int tmp = 0;
    for (int turn = 1; ready.TryDequeue(out var name); turn++)
    {
        var code = programs[name];
        var regs = saved[name];
        int acc = regs.Acc; // restore the saved registers
        if (saveTmp)
        {
            tmp = regs.Tmp;
        }
        var ran = code[regs.Pc..Math.Min(regs.Pc + Slice, code.Length)];
        foreach (var step in ran)
        {
            (acc, tmp) = Execute(step, acc, tmp);
        }
        var did = string.Join(", ", ran);
        Console.WriteLine($"  turn {turn}: {name} {did} -> acc {acc}, tmp {tmp}");
        (regs.Pc, regs.Acc) = (regs.Pc + ran.Length, acc); // save them when the timer fires
        if (saveTmp)
        {
            regs.Tmp = tmp;
        }
        if (regs.Pc < code.Length)
        {
            ready.Enqueue(name);
        }
        else
        {
            Console.WriteLine($"  {name} finished with {acc}");
        }
    }
}

static (int Acc, int Tmp) Execute(string step, int acc, int tmp) => step switch
{
    _ when step.StartsWith("load ", StringComparison.Ordinal) =>
        (acc, int.Parse(step[5..])),
    "square" => (acc, tmp * tmp),
    "add" => (acc + tmp, tmp),
    _ => (acc * tmp, tmp),
};

sealed class Saved
{
    public int Pc { get; set; }
    public int Acc { get; set; }
    public int Tmp { get; set; }
}

In C#: The range code[start..end] copies the instructions of one turn, and the switch expression returns both registers as a tuple.

Recorded with C# 14 on .NET 10.0.12 (SDK 10.0.401) on macOS 26 arm64. It runs in your browser with the .NET runtime for WebAssembly 10.0.12. To run it yourself: dotnet run app.cs

C23 · main.c

#include <stdbool.h>
#include <stddef.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

enum { SLICE = 2 }; // instructions until the timer interrupts

static const char *const A[] = {"load 1", "square", "add",
                                "load 2", "square", "add",
                                "load 3", "square", "add"};
static const char *const B[] = {"load 1", "add", "load 2", "mul",
                                "load 3", "mul", "load 4", "mul"};
static const char *const *const CODE[] = {A, B};
static const size_t LENGTH[] = {sizeof A / sizeof A[0], sizeof B / sizeof B[0]};
static const char NAMES[] = {'A', 'B'};

typedef struct {
  size_t pc;
  int acc;
  int tmp;
} Saved;

static void execute(const char *step, int *acc, int *tmp) {
  if (strncmp(step, "load ", 5) == 0)
    *tmp = atoi(step + 5);
  else if (strcmp(step, "square") == 0)
    *tmp *= *tmp;
  else if (strcmp(step, "add") == 0)
    *acc += *tmp;
  else
    *acc *= *tmp;
}

static void run(bool save_tmp) {
  puts(save_tmp ? "saving pc, acc and tmp:" : "saving pc and acc only:");
  Saved saved[2] = {{0}};
  size_t ready[16] = {0, 1};
  size_t head = 0, tail = 2;
  int tmp = 0;
  for (int turn = 1; head < tail; turn++) {
    const size_t p = ready[head++];
    size_t pc = saved[p].pc; // restore the saved registers
    int acc = saved[p].acc;
    if (save_tmp)
      tmp = saved[p].tmp;
    printf("  turn %d: %c", turn, NAMES[p]);
    for (size_t done = 0; pc < LENGTH[p] && done < SLICE; done++, pc++) {
      execute(CODE[p][pc], &acc, &tmp);
      printf("%s %s", done > 0 ? "," : "", CODE[p][pc]);
    }
    printf(" -> acc %d, tmp %d\n", acc, tmp);
    saved[p].pc = pc; // save them when the timer fires
    saved[p].acc = acc;
    if (save_tmp)
      saved[p].tmp = tmp;
    if (pc < LENGTH[p])
      ready[tail++] = p;
    else
      printf("  %c finished with %d\n", NAMES[p], acc);
  }
}

int main(void) {
  run(true);
  run(false);
  return 0;
}

In C: The programs are arrays of strings, execute changes the two registers through pointers, and the ready queue is an array with a head and a tail.

Recorded with Apple clang 21.0.0 (C23) and GCC 16.2.0 on macOS 26 arm64. It runs in your browser with Clang in WebAssembly (YoWASP) 22.0.0-git20542-10. To run it yourself: clang -std=c23 -O2 -Wall -Wextra -Werror -pedantic -fsanitize=address,undefined main.c -o main && ./main

PHP 8.5 · main.php

<?php

declare(strict_types=1);

const SLICE = 2; // instructions until the timer interrupts

const PROGRAMS = [
    'A' => 'load 1, square, add, load 2, square, add, load 3, square, add',
    'B' => 'load 1, add, load 2, mul, load 3, mul, load 4, mul',
];

final class Saved
{
    public int $pc = 0;
    public int $acc = 0;
    public int $tmp = 0;
}

/** @return array{int, int} */
function execute(string $step, int $acc, int $tmp): array
{
    return match (true) {
        str_starts_with($step, 'load ') => [$acc, (int) substr($step, 5)],
        $step === 'square' => [$acc, $tmp * $tmp],
        $step === 'add' => [$acc + $tmp, $tmp],
        default => [$acc * $tmp, $tmp],
    };
}

function run(bool $saveTmp): void
{
    echo $saveTmp ? "saving pc, acc and tmp:\n" : "saving pc and acc only:\n";
    $saved = ['A' => new Saved(), 'B' => new Saved()];
    $ready = ['A', 'B'];
    $tmp = 0;
    for ($turn = 1; $ready !== []; $turn++) {
        $name = array_shift($ready);
        $code = explode(', ', PROGRAMS[$name]);
        $regs = $saved[$name];
        $acc = $regs->acc; // restore the saved registers
        if ($saveTmp) {
            $tmp = $regs->tmp;
        }
        $ran = array_slice($code, $regs->pc, SLICE);
        foreach ($ran as $step) {
            [$acc, $tmp] = execute($step, $acc, $tmp);
        }
        echo "  turn {$turn}: {$name} " . implode(', ', $ran) . " -> acc {$acc}, tmp {$tmp}\n";
        $regs->pc += count($ran); // save them when the timer fires
        $regs->acc = $acc;
        if ($saveTmp) {
            $regs->tmp = $tmp;
        }
        if ($regs->pc < count($code)) {
            $ready[] = $name;
        } else {
            echo "  {$name} finished with {$acc}\n";
        }
    }
}

run(true);
run(false);

In PHP: array_slice returns one turn's instructions, fewer at the end, and match (true) tries each condition in order.

Recorded with PHP 8.5.11 (CLI) on macOS 26 arm64. It runs in your browser with php-wasm 0.2.0 (PHP 8.5.2). To run it yourself: php main.php

Rust 1.99 · main.rs

use std::collections::VecDeque;

const SLICE: usize = 2; // instructions until the timer interrupts

const PROGRAMS: [(char, &str); 2] = [
    (
        'A',
        "load 1, square, add, load 2, square, add, load 3, square, add",
    ),
    ('B', "load 1, add, load 2, mul, load 3, mul, load 4, mul"),
];

#[derive(Clone, Copy, Default)]
struct Saved {
    pc: usize,
    acc: i32,
    tmp: i32,
}

fn execute(step: &str, acc: i32, tmp: i32) -> (i32, i32) {
    if let Some(n) = step.strip_prefix("load ") {
        return (acc, n.parse().unwrap_or(0));
    }
    match step {
        "square" => (acc, tmp * tmp),
        "add" => (acc + tmp, tmp),
        _ => (acc * tmp, tmp),
    }
}

fn run(header: &str, save_tmp: bool) {
    println!("{header}");
    let mut saved = [Saved::default(); 2];
    let mut ready = VecDeque::from([0, 1]);
    let mut tmp = 0;
    for turn in 1.. {
        let Some(p) = ready.pop_front() else { break };
        let (name, text) = PROGRAMS[p];
        let code: Vec<&str> = text.split(", ").collect();
        let mut acc = saved[p].acc; // restore the saved registers
        if save_tmp {
            tmp = saved[p].tmp;
        }
        let ran = &code[saved[p].pc..(saved[p].pc + SLICE).min(code.len())];
        for step in ran {
            (acc, tmp) = execute(step, acc, tmp);
        }
        let did = ran.join(", ");
        println!("  turn {turn}: {name} {did} -> acc {acc}, tmp {tmp}");
        saved[p].pc += ran.len(); // save them when the timer fires
        saved[p].acc = acc;
        if save_tmp {
            saved[p].tmp = tmp;
        }
        if saved[p].pc < code.len() {
            ready.push_back(p);
        } else {
            println!("  {name} finished with {acc}");
        }
    }
}

fn main() {
    run("saving pc, acc and tmp:", true);
    run("saving pc and acc only:", false);
}

In Rust: split turns each program's text into instructions, and a slice capped with min is one turn. The header is passed in, as in Go, so run prints it with one line.

Recorded with rustc 1.99.0 (edition 2024) on macOS 26 arm64. It runs in your browser with a WebAssembly build on the WASI runner (browser_wasi_shim 0.4.2). To run it yourself: rustc --edition 2024 -O main.rs && ./main

TypeScript 6.0 · main.ts

const SLICE = 2; // instructions until the timer interrupts

const PROGRAMS: Record<string, string[]> = {
  A: "load 1, square, add, load 2, square, add, load 3, square, add".split(", "),
  B: "load 1, add, load 2, mul, load 3, mul, load 4, mul".split(", "),
};

interface Saved {
  pc: number;
  acc: number;
  tmp: number;
}

const execute = (step: string, acc: number, tmp: number): [number, number] => {
  if (step.startsWith("load ")) return [acc, Number(step.slice(5))];
  if (step === "square") return [acc, tmp * tmp];
  if (step === "add") return [acc + tmp, tmp];
  return [acc * tmp, tmp];
};

const run = (saveTmp: boolean): void => {
  console.log(saveTmp ? "saving pc, acc and tmp:" : "saving pc and acc only:");
  const saved: Record<string, Saved> = {
    A: { pc: 0, acc: 0, tmp: 0 },
    B: { pc: 0, acc: 0, tmp: 0 },
  };
  const ready = Object.keys(PROGRAMS);
  let tmp = 0;
  let turn = 0;
  for (let name = ready.shift(); name !== undefined; name = ready.shift()) {
    const code = PROGRAMS[name];
    const regs = saved[name];
    let { pc, acc } = regs; // restore the saved registers
    if (saveTmp) tmp = regs.tmp;
    const ran = code.slice(pc, pc + SLICE);
    for (const step of ran) [acc, tmp] = execute(step, acc, tmp);
    pc += ran.length;
    turn += 1;
    console.log(`  turn ${turn}: ${name} ${ran.join(", ")} -> acc ${acc}, tmp ${tmp}`);
    Object.assign(regs, { pc, acc }); // save them when the timer fires
    if (saveTmp) regs.tmp = tmp;
    if (pc < code.length) ready.push(name);
    else console.log(`  ${name} finished with ${acc}`);
  }
};

run(true);
run(false);

In TypeScript: execute returns a [number, number] tuple, and the destructuring assignment writes both registers in one statement.

Recorded with TypeScript 6.0.3 (strict), run by Node.js 24.21.0 on macOS 26 arm64. It runs in your browser with QuickJS (quickjs-emscripten 0.32.0) after Sucrase 3.35.1. To run it yourself: npx -p typescript@6.0.3 tsc --noEmit --strict main.ts && node main.ts

Go 1.27 · main.go

package main

import (
	"fmt"
	"strconv"
	"strings"
)

const slice = 2 // instructions until the timer interrupts

var programs = map[string][]string{
	"A": strings.Split("load 1, square, add, load 2, square, add, load 3, square, add", ", "),
	"B": strings.Split("load 1, add, load 2, mul, load 3, mul, load 4, mul", ", "),
}

type saved struct{ pc, acc, tmp int }

func execute(step string, acc, tmp int) (int, int) {
	if n, ok := strings.CutPrefix(step, "load "); ok {
		value, _ := strconv.Atoi(n)
		return acc, value
	}
	switch step {
	case "square":
		return acc, tmp * tmp
	case "add":
		return acc + tmp, tmp
	default:
		return acc * tmp, tmp
	}
}

func run(header string, saveTmp bool) {
	fmt.Println(header)
	regs := map[string]*saved{"A": {}, "B": {}}
	ready := []string{"A", "B"}
	tmp := 0
	for turn := 1; len(ready) > 0; turn++ {
		name := ready[0]
		ready = ready[1:]
		code, r := programs[name], regs[name]
		acc := r.acc // restore the saved registers
		if saveTmp {
			tmp = r.tmp
		}
		ran := code[r.pc:min(r.pc+slice, len(code))]
		for _, step := range ran {
			acc, tmp = execute(step, acc, tmp)
		}
		did := strings.Join(ran, ", ")
		fmt.Printf("  turn %d: %s %s -> acc %d, tmp %d\n", turn, name, did, acc, tmp)
		r.pc, r.acc = r.pc+len(ran), acc // save them when the timer fires
		if saveTmp {
			r.tmp = tmp
		}
		if r.pc < len(code) {
			ready = append(ready, name)
		} else {
			fmt.Printf("  %s finished with %d\n", name, acc)
		}
	}
}

func main() {
	run("saving pc, acc and tmp:", true)
	run("saving pc and acc only:", false)
}

In Go: A slice expression capped with min is one turn. The header is a parameter, as in Rust, so run prints it with one line.

Recorded with Go 1.27.1 on macOS 26 arm64. It runs in your browser with the Go compiler in WebAssembly 1.27.1. To run it yourself: go run main.go

Swift 6.4 · main.swift

let slice = 2  // instructions until the timer interrupts

let programs: [(name: String, code: [String])] = [
  ("A", ["load 1", "square", "add", "load 2", "square", "add", "load 3", "square", "add"]),
  ("B", ["load 1", "add", "load 2", "mul", "load 3", "mul", "load 4", "mul"]),
]

struct Saved {
  var pc = 0
  var acc = 0
  var tmp = 0
}

func execute(_ step: String, acc: Int, tmp: Int) -> (acc: Int, tmp: Int) {
  if step.hasPrefix("load ") {
    return (acc, Int(step.dropFirst(5)) ?? 0)
  }
  switch step {
  case "square": return (acc, tmp * tmp)
  case "add": return (acc + tmp, tmp)
  default: return (acc * tmp, tmp)
  }
}

func run(saveTmp: Bool) {
  print(saveTmp ? "saving pc, acc and tmp:" : "saving pc and acc only:")
  var saved = [Saved(), Saved()]
  var ready = [0, 1]
  var tmp = 0
  var turn = 0
  while !ready.isEmpty {
    let p = ready.removeFirst()
    let (name, code) = programs[p]
    var acc = saved[p].acc  // restore the saved registers
    if saveTmp {
      tmp = saved[p].tmp
    }
    let ran = code[saved[p].pc..<min(saved[p].pc + slice, code.count)]
    for step in ran {
      (acc, tmp) = execute(step, acc: acc, tmp: tmp)
    }
    turn += 1
    print("  turn \(turn): \(name) \(ran.joined(separator: ", ")) -> acc \(acc), tmp \(tmp)")
    (saved[p].pc, saved[p].acc) = (saved[p].pc + ran.count, acc)  // save them when the timer fires
    if saveTmp {
      saved[p].tmp = tmp
    }
    if saved[p].pc < code.count {
      ready.append(p)
    } else {
      print("  \(name) finished with \(acc)")
    }
  }
}

run(saveTmp: true)
run(saveTmp: false)

In Swift: The turn is an array slice; iterating over it, rather than over indices from zero, works because a slice keeps the indices of its array.

Recorded with Apple Swift 6.4 on macOS 26 arm64. It runs in your browser with a WebAssembly build on the WASI runner (browser_wasi_shim 0.4.2). To run it yourself: swiftc -O main.swift -o main && ./main

Ruby 4.0 · main.rb

# frozen_string_literal: true

SLICE = 2 # instructions until the timer interrupts

PROGRAMS = {
  'A' => 'load 1, square, add, load 2, square, add, load 3, square, add'.split(', '),
  'B' => 'load 1, add, load 2, mul, load 3, mul, load 4, mul'.split(', ')
}.freeze

Saved = Struct.new(:pc, :acc, :tmp)

def execute(step, acc, tmp)
  case step
  when /\Aload (\d+)\z/ then [acc, Regexp.last_match(1).to_i]
  when 'square' then [acc, tmp * tmp]
  when 'add' then [acc + tmp, tmp]
  else [acc * tmp, tmp]
  end
end

# Gives the two programs the CPU in turns, saving tmp at each switch or not.
class Scheduler
  def initialize(save_tmp)
    @save_tmp = save_tmp
    @saved = PROGRAMS.keys.to_h { |name| [name, Saved.new(0, 0, 0)] }
    @ready = PROGRAMS.keys
    @tmp = 0
    @turn = 0
  end

  def run
    puts @save_tmp ? 'saving pc, acc and tmp:' : 'saving pc and acc only:'
    take_turn(@ready.shift) until @ready.empty?
  end

  private

  def take_turn(name)
    regs = @saved[name]
    acc = regs.acc # restore the saved registers
    @tmp = regs.tmp if @save_tmp
    ran = PROGRAMS[name][regs.pc, SLICE]
    ran.each { |step| acc, @tmp = execute(step, acc, @tmp) }
    report(name, ran, acc)
    save(regs, regs.pc + ran.size, acc)
    after_turn(name, regs)
  end

  def report(name, ran, acc)
    @turn += 1
    puts "  turn #{@turn}: #{name} #{ran.join(', ')} -> acc #{acc}, tmp #{@tmp}"
  end

  def save(regs, counter, acc)
    regs.pc = counter # save them when the timer fires
    regs.acc = acc
    regs.tmp = @tmp if @save_tmp
  end

  def after_turn(name, regs)
    if regs.pc < PROGRAMS[name].size
      @ready.push(name)
    else
      puts "  #{name} finished with #{regs.acc}"
    end
  end
end

Scheduler.new(true).run
Scheduler.new(false).run

In Ruby: code[start, SLICE] returns at most SLICE instructions, fewer at the end, and the scheduler is a class so that each method stays short.

Recorded with CRuby 4.0.7 on macOS 26 arm64. It runs in your browser with ruby.wasm 2.10.1 (Ruby 4.0.0). To run it yourself: ruby main.rb

Kotlin 2.4 · Main.kt

const val SLICE = 2 // instructions until the timer interrupts

val PROGRAMS =
    mapOf(
        "A" to "load 1, square, add, load 2, square, add, load 3, square, add".split(", "),
        "B" to "load 1, add, load 2, mul, load 3, mul, load 4, mul".split(", "),
    )

class Saved {
    var pc = 0
    var acc = 0
    var tmp = 0
}

fun execute(
    step: String,
    acc: Int,
    tmp: Int,
): Pair<Int, Int> =
    when {
        step.startsWith("load ") -> acc to step.removePrefix("load ").toInt()
        step == "square" -> acc to tmp * tmp
        step == "add" -> acc + tmp to tmp
        else -> acc * tmp to tmp
    }

fun run(saveTmp: Boolean) {
    println(if (saveTmp) "saving pc, acc and tmp:" else "saving pc and acc only:")
    val saved = mapOf("A" to Saved(), "B" to Saved())
    val ready = ArrayDeque(listOf("A", "B"))
    var tmp = 0
    var turn = 0
    while (ready.isNotEmpty()) {
        val name = ready.removeFirst()
        val code = PROGRAMS.getValue(name)
        val regs = saved.getValue(name)
        var acc = regs.acc // restore the saved registers
        if (saveTmp) tmp = regs.tmp
        val ran = code.subList(regs.pc, minOf(regs.pc + SLICE, code.size))
        for (step in ran) {
            val (newAcc, newTmp) = execute(step, acc, tmp)
            acc = newAcc
            tmp = newTmp
        }
        turn += 1
        println("  turn $turn: $name ${ran.joinToString(", ")} -> acc $acc, tmp $tmp")
        regs.pc += ran.size // save them when the timer fires
        regs.acc = acc
        if (saveTmp) regs.tmp = tmp
        if (regs.pc < code.size) {
            ready.addLast(name)
        } else {
            println("  $name finished with $acc")
        }
    }
}

fun main() {
    run(true)
    run(false)
}

In Kotlin: Kotlin cannot assign a pair to variables that already exist, so execute's result is unpacked into new names first.

Recorded with Kotlin 2.4.20 on Java 25.0.4.1 on macOS 26 arm64. It runs in your browser with a WebAssembly build on the WASI runner (browser_wasi_shim 0.4.2). To run it yourself: kotlinc Main.kt -include-runtime -d main.jar && java -jar main.jar

Dart 3.13 · main.dart

import 'dart:collection';
import 'dart:math';

const slice = 2; // instructions until the timer interrupts

final programs = {
  'A': 'load 1, square, add, load 2, square, add, load 3, square, add'.split(
    ', ',
  ),
  'B': 'load 1, add, load 2, mul, load 3, mul, load 4, mul'.split(', '),
};

class Saved {
  int pc = 0;
  int acc = 0;
  int tmp = 0;
}

(int, int) execute(String step, int acc, int tmp) => switch (step) {
  _ when step.startsWith('load ') => (acc, int.parse(step.substring(5))),
  'square' => (acc, tmp * tmp),
  'add' => (acc + tmp, tmp),
  _ => (acc * tmp, tmp),
};

void run(bool saveTmp) {
  print(saveTmp ? 'saving pc, acc and tmp:' : 'saving pc and acc only:');
  final saved = {'A': Saved(), 'B': Saved()};
  final ready = Queue.of(['A', 'B']);
  var tmp = 0;
  for (var turn = 1; ready.isNotEmpty; turn++) {
    final name = ready.removeFirst();
    final code = programs[name]!;
    final regs = saved[name]!;
    var acc = regs.acc; // restore the saved registers
    if (saveTmp) tmp = regs.tmp;
    final ran = code.sublist(regs.pc, min(regs.pc + slice, code.length));
    for (final step in ran) {
      (acc, tmp) = execute(step, acc, tmp);
    }
    print('  turn $turn: $name ${ran.join(', ')} -> acc $acc, tmp $tmp');
    regs.pc += ran.length; // save them when the timer fires
    regs.acc = acc;
    if (saveTmp) regs.tmp = tmp;
    if (regs.pc < code.length) {
      ready.addLast(name);
    } else {
      print('  $name finished with $acc');
    }
  }
}

void main() {
  run(true);
  run(false);
}

In Dart: sublist capped with min gives one turn, and a record pattern assigns (acc, tmp) to the existing variables in one statement.

Recorded with Dart 3.13.5 on macOS 26 arm64. It runs in your browser with a WebAssembly build by dart2wasm 3.13.5. To run it yourself: dart run main.dart

What every language printed

saving pc, acc and tmp:
  turn 1: A load 1, square -> acc 0, tmp 1
  turn 2: B load 1, add -> acc 1, tmp 1
  turn 3: A add, load 2 -> acc 1, tmp 2
  turn 4: B load 2, mul -> acc 2, tmp 2
  turn 5: A square, add -> acc 5, tmp 4
  turn 6: B load 3, mul -> acc 6, tmp 3
  turn 7: A load 3, square -> acc 5, tmp 9
  turn 8: B load 4, mul -> acc 24, tmp 4
  B finished with 24
  turn 9: A add -> acc 14, tmp 9
  A finished with 14
saving pc and acc only:
  turn 1: A load 1, square -> acc 0, tmp 1
  turn 2: B load 1, add -> acc 1, tmp 1
  turn 3: A add, load 2 -> acc 1, tmp 2
  turn 4: B load 2, mul -> acc 2, tmp 2
  turn 5: A square, add -> acc 5, tmp 4
  turn 6: B load 3, mul -> acc 6, tmp 3
  turn 7: A load 3, square -> acc 5, tmp 9
  turn 8: B load 4, mul -> acc 24, tmp 4
  B finished with 24
  turn 9: A add -> acc 9, tmp 4
  A finished with 9

Eight of the nine turns printed the same values in both runs, and B finished with 24 both times. A resumed at turn 9 with tmp at 4, B’s last value, instead of its own 9, and added it: 9 instead of 14. The bug hid for most of the run because each program happened to load tmp before it used it, or found the same value there by chance. A register that is not saved corrupts results only sometimes, and silently.

Key idea

A switch is invisible to a program only if it restores every register the program can see. On 64-bit Arm Linux the trap saves the user registers on the kernel stack, and the switch saves the kernel’s own registers in the task_struct.

Checkpoint

Check: What a switch saves, and where

Question 1 of 2 On 64-bit Arm Linux, where are a process's user registers, x0 to x30, kept while another process runs?
Choose one answer.

In a frame on its kernel stack, where the trap saved them Nowhere; the next process overwrites them On its user stack In its task_struct, saved by cpu_switch_to

Show the answer: Question 1 of 2

Answer: In a frame on its kernel stack, where the trap saved them

There are two saves. The trap saves the user registers in a frame on the thread's kernel stack, and the switch saves the registers of the kernel's own code in the task_struct; both are restored when the thread runs again.
Question 2 of 2 In the model, at which turn did the run that saves pc and acc only first print something different from the run that saves all three?
Type a number.
Your answer turn

Show the answer: Question 2 of 2

Answer: 9 turn

Turns 1 to 8 printed the same values in both runs, because each program either loaded tmp before using it or found the same value by chance. At turn 9, A added B's tmp, 4, instead of its own 9, and finished with 9.

Mode, thread and process switches change different things

Three different events are called switches, and their costs differ because they change different things:

Aspect	Mode switch	Thread switch	Process switch
What happens	A trap into the kernel and back to the same thread, as in a system call	The kernel runs another thread of the same process	The kernel runs a thread of another process
Registers	Saved on entry, restored on exit	One thread’s saved, another’s restored	One thread’s saved, another’s restored
Page tables	Kept	Kept: both threads use the same mm	Changed to the new process’s
TLB and caches	Still warm	Still useful: the memory is the same	Cold, unless the TLB keeps tagged entries and the data is still cached
Scheduler	Not involved	Chooses the next thread	Chooses the next thread

The thread case is visible in Linux’s code: switch_mm for 64-bit Arm begins with if (prev != next), and two threads of one process share one mm_struct, so for them the address space is left alone. The lesson on user mode and kernel mode counted both kinds of event in a trace: a system call is two mode switches, and a context switch happens only when the kernel decides to run something else, for example because a call has to wait.

The page-table change is real work, but on a modern processor it can be small next to everything else a hand-off does. In the measurements of the next section, hand-offs between two processes had a median of 30.3 µs and those between two threads 50.9 µs: the process switch, with its new page tables, was not the slower of the two, because the spread from run to run was larger than any difference the page tables make.

Checkpoint

Check: Mode, thread and process switches

Question 1 of 2 The kernel switches from thread 1 to thread 2 of the same process. What does switch_mm do on 64-bit Arm Linux?
Choose one answer.

It flushes the TLB It loads thread 2's page tables Nothing to the address space, because both threads use the same mm_struct It saves thread 1's registers

Show the answer: Question 1 of 2

Answer: Nothing to the address space, because both threads use the same mm_struct

A thread switch keeps the address space, and with it the TLB's translations and much of the cache. That is the part of a process switch that a thread switch saves.
Question 2 of 2 A program calls getpid, which returns at once. Which switches happen?
Choose one answer.

Two mode switches and two context switches No switch at all One context switch Two mode switches and no context switch

Show the answer: Question 2 of 2

Answer: Two mode switches and no context switch

A system call that returns at once is a mode switch into the kernel and one back. It becomes a context switch only when it has to wait, as a read from an empty pipe does.

Measuring a hand-off: a ping-pong, run many times

A switch is too short to time on its own, so benchmarks make many of them and divide. lmbench’s lat_ctx passes a token between processes “connected in a ring of Unix pipes”, in the words of its manual. A thread that reads from an empty pipe sleeps until the other side writes, as pipe(7) describes, so every pass of the token includes a hand-off. The program below times one-byte round trips in three settings: one thread writing and reading the same pipe, which never waits, two threads of one process, and two processes. Each setting runs 9 times, and the program prints the median with the fastest and the slowest run:

One-byte round trips: alone, between threads, between processes Rust · switch_cost.rs

use std::io::{self, Read, Write};
use std::process::{Command, Stdio};
use std::thread;
use std::time::Instant;

const TRIPS: u32 = 10_000; // round trips of one byte in each run
const RUNS: usize = 9; // runs of each kind: the median and the spread are printed

/// Sends one byte and waits for it to come back, TRIPS times: microseconds a trip.
fn time_trips(send: &mut impl Write, receive: &mut impl Read) -> io::Result<f64> {
    let mut byte = [0u8; 1];
    let start = Instant::now();
    for _ in 0..TRIPS {
        send.write_all(b"x")?;
        receive.read_exact(&mut byte)?;
    }
    Ok(start.elapsed().as_secs_f64() * 1e6 / f64::from(TRIPS))
}

/// The other side of a ping-pong: every byte that arrives goes straight back.
fn echo(input: &mut impl Read, output: &mut impl Write) -> io::Result<()> {
    let mut byte = [0u8; 1];
    while input.read(&mut byte)? == 1 {
        output.write_all(&byte)?;
        output.flush()?;
    }
    Ok(())
}

/// The median of RUNS runs, and the fastest and the slowest run.
fn summary(mut run: impl FnMut() -> io::Result<f64>) -> io::Result<(f64, f64, f64)> {
    let mut times = (0..RUNS).map(|_| run()).collect::<io::Result<Vec<f64>>>()?;
    times.sort_by(f64::total_cmp);
    Ok((times[RUNS / 2], times[0], times[RUNS - 1]))
}

fn main() -> io::Result<()> {
    if std::env::args().nth(1).as_deref() == Some("echo") {
        return echo(&mut io::stdin().lock(), &mut io::stdout().lock());
    }
    // One thread writes and reads the same pipe: two system calls, no switch.
    let (mut reader, mut writer) = io::pipe()?;
    let alone = summary(|| time_trips(&mut writer, &mut reader))?;
    // Two threads of one process: ping carries a byte to the echo, pong brings it back.
    let (mut ping, pong) = (io::pipe()?, io::pipe()?);
    let (mut back, mut reply) = pong;
    let helper = thread::spawn(move || echo(&mut ping.0, &mut reply));
    let threads = summary(|| time_trips(&mut ping.1, &mut back))?;
    drop(ping.1); // the helper's next read sees the end of the pipe
    helper
        .join()
        .map_err(|_| io::Error::other("the echo thread panicked"))??;
    // Two processes: this program again, started as the echo.
    let mut child = Command::new(std::env::current_exe()?)
        .arg("echo")
        .stdin(Stdio::piped())
        .stdout(Stdio::piped())
        .spawn()?;
    let (mut input, mut output) = (child.stdin.take(), child.stdout.take());
    let processes = match (&mut input, &mut output) {
        (Some(input), Some(output)) => summary(|| time_trips(input, output))?,
        _ => return Err(io::Error::other("no pipes to the child")),
    };
    drop(input);
    child.wait()?;
    println!("one-byte round trips, {TRIPS} a run, {RUNS} runs: median (fastest to slowest)");
    for (label, (median, fast, slow)) in [
        ("one thread, no switch", alone),
        ("two threads", threads),
        ("two processes", processes),
    ] {
        println!("{label:<22} {median:5.1} µs ({fast:.1} to {slow:.1})");
    }
    // Each trip hands the byte over twice: block, wake the other side, switch to it.
    let hand_off = |round_trip: f64| (round_trip - alone.0) / 2.0;
    let (between_threads, between_processes) = (hand_off(threads.0), hand_off(processes.0));
    println!(
        "a hand-off: about {between_threads:.1} µs between threads, {between_processes:.1} µs between processes"
    );
    Ok(())
}

Output

one-byte round trips, 10000 a run, 9 runs: median (fastest to slowest)
one thread, no switch    0.8 µs (0.8 to 1.1)
two threads            102.5 µs (59.0 to 136.1)
two processes           61.4 µs (25.5 to 88.2)
a hand-off: about 50.9 µs between threads, 30.3 µs between processes

This output changes from run to run: the times depend on the machine and on what else it is running

Recorded with rustc 1.99.0 (b940084d7) on macOS 26 arm64. To run it yourself: rustc --edition 2024 -O switch_cost.rs && ./switch_cost

Writing and reading a pipe without waiting took 0.8 µs a round trip, the cost of the two system calls. Handing the byte over took far longer: a median of 102.5 µs a round trip between two threads and 61.4 µs between two processes, or about 51 and 30 µs a hand-off. The spread matters as much as the medians. The slowest run between threads took 136.1 µs, more than twice the fastest, 59.0 µs, because other programs were running on the same machine at the same time.

A hand-off contains more than the switch itself. The sleeping side must be woken: the kernel marks it ready, and on a machine with several cores the two sides usually run on different cores, so the woken thread’s core may first have to leave an idle state. To measure the switch on one core, pin both sides to it: on Linux, taskset -c 0 ./switch_cost starts the program with a CPU affinity: The set of CPUs a thread or process may run on. On Linux, taskset and sched_setaffinity set it, for example to keep both sides of a benchmark on one core. of core 0 only, and sched_setaffinity does the same from inside a program. macOS has no such call: Apple’s notes on its affinity API state that “explicit thread to processor binding is not supported”, so runs on a Mac always include the trip between cores.

Timing one run and calling it the cost

One run measures one moment of a busy machine. Run the benchmark many times, report the median, which a few disturbed runs barely move, together with the spread, and say what the machine was doing. The exercise below turns raw runs into exactly that.

Checkpoint

Check: Measuring a hand-off

Question 1 of 2 Why does the benchmark also time one thread writing and reading its own pipe, which never waits?
Choose one answer.

It warms the CPU up before the real runs It measures a context switch on one core It checks that the pipe works It measures the two system calls of a trip without any hand-off, so subtracting it leaves the cost of the hand-offs

Show the answer: Question 1 of 2

Answer: It measures the two system calls of a trip without any hand-off, so subtracting it leaves the cost of the hand-offs

A round trip is two writes, two reads and two hand-offs. With the same calls timed alone, the difference divided by two is the cost of one hand-off.
Question 2 of 2 Nine runs give hand-offs of 40, 41, 41, 42, 43, 44, 44, 45 and 160 microseconds. What is the median?
Type a number.
Your answer microseconds

A hint

Sort them and take the middle one, the fifth of nine.

Show the answer: Question 2 of 2

Answer: 43 microseconds

The middle value of nine sorted runs is the fifth, 43. The mean, about 55.6, is pulled up by one disturbed run, while the median stays where the typical runs are; that is why the program prints medians.

The cost after the switch: caches, the TLB and branch predictors

Saving and loading registers is not the end of the bill. The new thread finds the caches full of the old thread’s data and fetches its own from memory, so it runs slower for a while after it starts. lmbench measures this with a size parameter: each process sums an array of that size before it passes the token on, and the manual notes that this cache pollution “results in larger context switching times for the larger processes”.

The TLB, the translation lookaside buffer (TLB): A small cache in each core of recent translations from virtual to physical addresses, so that most memory accesses need no walk through the page tables., suffers in the same way. It caches recent address translations, and a process switch changes the page tables, so translations of the old process are useless to the new one. A TLB that cannot tell them apart must be flushed: Linux’s documentation on page table isolation states that without PCIDs, “each CR3 write flushes the entire TLB”, CR3 being the x86-64 register that points at the page tables. Tags fix it. Step through one core running A, then B, then A again:

Interactive · step by step The TLB across a process switch, with and without tags Step through A, then B, then A again on one core, first with a TLB that must be flushed at every switch, then with ASID tags.

The drawing shows the whole scene; its 7 steps are listed below.

Every step as text

A starts. A touches four pages. Each first access misses in the TLB, so the core walks A's page tables in memory and keeps the translation.
A runs warm. Every later access to those pages is a TLB hit, answered in the core without touching the page tables.
Switch to B, untagged. Without tags, the TLB cannot tell A's entries from B's, so switching page tables flushes it. On x86-64 without PCIDs, every write of CR3 does this.
B pays. B's four pages all miss and are walked, although B may have run on this core a moment ago.
Back to A, untagged. The switch back flushes B's entries, and A's four pages miss again. Every switch has turned the TLB cold.
Now with ASID tags. With tags, each entry records its address space, A's as ASID 7 and B's as ASID 9, and a switch changes only the current tag.
Back to A, tagged. A's entries are still there, so A's next accesses hit at once. Linux on 64-bit Arm flushes the TLB only when it runs out of ASIDs and starts a new generation.

An address space identifier (ASID): A tag the processor keeps with each TLB entry to say which address space it belongs to, so that the entries of several processes can stay in the TLB across context switches. x86-64 calls it a PCID. lets entries of several processes stay in the TLB at once. On 64-bit Arm, Linux gives each address space an ASID of 8 or 16 bits, as the processor’s ID register reports, and flushes the TLB only when it runs out of ASIDs and starts a new generation (arch/arm64/mm/context.c).

Branch predictors keep state too, and security can throw it away on purpose. On x86, Linux can issue an Indirect Branch Prediction Barrier when it switches to or from a process that has turned off indirect branch speculation, which clears the branch target buffer, so the next process starts with cold predictions as well. The CS2023 curriculum guidelines of the ACM, the IEEE Computer Society and AAAI list exactly this as a topic of its own: the performance costs of context switches and the cache flushes of process switches in Spectre-mitigated environments.

Checkpoint

Check: The cost after a switch

One question A, then B, then A run on one core whose TLB tags every entry with an ASID. What does A find when it resumes?
Choose one answer.

Its translations, because the TLB is saved in the task_struct B's translations, used for A's addresses An empty TLB, because every switch flushes it Its translations still in the TLB, unless B's entries have pushed them out

Show the answer: One question

Answer: Its translations still in the TLB, unless B's entries have pushed them out

With ASIDs, as in the last steps of the figure, A's entries are still there when A comes back, so A's next accesses hit. Without tags, each switch would have flushed them.

Counting switches: voluntary and involuntary

The kernel counts the switches of every process. getrusage(2) reports two numbers: voluntary context switch: A context switch that happens because the running thread waits for something, such as I/O, a lock or a timer, and gives the CPU up before its time slice ends., in ru_nvcsw, when a process gives the processor up “before its time slice was completed”, usually to wait for a resource, and involuntary context switch: A context switch forced on a running thread, because its time slice ended or because a thread of higher priority became ready to run., in ru_nivcsw, when a process of higher priority became runnable or the time slice ran out. Linux shows the same counts in /proc/<pid>/status, as voluntary_ctxt_switches and nonvoluntary_ctxt_switches. This program does three kinds of work and prints how much each count grew:

Which switches a computation, a sleep and a pipe wait add Python · switch_counts.py

import os
import resource
import time


def switches() -> tuple[int, int]:
    usage = resource.getrusage(resource.RUSAGE_SELF)
    return usage.ru_nvcsw, usage.ru_nivcsw


def measure(label: str, work) -> None:
    voluntary, involuntary = switches()
    work()
    after = switches()
    print(
        f"{label:<34} voluntary {after[0] - voluntary:5d}  involuntary {after[1] - involuntary:5d}"
    )


def compute() -> None:
    deadline = time.perf_counter() + 0.5
    while time.perf_counter() < deadline:  # never waits for anything
        pass


def sleep_often() -> None:
    for _ in range(200):
        time.sleep(0.001)  # gives the CPU up each time


def wait_for_a_pipe() -> None:
    reader, writer = os.pipe()
    pid = os.fork()
    if pid == 0:  # the child sends a byte every millisecond
        for _ in range(200):
            time.sleep(0.001)
            os.write(writer, b"x")
        os._exit(0)
    for _ in range(200):
        os.read(reader, 1)  # waits until the child's next byte arrives
    os.waitpid(pid, 0)


measure("computing for 0.5 s", compute)
measure("200 sleeps of 1 ms", sleep_often)
measure("200 reads that wait for a pipe", wait_for_a_pipe)

Output

computing for 0.5 s                voluntary     0  involuntary   546
200 sleeps of 1 ms                 voluntary     0  involuntary   281
200 reads that wait for a pipe     voluntary   172  involuntary    35

This output changes from run to run: the counts depend on what else the machine is running

Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 switch_counts.py

Computing for half a second added 546 involuntary switches and no voluntary ones: the computation never waited, and other programs kept taking its core. The 200 reads from the pipe added 172 voluntary switches: a read that finds the pipe empty sleeps until the next byte arrives, while a read whose byte is already there returns without a switch. The 200 sleeps, though, added no voluntary switches at all, and 281 involuntary ones. XNU’s source explains why. It counts a voluntary switch in one place only, _sleep, the routine behind the kernel’s BSD sleeps, which a pipe read uses while it waits for data (msleep with the wait message “piperd”), and calcru reports every other switch as involuntary. A sleep of 1 ms on macOS waits through another path, so it lands among the involuntary switches, although by getrusage(2)’s definition it is voluntary.

Comparing switch counts between systems

The same program can report its waits as voluntary on one system and as involuntary on another, as the recorded run shows. Compare switch counts only between runs on the same operating system, and read what the counters mean there before drawing conclusions.

Checkpoint

Check: Voluntary and involuntary switches

One question In the recorded run on macOS, which of the three kinds of work added voluntary switches?
Choose one answer.

The computation Only the reads from the pipe: 172 switches for 200 reads Only the 200 sleeps of 1 ms All three

Show the answer: One question

Answer: Only the reads from the pipe: 172 switches for 200 reads

XNU increments ru_nvcsw only in _sleep, the routine behind its BSD sleeps, which pipe_read uses while it waits for data; calcru reports every other switch as involuntary.

Practise: summarise ping-pong timings

Write summarize, which turns the round trips of several benchmark runs into the median cost of one hand-off and its interquartile range. The sample tests run in your browser, and one of them has a run disturbed the way a busy machine disturbs a measurement:

Exercise · Easy · Python

Turn ping-pong timings into a hand-off latency

A ping-pong benchmark ran several times. Each run reports its round-trip time in microseconds, and a run of the same pipe calls in one thread, with no switch at all, gives the cost of the calls themselves. Write summarize(round_trips, no_switch) in summary.py, which turns them into the cost of one hand-off:

a run's hand-off time is (round_trip - no_switch) / 2, because every round trip hands over twice;
return a tuple (median, spread) of the runs' hand-off times, where median is statistics.median of them and spread is their interquartile range: q3 - q1, with q1, _, q3 = statistics.quantiles(times, n=4);
round both to 2 decimal places.

For example, round trips of [10.0, 12.0, 14.0, 16.0, 18.0] with no_switch = 2.0 are hand-offs of 4 to 8 microseconds, and summarize returns (6.0, 3.0). There are always at least two runs.

Starter code · summary.py

import statistics


def summarize(round_trips, no_switch):
    """The median and the interquartile range of the hand-off times, in microseconds, rounded to 2 places."""
    # Replace this line with your code.
    return 0.0, 0.0

The sample tests · test_summary.py

from summary import summarize


def test_the_example():
    """gives a median of 6 and a spread of 3 for hand-offs of 4 to 8 microseconds"""
    assert summarize([10.0, 12.0, 14.0, 16.0, 18.0], 2.0) == (6.0, 3.0)


def test_the_order_does_not_matter():
    """gives the same answer when the runs come in another order"""
    assert summarize([18.0, 10.0, 16.0, 12.0, 14.0], 2.0) == (6.0, 3.0)


def test_an_even_number_of_runs():
    """takes the median of an even number of runs between the two middle values"""
    assert summarize([20.0, 22.0, 30.0, 24.0], 0.0) == (11.5, 4.0)


def test_one_slow_run_barely_moves_the_median():
    """keeps the median near the typical run when one run was disturbed"""
    runs = [91.2, 88.7, 95.0, 120.4, 89.9, 90.3, 93.8, 150.6, 92.1]
    assert summarize(runs, 1.1) == (45.5, 8.8)

A hint

First build the list of hand-off times with a comprehension, [(trip - no_switch) / 2 for trip in round_trips]. The median and the quartiles both come from that list, and their order in it does not matter.

The sample tests run on this device, in your browser (Pyodide): nothing is sent to mysmartcopilot.com. The first run downloads Python (about 13.5 MB), which is kept for the next runs. A check in your browser is feedback for you, not proof that the code is right for every input.

Then answer questions on all five ideas of this lesson, one at a time, starting with the one your checkpoint answers suggest you know least:

Interview questions

Here is a warm-up with a complete answer. The module’s bank adds two harder ones from this lesson: a service that slows down when it is given more threads, and a context-switch benchmark whose results differ between two machines.

Fresher to mid · Warm-up

What does the kernel save and restore in a context switch?

Everything the interrupted thread could be using, so that it later continues as if it had never stopped: its general-purpose registers, its stack pointer and program counter, the processor's status flags, and its floating-point and vector registers. For a switch to another process, the kernel also points the MMU at the new process's page tables. On 64-bit Arm Linux the user registers are saved at the trap, in a frame on the thread's kernel stack, and cpu_switch_to saves the kernel's own registers, x19 to x29, the stack pointer and the link register, in the task_struct; the next thread's are loaded the same way in reverse. What is not saved matters as well: caches, the TLB and the branch predictors keep what they held, so the new thread often starts cold, which is part of what a switch really costs.

Model answers open with any pass: ₹79 for 1 week ($2.99), one payment that never renews.

Choose a pass

Mid to senior

To raise throughput, a service on a 4-core machine went from 8 to 64 worker threads. Throughput fell by a fifth, and context switches rose from about 5,000 to 250,000 a second. Explain what happened and what you would change.

Model answer, Pro. Model answers open with any pass; prices are in the notice on this page. First step and follow-ups
Mid to staff

Your pipe ping-pong benchmark reports about 3 microseconds per hand-off on one Linux server and about 40 on a laptop. How do you make the measurement trustworthy, and what could explain the gap?

Model answer, Pro. First step and follow-ups

Key takeaways

A context switch must restore every register the program can see; on 64-bit Arm Linux the trap saves the user registers on the kernel stack and cpu_switch_to saves the kernel’s own registers in the task_struct.
A register left out of the save corrupts results only sometimes: in the model, eight of nine turns were the same, and the ninth gave A 9 instead of 14.
A mode switch keeps the thread, a thread switch keeps the address space, and only a process switch changes the page tables.
A pipe ping-pong measures a hand-off, which includes waking the other side and often another core, so report the median of many runs with its spread, and pin both sides to one core where the system allows it.
After a switch the caches and the TLB are cold for the new thread unless their contents survive; ASIDs and PCIDs keep TLB entries across switches, and Spectre mitigations can clear the branch predictors on purpose.
getrusage counts voluntary and involuntary switches, but systems differ in what they count as voluntary: on macOS, sleeps landed among the involuntary ones.

Terms in this lesson

address space identifier (ASID): A tag the processor keeps with each TLB entry to say which address space it belongs to, so that the entries of several processes can stay in the TLB across context switches. x86-64 calls it a PCID.
CPU affinity: The set of CPUs a thread or process may run on. On Linux, taskset and sched_setaffinity set it, for example to keep both sides of a benchmark on one core.
involuntary context switch: A context switch forced on a running thread, because its time slice ended or because a thread of higher priority became ready to run.
kernel stack: The stack a thread uses while it runs in the kernel. Linux gives every thread its own, and the registers saved when a trap enters the kernel are kept on it.
translation lookaside buffer (TLB): A small cache in each core of recent translations from virtual to physical addresses, so that most memory accesses need no walk through the page tables.
voluntary context switch: A context switch that happens because the running thread waits for something, such as I/O, a lock or a timer, and gives the CPU up before its time slice ends.

References

Linux arch/arm64/kernel/entry.S: kernel_entry and cpu_switch_to (The Linux Kernel Organization (source code))
Linux arch/arm64/kernel/process.c: __switch_to (The Linux Kernel Organization (source code))
Linux arch/arm64/include/asm/mmu_context.h: switch_mm (The Linux Kernel Organization (source code))
Linux arch/arm64/mm/context.c: ASID allocation (The Linux Kernel Organization (source code))
Page Table Isolation (PTI) (The Linux Kernel Organization)
Spectre side channels (The Linux Kernel Organization)
getrusage(2): get resource usage (Linux man-pages project (man7.org))
proc_pid_status(5): memory usage and status information (Linux man-pages project (man7.org))
sched_setaffinity(2): set and get a thread CPU affinity mask (Linux man-pages project (man7.org))
taskset(1): set or retrieve a process CPU affinity (Linux man-pages project (man7.org))
pipe(7): overview of pipes and FIFOs (Linux man-pages project (man7.org))
lat_ctx(8): context switching benchmark (lmbench) (lmbench project)
XNU bsd/kern/kern_resource.c: calcru (Apple Open Source (XNU source code))
XNU bsd/kern/kern_synch.c: _sleep (Apple Open Source (XNU source code))
XNU bsd/kern/sys_pipe.c: pipe_read (Apple Open Source (XNU source code))
Thread Affinity API Release Notes (Apple Developer Documentation Archive)
Function std::io::pipe (Rust standard library) (The Rust Project)
Computer Science Curricula 2023: Operating Systems knowledge area (Version Gamma) (ACM, IEEE Computer Society and AAAI (CS2023 Task Force))

Report a problem with this lesson

In this module: Processes: creation, life cycle, signals and IPC

The process: address space, PCB and states Free: 20 minutes
Context switches and their real cost Free lesson, Pro answers: 15 minutes
Creating processes: fork, exec and wait Free lesson, Pro answers: 15 minutes
Exit, zombies, orphans and daemons Free lesson, Pro answers: 15 minutes
Signals and safe signal handlers Free lesson, Pro answers: 25 minutes
IPC: pipes, FIFOs and Unix domain sockets Free lesson, Pro answers: 20 minutes
IPC: shared memory and message queues Free lesson, Pro answers: 20 minutes

Your country

Context switches and their real cost

What you will learn

Before you start

What you will learn

Before you start

A context switch saves every register the program can see

Mode, thread and process switches change different things

Measuring a hand-off: a ping-pong, run many times

The cost after the switch: caches, the TLB and branch predictors

Counting switches: voluntary and involuntary

Practise: summarise ping-pong timings

Turn ping-pong timings into a hand-off latency

Results of the sample tests

Interview questions

What does the kernel save and restore in a context switch?

To raise throughput, a service on a 4-core machine went from 8 to 64 worker threads. Throughput fell by a fifth, and context switches rose from about 5,000 to 250,000 a second. Explain what happened and what you would change.

Your pipe ping-pong benchmark reports about 3 microseconds per hand-off on one Linux server and about 40 on a laptop. How do you make the measurement trustworthy, and what could explain the gap?

Key takeaways

References