Measuring cycles: counters, perf and llvm-mca

Two runs of the same loop can take the same number of core cycles and very different numbers of nanoseconds, because a core’s clock speed changes with its power state and with which core runs the work. So before you trust a measurement, answer one question: does your counter count time or cycles? The microbenchmarkmicrobenchmark: A small program that times one operation or loop in isolation; useful for comparing instruction sequences but easily misled by effects a whole program would not see. below times a loop on Apple silicon and reports its best of 15 runs, the counter’s frequency and the hardware, so that its one number means something.

A fixed-frequency counter, read with the hygiene that makes the number repeat, gives you time. Cycles take one of three routes: the cycle counter that each ISA guards behind the kernel, perfperf: A Linux profiling tool that reads the hardware performance counters while a program runs and reports its cycles, instructions, cache misses and branch misses., which reads the real hardware counters on Linux, and llvm-mca, which estimates cycles from a model of one core without running the code.

A good measurement repeats and reports the hardware

A single timing on a shared machine means little. The benchmark follows a short checklist, which the figure steps through: fixed inputs, so the run repeats; warm-upwarm-up: Running a workload a few times before timing it, so caches are filled and the clock has settled and the timed runs measure the steady state rather than the first cold pass. runs, so the caches and the clock settle; many timed runs, of which it keeps the best and the median; and the hardware and the counter’s frequency printed beside the result.

It reads the fixed-frequency counterfixed-frequency counter: A counter that advances at a constant frequency independent of the core clock, such as AArch64's CNTVCT_EL0; it measures elapsed time, not the number of core cycles. in counters.s around a reduction over 4,096 doubles:

import pathlib
import sys

sys.path.insert(0, str(pathlib.Path(__file__).parent / "sessions"))
import session

session.run("""
# Build the benchmark (the counter reads are in counters.s) and run it
clang -O2 -o bench bench_main.c counters.s
./bench
""")

# Build the benchmark (the counter reads are in counters.s) and run it
$ clang -O2 -o bench bench_main.c counters.s
$ ./bench
Apple M5 Pro, counter 1000 MHz
sum of 4096 doubles = 8.8951038970
best 3333 ns (0.814 ns/elem), median 3500 ns, over 15 timed runs after 3 warm-ups

The best run is the one least disturbed by other work, because noise on a shared machine only ever adds time; the median beside it shows how much the machine is adding. The checksum shows that the run repeats and that the loop really ran. A compiler may delete a call whose result nobody uses, and the timer then brackets nothing.

The counter you read measures time, not cycles

The benchmark printed counter 1000 MHz. CNTFRQ_EL0 reports the counter’s effective frequency, so the counter’s value counts nanoseconds, whatever the core is doing. That is what fixed-frequency means, and it is why the counter is a good clock. It is also why you cannot read core cycles from it. A performance core and an efficiency core run at different speeds, and each changes speed with its power state, so a fixed number of time ticks covers a changing number of core cycles.

A frequency is the counter’s unit, not a promise of how often its value changes. A counter that changes less often than its unit suggests cannot time work shorter than one change: such work reads as zero or as one whole change.

So the benchmark honestly reports nanoseconds, not cycles. Each ISA pairs its time counter with a cycle countercycle counter: A counter of the core's own clock cycles, such as a performance unit's cycle count; reading it from user code usually needs the operating system's permission., which user code may usually read only with the kernel’s permission. perf is the Linux profiler that reads the processor’s performance countersperformance counter: A hardware counter of microarchitectural events such as cycles, retired instructions, cache misses and branch mispredictions, read through the operating system's profiling interface. while a program runs: cycles, cache misses, branch mispredictions. llvm-mca reads machine code and a model of one named core and estimates the cycles without running anything, so it gives the same answer on every machine.

Practise

Work on copies of the benchmark’s two files:

// bench_main.c: a small benchmark that follows the hygiene checklist and reports what a reader needs to trust it.
// It times sum_doubles over a fixed 4,096-element array with the fixed-frequency counter, after warm-up runs,
// and reports the best and the median of the timed runs, the counter's frequency, and the hardware.
#include <pthread/qos.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/sysctl.h>

uint64_t read_counter(void);
uint64_t counter_freq(void);
double sum_doubles(const double *a, long n);

enum { N = 4096, WARMUP = 3, TIMED = 15 };
static double a[N];

static int by_value(const void *x, const void *y) {
    uint64_t p = *(const uint64_t *)x, q = *(const uint64_t *)y;
    return (p > q) - (p < q);
}

int main(void) {
    pthread_set_qos_class_self_np(QOS_CLASS_USER_INTERACTIVE, 0);   // ask for a performance core
    for (long i = 0; i < N; i++) a[i] = 1.0 / (double)(i + 1);       // fixed inputs: the run repeats exactly
    double freq = (double)counter_freq();
    volatile double sink = 0;
    for (int w = 0; w < WARMUP; w++) sink += sum_doubles(a, N);      // warm up: caches and clocks settle

    uint64_t ticks[TIMED];
    for (int r = 0; r < TIMED; r++) {
        uint64_t t0 = read_counter();
        sink += sum_doubles(a, N);
        ticks[r] = read_counter() - t0;
    }
    qsort(ticks, TIMED, sizeof ticks[0], by_value);
    double best_ns = ticks[0] * 1e9 / freq;
    double median_ns = ticks[TIMED / 2] * 1e9 / freq;

    char cpu[64] = "?";
    size_t len = sizeof cpu;
    sysctlbyname("machdep.cpu.brand_string", cpu, &len, NULL, 0);
    printf("%s, counter %.0f MHz\n", cpu, freq / 1e6);
    printf("sum of %d doubles = %.10f\n", N, sum_doubles(a, N));
    printf("best %.0f ns (%.3f ns/elem), median %.0f ns, over %d timed runs after %d warm-ups\n",
           best_ns, best_ns / N, median_ns, TIMED, WARMUP);
    (void)sink;
    return 0;
}

// counters.s (AArch64): read the fixed-frequency time counter from user mode, and a small workload to time.
        .text

// uint64_t read_counter(void): the current value of the virtual count register.
        .globl  _read_counter
_read_counter:
        isb                             // finish earlier work before sampling, so the reading is not early
        mrs     x0, cntvct_el0          // CNTVCT_EL0: a fixed-frequency counter, readable in user mode here
        ret

// uint64_t counter_freq(void): how many counter ticks make one second.
        .globl  _counter_freq
_counter_freq:
        mrs     x0, cntfrq_el0          // CNTFRQ_EL0: the counter's frequency in Hz
        ret

// double sum_doubles(const double *a, long n): the fixed workload the benchmark times.
        .globl  _sum_doubles
_sum_doubles:
        movi    d0, #0
        mov     x2, #0
1:      cmp     x2, x1
        b.ge    2f
        ldr     d1, [x0, x2, lsl #3]
        fadd    d0, d0, d1
        add     x2, x2, #1
        b       1b
2:      ret

1. Lower the warm-up count to zero and raise the timed runs to 50, then compare the best and the median with the original. Which moves more, and why?
2. Change the workload from 4,096 doubles to 400,000 and predict, before you run, whether the nanoseconds per element rise (the array no longer fits in the fastest cache).

The exercise writes the benchmark’s report step: drop the warm-ups, then return the best and the median of the timed runs.

def summarise(times_ns, warmups):
    """Drop the first `warmups` runs, then return (best, median) of the timed runs."""
    return 0, 0  # replace with your code

"""Sample tests: a run like the benchmark's 3 warm-ups and 15 timed runs, and the edges of a report."""

from report import summarise

BENCHMARK = [9000, 4100, 3000,  # the 3 warm-ups
             2375, 2250, 2400, 2375, 2500, 2333, 2291, 2375, 2416, 2458, 2291, 2375, 2541, 2333, 2750]


def test_a_run_like_the_benchmark():
    assert summarise(BENCHMARK, 3) == (2250, 2375)


def test_the_example_of_the_prompt():
    assert summarise([9000, 4100, 3000, 2400, 2250, 2375], 3) == (2250, 2375)


def test_an_even_number_of_timed_runs():
    assert summarise([10, 4, 3, 1, 2], 1) == (1, 2.5)


def test_warm_ups_faster_than_every_timed_run_are_still_left_out():
    assert summarise([1, 7, 9, 8], 1) == (7, 8)


def test_no_warm_ups_and_one_run():
    assert summarise([5], 0) == (5, 5)


def test_the_callers_list_is_left_as_it_was():
    times = [30, 10, 20]
    summarise(times, 0)
    assert times == [30, 10, 20]

Two benchmarks that lie, and the fix for each Pro: opens with a Pro, Premium or Ultimate pass

Three ISAs, three ways to read a counter, and their limits Pro: opens with a Pro, Premium or Ultimate pass

A profiler and a model: perf and llvm-mca Pro: opens with a Pro, Premium or Ultimate pass

Interview questions

A warm-up: you time a function with rdtsc, divide the ticks by the clock speed you looked up, and report cycles. What is wrong with that?

The time-stamp counter is invariant on current x86-64: it ticks at a fixed reference rate, not at the core’s clock, so dividing its ticks by a nominal clock speed does not give the core cycles the code took. The core’s clock is not even constant: it changes with the power state and turbo, so there is no single number to divide by. What rdtsc gives honestly is elapsed time: ticks divided by the invariant rate are seconds. For core cycles, read the core’s cycle counter through the kernel, or use a tool such as perf.

Three deeper questions follow: a counter that disagrees with a profiler, trusting a model, and getting core cycles at all:

Key takeaways

• A trustworthy benchmark uses fixed inputs, warms up, reports the best and median of many runs, and names the hardware and the counter.
• A fixed-frequency counter, such as AArch64’s CNTVCT_EL0, measures elapsed time at a steady rate, not core cycles, because the core clock varies with the power state and the core.
• A counter’s frequency is its unit, not how often its value changes, and a call whose result nobody uses may be compiled away: check both before you trust a short timing.
• Cycles come from a cycle counter the kernel guards, from perf on Linux, or from a model such as llvm-mca, and a reported number names which one it came from.