Fakes for machine, time and network

A greenhouse controller switches a pump with machine.Pin, reads the soil through machine.ADC and times the pump with time.ticks_ms(), and it has a bug that shows only when the tick counter wraps, once every 12.4 days. Nobody waits 12 days beside a board for a test; with a fake clockfake clock: An object with ticks_ms(), ticks_add() and ticks_diff() that a test moves forward by hand. It keeps ticks modulo 2**30 like a 32-bit board, so it can start just before the wrap., the wrap arrives in the first millisecond.

This lesson builds fakes for machine, time and network, small objects with the same methods as the real ones, and the rules that keep them honest. The programs run on CPython and are written in the part of Python that MicroPython shares, as host tests are.

Pass the hardware in, and a fake can take its place

There are two ways to put a fakefake: A small stand-in for a module or an object, used in tests: a fake machine module whose I2C class returns prepared bytes lets a driver run without the hardware. More in “Imports, modules and packages on a device” under board code. One is to shadow the module: a file named machine.py earlier on the path replaces the built-in machine on the unix port and on a board, because machine is an extensible module, and CPython has no machine of its own. Shadowing is fragile, though. A file named time.py wins on MicroPython and loses on CPython, whose time is built into the interpreter, and a fake file that reaches a board replaces the real module there. The other way is to pass the hardware in, the dependency injectiondependency injection: Giving an object the things it works with, such as a bus, as arguments instead of letting it create them. A driver that takes its bus this way runs on any bus, a fake one in a test included. More in “Write a reusable device driver class” of the drivers lesson: the controller receives its pin, its ADC and its clock as arguments and creates none of them, so on the board main.py passes Pin(15, Pin.OUT), ADC(26) and the time module itself, and a test passes fakes. Follow the controller through both:

Fakes for machine are the first kind a board’s tests need: a pin that remembers its levels, an ADC that returns prepared readings, a bus that answers from a table. Each copies only the real module’s methods that the code needs and adds none of its own, so a call that the board’s module does not offer fails in the test as well. Time and network follow the same rule. A fake network module and WLAN return the documented status codes in a scripted order, a fake I2C bus raises the board’s errors on chosen calls, and one contract testcontract test: A test of what code relies on from a module, such as a bus that returns bytes of the length asked for, run against the fake on a computer and against the real module on a board., run on the fake and on the real module, keeps each fake true to the board.

A fake clock brings the tick wrap into the test

On the rp2 and esp32 ports the tick counters wrap at 2³⁰, and time.ticks_diff() gives the right difference across the wrap while the two values are less than half that period apart. The unix port of a 64-bit computer wraps at 2⁶², so a test there never meets the wrap a Pico meets every 12.4 days. A fake clock fixes both: it keeps ticks modulo 2³⁰, has ticks_ms(), ticks_add() and ticks_diff(), and advance() moves it on at once. Step through the controller’s bug, the elapsed time computed as plain subtraction, with the start one second before the wrap:

PERIOD = 1 << 30
HALF = PERIOD // 2


def diff(a, b):
    return (a - b + HALF) % PERIOD - HALF


started = PERIOD - 1000
for after in (10_000, 31_000):
    now = (started + after) % PERIOD
    wrong = now - started
    right = diff(now, started)
    print(after, wrong, right)

10000 -1073731824 10000
31000 -1073710824 31000

The program below runs three tests on two versions of the controller, the right one and the one that subtracts, with the fake clock started at 0 and at one second before the wrap:

"""Three tests of the greenhouse controller, run on both versions, with the fake clock started at 0 and at one second
before the tick counter wraps. A tiny runner prints each result."""
from fakes import PERIOD, FakeADC, FakeClock, FakePin
from greenhouse import DRY, Greenhouse, GreenhouseWithBug


def run_for(controller, clock, ms, every=1000):
    for _ in range(ms // every):
        controller.step()
        clock.advance(every)


def test_dry_soil_starts_pump(cls, start):
    pump, clock = FakePin(), FakeClock(start)
    cls(pump, FakeADC([DRY + 5000]), clock).step()
    assert pump.value() == 1


def test_wet_soil_stops_pump(cls, start):
    pump, clock = FakePin(), FakeClock(start)
    g = cls(pump, FakeADC([DRY + 5000, DRY + 5000, DRY - 4000]), clock)
    run_for(g, clock, 3000)
    assert pump.history == [0, 1, 0], pump.history


def test_pump_stops_after_30_s(cls, start):
    pump, clock = FakePin(), FakeClock(start)
    run_for(cls(pump, FakeADC([DRY + 5000]), clock), clock, 32_000)
    assert pump.history[:3] == [0, 1, 0], "the pump still ran after 32 s"


for cls in (Greenhouse, GreenhouseWithBug):
    for start in (0, PERIOD - 1000):
        results = []
        for test in (test_dry_soil_starts_pump, test_wet_soil_stops_pump, test_pump_stops_after_30_s):
            try:
                test(cls, start)
                results.append("ok")
            except AssertionError:
                results.append("FAIL " + test.__name__[5:])
        print("%-17s clock from %10d: %s" % (cls.__name__, start, ", ".join(results)))

Greenhouse        clock from          0: ok, ok, ok
Greenhouse        clock from 1073740824: ok, ok, ok
GreenhouseWithBug clock from          0: ok, ok, ok
GreenhouseWithBug clock from 1073740824: ok, ok, FAIL pump_stops_after_30_s

"""A greenhouse pump controller, as it runs on a board. It creates no hardware itself: the pump pin, the soil sensor's
ADC and the clock are passed in. On a board the clock is the time module itself, which has ticks_ms() and
ticks_diff(); in a test it is a fake."""
DRY = 30_000  # read_u16() above this: the soil is dry
MAX_PUMP_MS = 30_000  # never run the pump longer than this in one go


class Greenhouse:
    def __init__(self, pump, soil, clock):
        self.pump, self.soil, self.clock = pump, soil, clock
        self.started = None

    def step(self):
        now = self.clock.ticks_ms()
        if self.started is None:
            if self.soil.read_u16() > DRY:
                self.pump.value(1)
                self.started = now
        elif self.clock.ticks_diff(now, self.started) >= MAX_PUMP_MS or self.soil.read_u16() <= DRY:
            self.pump.value(0)
            self.started = None


class GreenhouseWithBug(Greenhouse):
    """The same controller with the comparison a board meets every 12.4 days: plain subtraction of ticks."""

    def step(self):
        now = self.clock.ticks_ms()
        if self.started is None:
            if self.soil.read_u16() > DRY:
                self.pump.value(1)
                self.started = now
        elif now - self.started >= MAX_PUMP_MS or self.soil.read_u16() <= DRY:
            self.pump.value(0)
            self.started = None

"""Fakes for the three things the greenhouse controller uses: a pin, an ADC and a clock. They copy only the board's
methods that the controller calls and add none of their own, so a call the board does not have fails in the test too.

FakeClock keeps ticks the way MicroPython's 32-bit ports do (extmod/modtime.c and py/mpconfig.h, v1.29.0): they wrap
at 2**30, ticks_add() and ticks_diff() work modulo that period, and ticks_diff() is right while the two values are less
than half a period apart."""
PERIOD = 1 << 30
HALF = PERIOD // 2


class FakePin:
    def __init__(self, value=0):
        self._value = value
        self.history = [value]

    def value(self, v=None):
        if v is None:
            return self._value
        self._value = 1 if v else 0
        self.history.append(self._value)


class FakeADC:
    """read_u16() returns the readings it was given, one per call, then the last one again."""

    def __init__(self, readings):
        self._readings = list(readings)

    def read_u16(self):
        return self._readings.pop(0) if len(self._readings) > 1 else self._readings[0]


class FakeClock:
    def __init__(self, start=0):
        self._now = start % PERIOD

    def ticks_ms(self):
        return self._now

    def ticks_add(self, ticks, delta):
        return (ticks + delta) % PERIOD

    def ticks_diff(self, ticks1, ticks2):
        return ((ticks1 - ticks2 + HALF) % PERIOD) - HALF

    def advance(self, ms):
        self._now = (self._now + ms) % PERIOD

From 0 both versions passed everything, which is what a test with the real clock of a computer sees. Started just before the wrap, the subtracting version kept the pump running past 30 s, and only that test failed: the bug a board meets after 12 days, found in milliseconds.

Check yourself

Write the three fakes the greenhouse tests used, then work through the self-check on passing hardware in and on fake clocks.

PERIOD = 1 << 30
HALF = PERIOD // 2


class Pin:
    def __init__(self, value=0):
        self.history = [value]

    def value(self, v=None):
        return 0


class ADC:
    def __init__(self, readings):
        self.readings = list(readings)

    def read_u16(self):
        return 0


class Clock:
    def __init__(self, start=0):
        self.now = start

    def ticks_ms(self):
        return self.now

    def advance(self, ms):
        self.now += ms

    def ticks_add(self, t, delta):
        return t + delta

    def ticks_diff(self, a, b):
        return a - b

from myfakes import ADC, HALF, PERIOD, Clock, Pin


def test_pin_levels_and_history():
    """value(v) stores 0 or 1, value() reads it, history keeps every level"""
    pin = Pin()
    pin.value(True)
    pin.value(0)
    pin.value(7)
    assert pin.value() == 1
    assert pin.history == [0, 1, 0, 1]


def test_adc_repeats_the_last_reading():
    """readings come in order, then the last one again"""
    adc = ADC([100, 40000])
    assert [adc.read_u16() for _ in range(4)] == [100, 40000, 40000, 40000]


def test_adc_refuses_impossible_readings():
    """65536, -1 and an empty list are not readings an ADC can give"""
    for bad in ([65536], [-1], []):
        try:
            ADC(bad)
        except ValueError:
            continue
        raise AssertionError("ADC(%r) did not raise ValueError" % (bad,))


def test_clock_wraps():
    """advance() wraps at 2**30, like a board's ticks_ms()"""
    clock = Clock(PERIOD - 1000)
    clock.advance(1500)
    assert clock.ticks_ms() == 500


def test_ticks_diff_across_the_wrap():
    """500 ms after a reading taken 1,000 ms before the wrap is 1,500 ms later"""
    clock = Clock()
    assert clock.ticks_diff(500, PERIOD - 1000) == 1500
    assert clock.ticks_diff(PERIOD - 1000, 500) == -1500


def test_ticks_diff_range():
    """half a period apart, ticks_diff() turns negative"""
    clock = Clock()
    assert clock.ticks_diff(HALF - 1, 0) == HALF - 1
    assert clock.ticks_diff(HALF, 0) == -HALF


def test_ticks_add():
    """ticks_add() wraps, and refuses deltas of half a period or more"""
    clock = Clock()
    assert clock.ticks_add(PERIOD - 10, 30) == 20
    assert clock.ticks_add(5, -10) == PERIOD - 5
    for bad in (HALF, -HALF):
        try:
            clock.ticks_add(0, bad)
        except OverflowError:
            continue
        raise AssertionError("ticks_add(0, %d) did not raise OverflowError" % bad)

Fake buses that fail on cue, and a driver's retries proved Pro: opens with a Pro, Premium or Ultimate pass

Fake Wi-Fi that follows each port's status codes Pro: opens with a Pro, Premium or Ultimate pass

Keeping fakes honest: one contract for the fake and the real module Pro: opens with a Pro, Premium or Ultimate pass

Interview questions

Warm-up (fresher): Why pass hardware objects into a class instead of creating them inside it?

Because then the class runs anywhere it is given objects with the right methods. A driver that creates machine.I2C(0) itself can only run where that bus exists, on one kind of board, and a test can replace the bus only by shadowing the machine module, which behaves differently on CPython, the unix port and a board. A driver that receives its bus runs on the board’s hardware bus, on a SoftI2C, and in a test on a fake that returns prepared bytes or raises errors on cue. The same holds for clocks and networks: pass in the time module on the board and a fake clock in a test.

Three longer questions on fakes start from this lesson:

Key takeaways

• Passing pins, buses and clocks into code lets a test give it fakes, while shadowing a module behaves differently on CPython, the unix port and a board.
• A fake copies only the real module’s methods that the code needs and adds none, so a call the board lacks fails in the test too.
• A fake clock that wraps at 2³⁰ and starts just before the wrap finds a tick bug at once, which neither the unix port of a 64-bit computer nor days of waiting find.