Files, safe writes and power cuts

A Pico saves its settings with open('config.json', 'w'), and one day the sensor it reads during the save stops answering: the file is left with the first 32 bytes of the new settings, and at the next boot json.loads() raises ValueError and the board starts from its defaults. On FAT, a power cut during the save can break the file the same way.

Safe writes come down to two patterns: write a new version of a file beside the old one and rename it into place, and give every record of a log its length and a checksum. Both can be proved by trying every point where power cuts could stop a save, and checking what is left.

A save in place can leave a file half-written

open(path, 'w') empties the file before the first byte of the new settings is written, so files saved this way pass through states that hold no valid settings. If an error stops the save before its last write, the file keeps only what was written so far. This program saves twice; during the second save, reading the sensor’s calibration fails:

"""A configuration saved in place, and what an error halfway through the save leaves behind.

open(path, "w") empties the file at once. When the save stops before its last write,
leaving the with block closes the file, and the part already written is what remains.
"""
import json

DEFAULTS = {"ssid": "", "interval": 60, "offset": 0}


def read_offset(sensor_ok):
    """Stands in for reading the sensor's calibration over I2C."""
    if not sensor_ok:
        raise OSError(5, "EIO")  # the sensor stopped answering
    return -12


def save_in_place(path, ssid, interval, sensor_ok=True):
    with open(path, "w") as f:
        f.write('{"ssid": %s, ' % json.dumps(ssid))
        f.write('"interval": %d, ' % interval)
        f.write('"offset": %d}' % read_offset(sensor_ok))


def load(path):
    try:
        with open(path) as f:
            return json.loads(f.read())
    except OSError:
        print("  no file -> defaults")
    except ValueError:
        print("  not valid JSON -> defaults")
    return dict(DEFAULTS)


save_in_place("config.json", "lab", 10)
print("save 1:", load("config.json"))

try:
    save_in_place("config.json", "lab", 30, sensor_ok=False)
except OSError as e:
    print("save 2 stopped: OSError", e.args[0])
with open("config.json") as f:
    print("config.json now holds %r" % f.read())
print("next boot:", load("config.json"))

save 1: {'ssid': 'lab', 'interval': 10, 'offset': -12}
save 2 stopped: OSError 5
config.json now holds '{"ssid": "lab", "interval": 30, '
  not valid JSON -> defaults
next boot: {'ssid': '', 'interval': 60, 'offset': 0}

The with block did what it should and closed the file when the exception left it, and closing is what made those 32 bytes the file. On LittleFSLittleFS: A small filesystem designed for flash: it writes changes to free blocks, commits them in small checked logs and spreads erases over the blocks. The rp2 and esp32 ports of MicroPython format their flash with LittleFS v2. More in “Filesystems: LittleFS, FAT and vfs.mount”, whose documentation says a file’s changes are committed only when it is closed or synced, a power cut in the middle of the writes keeps the old version. An error inside the with block, or a flush() halfway through, commits the half-written file all the same. FAT has no such commit, so a power cut during the save can leave the file without its settings, or damage the filesystem itself, as MicroPython’s documentation warns.

Write a temporary file, then rename it over the old one

The safe pattern never touches config.json until the new version is complete. Write the new settings to config.json.tmp, close it, and only then rename it over config.json with os.rename(). LittleFS’s documentation states that rename is atomicatomic: Done in one step that cannot be seen half-done: after a power cut, an atomic rename has either happened completely or not at all., even if power is lost, so after a cut config.json is the old file or the new one. MicroPython’s FAT driver renames in two steps when the target exists: it removes config.json, then renames the temporary file, so a cut between them leaves only config.json.tmp, complete. Step through a save and the boot that follows it:

The program below is that pattern, with a replay of one save on a model of a filesystem without a commit, which keeps every byte as soon as it is written. It stops the save at every point a cut could, then runs the boot code on what is left:

"""A configuration saved safely: write config.tmp, close it, rename it over config.json.

save() and load() are what a board runs. The replay models a filesystem without a commit,
which applies each operation in order and keeps every byte as soon as it is written: it stops
one save at every point a power cut could, then boots, that is, runs load() on what is left.
"""
import json
import os

DEFAULTS = {"ssid": "", "interval": 60}


def save(cfg, path="config.json"):
    tmp = path + ".tmp"
    with open(tmp, "w") as f:
        f.write(json.dumps(cfg))
    os.rename(tmp, path)  # one atomic step on LittleFS


def read(name):
    try:
        with open(name) as f:
            return json.loads(f.read())
    except (OSError, ValueError):
        return None


def remove(name):
    try:
        os.remove(name)
    except OSError:
        pass


def load(path="config.json"):
    tmp = path + ".tmp"
    cfg = read(path)
    if cfg is not None:
        remove(tmp)  # a save that did not finish: keep the old version
        return cfg
    cfg = read(tmp)
    if cfg is not None:
        os.rename(tmp, path)  # the save stopped after its write: finish it
        return cfg
    remove(tmp)
    return dict(DEFAULTS)


# ---- the replay ------------------------------------------------------------------
OLD = {"ssid": "lab", "interval": 10}
NEW = {"ssid": "lab", "interval": 30}
J, T = "config.json", "config.json.tmp"
data = json.dumps(NEW)

WAYS = {
    "in place": [("open_w", J), ("write", J)],
    "tmp, then rename": [("open_w", T), ("write", T), ("rename",)],
    "tmp, rename on FAT": [("open_w", T), ("write", T), ("remove", J), ("rename",)],
}


def states(steps):
    """Every state a cut can leave: after each step, and after each byte of a write."""
    fs = {J: json.dumps(OLD)}
    yield dict(fs)
    for step in steps:
        if step[0] == "write":
            for k in range(len(data)):
                yield dict(fs, **{step[1]: data[:k]})
            fs[step[1]] = data
        elif step[0] == "open_w":
            fs[step[1]] = ""
        elif step[0] == "remove":
            del fs[step[1]]
        else:  # rename T over J
            fs[J] = fs.pop(T)
        yield dict(fs)


print("one save cut at every point, then load() at boot")
print("%-20s %6s %4s %4s %7s" % ("", "cuts", "old", "new", "broken"))
for way, steps in WAYS.items():
    count = {"old": 0, "new": 0, "broken": 0}
    for fs in states(steps):
        for name in (J, T):
            remove(name)
        for name, text in fs.items():
            with open(name, "w") as f:
                f.write(text)
        cfg = load()
        count["old" if cfg == OLD else "new" if cfg == NEW else "broken"] += 1
    print("%-20s %6d %4d %4d %7d" % (way, sum(count.values()), count["old"], count["new"], count["broken"]))

one save cut at every point, then load() at boot
                       cuts  old  new  broken
in place                 34    1    1      32
tmp, then rename         35   34    1       0
tmp, rename on FAT       36   34    2       0

Every state the pattern can leave, 35 with LittleFS’s rename and 36 with FAT’s, loads the old settings or the new ones, while the save in place breaks 32 of its 34. The new settings win once the rename has run, and on FAT also when only the temporary file is left, which the boot code renames into place.

Other files need more than this pattern. A log only grows, so each record carries its length and a CRC-32, and the reader stops at a torn recordtorn record: A record that a power cut or a reset stopped partway through its write, so that only its first bytes reached the storage. A reader recognises it by a length that runs past the end of the data or a checksum that does not match.. A setting kept in raw flash, without any filesystem, needs two blocks that take turns, so that one always holds a whole copy. Each erase wears a flash block a little, so settings saved every second wear the flash out. Batching the writes in RAM cuts the erases. A replay of every cut point, like the one above, proves such designs safe.

Check yourself

The questions below cover saves in place and the temporary-file pattern, and explain every choice.

Logs that stop cleanly at a torn record Pro: opens with a Pro, Premium or Ultimate pass

Proving a save safe by replaying every cut Pro: opens with a Pro, Premium or Ultimate pass

Flash wear, counted: batching writes Pro: opens with a Pro, Premium or Ultimate pass

Interview questions

Key takeaways

• open(path, 'w') empties a file at once, so a save stopped by an error, or by a power cut on FAT, leaves it without valid settings, and on LittleFS closing or flushing commits whatever is there.
• Safe writes put the new version in config.json.tmp, close it and rename it over config.json, which LittleFS does in one atomic step, even through power cuts.
• At boot, a valid config.json wins and a leftover .tmp is deleted; a missing or broken config.json is replaced by a valid .tmp, which covers FAT’s two-step rename; with neither, the board uses its defaults.