Data Formats (JSON, YAML, TOML, XML, CSV) Module 6 – TOML 1.1 and other configuration formats
TOML in practice: pyproject.toml and Cargo.toml
Read and check pyproject.toml and Cargo.toml with tomllib, change a value without losing a comment, and convert TOML to JSON and back past null, dates and inf.
What you will learn
- Read pyproject.toml and Cargo.toml with tomllib and check the rules their specifications set
- Change one value of a TOML file without losing its comments or layout, and prove it by parsing again
- Choose between a writer such as Tomli-W and a style-preserving library such as TOML Kit for a change
- Convert TOML to JSON and JSON to TOML, deciding on purpose what happens to null, dates, inf and nan
Before you start
What you’ll learn · Before you start
What you will learn
- Read pyproject.toml and Cargo.toml with tomllib and check the rules their specifications set
- Change one value of a TOML file without losing its comments or layout, and prove it by parsing again
- Choose between a writer such as Tomli-W and a style-preserving library such as TOML Kit for a change
- Convert TOML to JSON and JSON to TOML, deciding on purpose what happens to null, dates, inf and nan
Before you start
On this page
The bookshop’s stock service has a pyproject.toml of 27 lines, and its release script raises the version before
every release. Done one way, the script changes one line and keeps both comments; done another way, it replaces six
lines with two and drops both comments, although the data that tomllib reads back is right either time. TOML files
in real projects are read by tools and edited by hand, so a program that touches them must respect both readers.
You will read pyproject.toml and Cargo.toml with tomllib and check the rules their specifications set, compare
your check with what Cargo itself reports, change one value without losing a comment, choose a library for the change,
and convert between TOML and JSON where null, dates and inf have no form on the other side.
pyproject.toml has three tables, and each has its own reader
A pyproject.toml holds three tables that the
pyproject.toml specification defines.
[build-system] says what is needed to build the project. [project] holds its metadata: the name, the version and
the dependencies, each a dependency specifier: A string that names a Python package and the versions a project accepts, such as httpx>=0.27. The dependencies array of pyproject.toml holds them. such as httpx>=0.27. And
[tool] holds one sub-table per tool, such as [tool.ruff], which the specification reserves for the project that
owns that name on PyPI. Here is the stock service’s file:
# The stock service of the bookshop's branches
[build-system]
requires = ["hatchling>=1.27"]
build-backend = "hatchling.build"
[project]
name = "shelf"
version = "1.4.2" # keep in step with CHANGELOG.md
description = "Stock service of the bookshop's branches"
requires-python = ">=3.12"
dependencies = [
"httpx>=0.27",
"pydantic>=2.9,<3",
]
[project.optional-dependencies]
postgres = ["psycopg[binary]>=3.2"]
docs = ["mkdocs>=1.6"]
[project.scripts]
shelf = "shelf.cli:main"
[tool.pytest.ini_options]
testpaths = ["tests"]
[tool.ruff]
line-length = 100Each table has a different reader, and none of them reads all three. Step through an install of the project and one test run:
The drawing shows the whole scene; its 6 steps are listed below.
Every step as text
- An install starts. Someone runs pip install . in the project folder. The installer cannot build the project itself, so it first reads [build-system] to learn what the build needs.
- requires and build-backend. requires lists the packages the build needs, here hatchling>=1.27, and the installer installs them for the build. build-backend names the code to call: hatchling.build.
- The back-end is called. The installer asks the back-end to build a wheel. From here on, the back-end is the program that reads the project's metadata.
- [project] becomes metadata. The back-end reads [project]: name, version, requires-python, dependencies and the optional dependencies become the metadata inside the wheel.
- Dependencies are installed. The installer reads that metadata and installs httpx and pydantic. An extra such as postgres is installed only when asked for, as in pip install ".[postgres]".
- A tool reads its own table. Later, pytest reads [tool.pytest.ini_options] and ruff reads [tool.ruff], each from the same file and each only its own table. The installer and the back-end never read them.
The installer reads [build-system], the build back-end: The library that builds a Python project into a wheel or a source distribution, such as hatchling or setuptools. pyproject.toml names it in the build-backend key of [build-system]. reads [project], and each
tool reads its own table. The specification sets rules that a program can check after parsing: name is required;
version must be given, either as a string or by listing it in dynamic, and a back-end must raise an error when
version is both given and listed in dynamic; dependencies is an array of strings; and [project] has a fixed
list of keys. Here is a colleague’s first draft of the same file:
# A colleague's first draft of the same file
[build-system]
requires = "hatchling>=1.27"
build-backend = "hatchling.build"
[project]
name = "shelf"
version = "1.4.2"
dynamic = ["version"]
dependencies = { httpx = ">=0.27" }
optional_dependencies = { docs = ["mkdocs>=1.6"] }This program reads both files, lists what the real one declares, and checks those rules on each:
"""Read pyproject.toml with tomllib, list what it declares, and check it against rules of the pyproject.toml specification."""
import tomllib
# The keys the specification allows in [project].
PROJECT_KEYS = {
"authors", "classifiers", "dependencies", "description", "dynamic", "entry-points", "gui-scripts", "import-names",
"import-namespaces", "keywords", "license", "license-files", "maintainers", "name", "optional-dependencies",
"readme", "requires-python", "scripts", "urls", "version",
}
def strings(value):
return isinstance(value, list) and all(isinstance(item, str) for item in value)
def problems(data):
found = []
build = data.get("build-system", {})
if "requires" in build and not strings(build["requires"]):
found.append("[build-system] requires must be an array of strings")
project = data.get("project")
if project is None:
return found + ["there is no [project] table"]
if not isinstance(project.get("name"), str):
found.append("[project] needs a name, as a string")
dynamic = project.get("dynamic", [])
if ("version" in project) == ("version" in dynamic):
found.append("version must be given as a string or listed in dynamic, exactly one of the two")
if "dependencies" in project and not strings(project["dependencies"]):
found.append("dependencies must be an array of strings, such as \"httpx>=0.27\"")
extras = project.get("optional-dependencies", {})
for extra, deps in extras.items():
if not strings(deps):
found.append(f"optional-dependencies.{extra} must be an array of strings")
for key in sorted(project.keys() - PROJECT_KEYS):
found.append(f"[project] has no key {key!r} in the specification")
return found
for name in ["pyproject.toml", "broken.toml"]:
with open(name, "rb") as f:
data = tomllib.load(f)
project = data["project"]
print(f"{name}: {project['name']} {project.get('version', '(dynamic)')}")
if name == "pyproject.toml":
print(" needs:", ", ".join(project["dependencies"]))
for extra, deps in project["optional-dependencies"].items():
print(f" extra {extra}:", ", ".join(deps))
print(" tools configured:", ", ".join(data["tool"]))
found = problems(data)
for problem in found:
print(" problem:", problem)
if not found:
print(" no problems")Output
pyproject.toml: shelf 1.4.2 needs: httpx>=0.27, pydantic>=2.9,<3 extra postgres: psycopg[binary]>=3.2 extra docs: mkdocs>=1.6 tools configured: pytest, ruff no problems broken.toml: shelf 1.4.2 problem: [build-system] requires must be an array of strings problem: version must be given as a string or listed in dynamic, exactly one of the two problem: dependencies must be an array of strings, such as "httpx>=0.27" problem: [project] has no key 'optional_dependencies' in the specification
Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 check_pyproject.py
Runs on this device, in your browser. The first run downloads Python (about 13.5 MB), which is kept for the next runs.
Your run, in this browser
The real file passed. The draft parsed without an error, because every line in it is valid TOML, and still broke four
rules: requires is a string, not an array; the version is both given and dynamic; dependencies is a table, as some
other tools write it; and optional_dependencies, with an underscore, is not a key of [project] at all. Parsing
proves the text is TOML; only a check of the data proves it is a pyproject.toml.
Check the data with a schema
The same four rules fit in a JSON Schema of the parsed data, which also checks every other key and says where each problem is. The JSON Schema module of this track shows how to write one.
Cargo.toml: check it yourself, then ask Cargo
Cargo’s manifest reference says that each package’s Cargo.toml “is written in the TOML format”. Of its
[package] keys, Cargo requires only name; version defaults to 0.0.0, and a package without edition is built
as the 2015 edition. Dependencies may be optional, and a feature (Cargo): A named option of a Rust package, listed in the [features] table of Cargo.toml, that turns on code or optional dependencies, written as dep:name. turns one on by
naming it as dep:serde. Here are two manifests of the bookshop’s stockcheck tool, and a check with tomllib:
TOML (ok.toml) · cargo-check/ok.toml
[package]
name = "stockcheck"
version = "0.3.0"
edition = "2024"
[dependencies]
csv = "1.3"
serde = { version = "1", features = ["derive"], optional = true }
[features]
json = ["dep:serde"]
[[bin]]
name = "stockcheck"
path = "src/main.rs"
[[bin]]
name = "restock-report"
path = "src/report.rs"
required-features = ["json"]TOML (bad.toml) · cargo-check/bad.toml
[package]
name = "stockcheck"
version = "0.3.0"
edition = "2024"
[dependencies]
csv = "1.3"
serde = { version = "1", features = ["derive"] }
[features]
json = ["dep:serde"]
[[bin]]
name = "stockcheck"
path = "src/main.rs"
[[bin]]
name = "stockcheck"
path = "src/report.rs""""Read two Cargo manifests with tomllib and check rules of the Cargo reference: the package name, the edition,
features that name optional dependencies, and unique program names."""
import re
import tomllib
def problems(manifest):
found = []
package = manifest.get("package", {})
name = package.get("name")
if not isinstance(name, str) or not re.fullmatch(r"[A-Za-z0-9_-]+", name):
found.append("[package] needs a name of letters, digits, - and _")
if "edition" not in package:
found.append("no edition: Cargo assumes the 2015 edition")
deps = manifest.get("dependencies", {})
for feature, parts in manifest.get("features", {}).items():
for part in parts:
if part.startswith("dep:"):
dep = deps.get(part[4:])
if dep is None:
found.append(f"feature {feature!r} names {part}, which is not a dependency")
elif not (isinstance(dep, dict) and dep.get("optional") is True):
found.append(f"feature {feature!r} names {part}, but that dependency is not optional")
names = [b.get("name") for b in manifest.get("bin", [])]
for repeated in sorted({n for n in names if names.count(n) > 1}):
found.append(f"two [[bin]] targets are named {repeated!r}")
return found
for name in ["ok.toml", "bad.toml"]:
with open(name, "rb") as f:
manifest = tomllib.load(f)
package = manifest["package"]
programs = ", ".join(b["name"] for b in manifest.get("bin", []))
print(f"{name}: {package['name']} {package['version']}, edition {package['edition']}, programs: {programs}")
found = problems(manifest)
for problem in found:
print(" problem:", problem)
if not found:
print(" no problems")Output
ok.toml: stockcheck 0.3.0, edition 2024, programs: stockcheck, restock-report no problems bad.toml: stockcheck 0.3.0, edition 2024, programs: stockcheck, stockcheck problem: feature 'json' names dep:serde, but that dependency is not optional problem: two [[bin]] targets are named 'stockcheck'
Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 check_cargo.py
Runs on this device, in your browser. The first run downloads Python (about 13.5 MB), which is kept for the next runs.
Your run, in this browser
The check found two problems in bad.toml: the feature json names dep:serde, but serde is not marked
optional = true, and two programs share the name stockcheck. A checker of your own is only as good as its reading
of the rules, so compare it with Cargo itself. This terminal session runs Cargo 1.99.0 on the good manifest, the bad
one, and the bad one with its second program renamed:
"""Prints a terminal session with Cargo 1.99.0 reading the two manifests: each command after "$ ", then a summary of
what Cargo understood, or the error it gave.
Cargo is the pinned Rust toolchain of the registry (1.99.0), run straight from its rustup folder in the same home folder
as the mise installations of the Python that runs this file. `cargo metadata --no-deps --offline` reads a manifest and
reports the package without resolving or downloading anything; CARGO_HOME is a new folder inside this one. The session
prints only the fields it names, never a path of this computer, and it exits with status 1 when a command that should
have worked fails, so a run that could not use Cargo never becomes a record.
"""
import json
import os
import shutil
import subprocess
import sys
import tempfile
from pathlib import Path
# .../home/.local/share/mise/installs/python/3.14.8/bin/python3 -> .../home
HOME = Path(sys.executable).resolve().parents[7]
TOOLCHAINS = sorted((HOME / ".rustup" / "toolchains").glob("1.99.0-*"))
if not TOOLCHAINS:
sys.exit("Cargo 1.99.0 was not found next to this Python")
BIN = TOOLCHAINS[0] / "bin"
def cargo(args, cwd, cargo_home):
env = {"PATH": f"{BIN}{os.pathsep}/usr/bin{os.pathsep}/bin", "CARGO_HOME": cargo_home, "CARGO_TERM_COLOR": "never",
"HOME": cargo_home, "TMPDIR": cargo_home}
return subprocess.run([str(BIN / "cargo"), *args], cwd=cwd, env=env, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
text=True)
home = tempfile.mkdtemp(prefix="cargo-home-", dir=".")
try:
version = cargo(["--version"], ".", home).stdout.split()
print("#", " ".join(version[:2]))
failures = 0
for folder, should_work in [("ok", True), ("bad", False), ("half-fixed", False)]:
print(f"$ cd {folder} && cargo metadata --no-deps --offline --format-version 1")
done = cargo(["metadata", "--no-deps", "--offline", "--format-version", "1"], folder, os.path.abspath(home))
if done.returncode == 0:
package = json.loads(done.stdout)["packages"][0]
programs = [t["name"] for t in package["targets"] if "bin" in t["kind"]]
print(f"package {package['name']} {package['version']}, edition {package['edition']}")
print("programs:", ", ".join(programs))
print("features:", json.dumps(package["features"]))
print("optional dependencies:", ", ".join(d["name"] for d in package["dependencies"] if d["optional"]))
else:
here = os.path.abspath(folder)
print(done.stderr.replace(here + os.sep, "").replace(here, "."), end="")
print(f"[exit status {done.returncode}]")
if (done.returncode == 0) != should_work:
failures += 1
if failures:
sys.exit(1)
finally:
shutil.rmtree(home, ignore_errors=True)Output
# cargo 1.99.0
$ cd ok && cargo metadata --no-deps --offline --format-version 1
package stockcheck 0.3.0, edition 2024
programs: restock-report, stockcheck
features: {"json": ["dep:serde"]}
optional dependencies: serde
$ cd bad && cargo metadata --no-deps --offline --format-version 1
error: failed to parse manifest at `Cargo.toml`
Caused by:
found duplicate binary name stockcheck, but all binary targets must have a unique name
[exit status 101]
$ cd half-fixed && cargo metadata --no-deps --offline --format-version 1
error: failed to parse manifest at `Cargo.toml`
Caused by:
feature `json` includes `dep:serde`, but `serde` is not an optional dependency
A non-optional dependency of the same name is defined; consider adding `optional = true` to its definition.
[exit status 101]
Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 cargo_session.py
Cargo read the good manifest and reported its two programs, its feature and its one optional dependency. It refused the bad one, and named only the first problem it met, the repeated program name; with that one fixed, it named the second, the feature that turns on a dependency that is not optional. Cargo stops at the first problem, while a check of your own can list them all, which is what a CI job wants.
Checkpoint
Check: The tables of pyproject.toml and their rules · Checking a Cargo manifest
Change one line and keep the rest
A release script changes the version from 1.4.2 to 1.5.0. The obvious way, read the file with tomllib, change the
value and write the data out again, cannot keep what the data does not hold. tomllib “does not support writing
TOML”, so writing takes a second library, and the Python documentation names two kinds. Tomli-W describes itself as
“a write-only counterpart to Tomli”: it writes data, and its documentation answers “No” when asked whether it writes
comments. TOML Kit is, in the Python documentation’s words, “style-preserving”: its parser keeps comments,
indentation, white space and the order of elements. Both are
packages from PyPI, so this program uses the standard library only. It changes the version twice, once by editing the
one line in place, and once by writing the whole file again from its data, as any writer from data must:
"""Change the version in pyproject.toml twice: one line edited in place, then the whole file written again from its data."""
import difflib
import json
import re
import tomllib
# A table header, [name] or [[name]]: either one ends the table above it.
HEADER = re.compile(r"^\s*\[\[?([^\[\]]+)\]\]?\s*(?:#.*)?$")
VERSION = re.compile(r"""^(\s*version\s*=\s*)(["'])([^"']*)\2(.*)$""", re.DOTALL)
def set_project_version(text, new):
"""Rewrites the value of the one version line in [project]; every other character stays as it was."""
out, table, done = [], None, False
for line in text.splitlines(keepends=True):
header = HEADER.match(line)
if header:
table = header.group(1).strip()
match = VERSION.match(line)
if table == "project" and match and not done:
line = f"{match.group(1)}{match.group(2)}{new}{match.group(2)}{match.group(4)}"
done = True
out.append(line)
if not done:
raise ValueError("no version line in [project]")
return "".join(out)
def dump_toml(data, prefix=()):
"""A small writer that works from data alone, as every TOML writer must: values first, then one header per table."""
lines = []
tables = {k: v for k, v in data.items() if isinstance(v, dict)}
for key, value in data.items():
if key not in tables:
lines.append(f"{key} = {json.dumps(value, ensure_ascii=False)}")
for key, value in tables.items():
path = (*prefix, key)
if any(not isinstance(v, dict) for v in value.values()):
lines.append(f"\n[{'.'.join(path)}]")
lines.extend(dump_toml(value, path))
return lines
with open("pyproject.toml", encoding="utf-8") as f:
text = f.read()
before = tomllib.loads(text)
edited = set_project_version(text, "1.5.0")
after = tomllib.loads(edited)
after["project"]["version"] = before["project"]["version"]
print("edited in place: data the same apart from the version:", after == before)
print("".join(difflib.unified_diff(text.splitlines(True), edited.splitlines(True), "before", "after", n=0)), end="")
before["project"]["version"] = "1.5.0"
rewritten = "\n".join(dump_toml(before)).lstrip() + "\n"
print("written again: tomllib reads it back:", tomllib.loads(rewritten) == before)
print("".join(difflib.unified_diff(text.splitlines(True), rewritten.splitlines(True), "before", "written again", n=0)), end="")Output
edited in place: data the same apart from the version: True --- before +++ after @@ -8 +8 @@ -version = "1.4.2" # keep in step with CHANGELOG.md +version = "1.5.0" # keep in step with CHANGELOG.md written again: tomllib reads it back: True --- before +++ written again @@ -1 +0,0 @@ -# The stock service of the bookshop's branches @@ -8 +7 @@ -version = "1.4.2" # keep in step with CHANGELOG.md +version = "1.5.0" @@ -11,4 +10 @@ -dependencies = [ - "httpx>=0.27", - "pydantic>=2.9,<3", -] +dependencies = ["httpx>=0.27", "pydantic>=2.9,<3"]
Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 bump.py
Runs on this device, in your browser. The first run downloads Python (about 13.5 MB), which is kept for the next runs.
Your run, in this browser
The edit in place changed line 8 only, kept the comment after the version, and the program proved it by parsing both
texts: the data is the same apart from the version. The rewrite was valid TOML too, and tomllib read the same data
back, but it lost the comment at the top and the one after the version, and it put the list of dependencies on one
line. Both comments were for people, and the parse cannot tell that they are gone.
A line edit is safe only within its limits: one simple key = value line, inside the table you meant, with a parse
before and after to compare. The [tool] tables may have version keys of their own, which is why bump.py tracks
the table it is in. For anything larger, adding a table or editing a list, use a style-preserving library. Answer
three questions about your own file to see which way fits:
Answer the questions above: the table then marks what fits your case and says why the rest does not.
| Way to change it | Keeps comments and layout | Handles any change | Needs no extra package | Output always parses |
|---|---|---|---|---|
| TOML Kit (tomlkit) Works from: the document with its text | Good style-preserving, as the Python documentation describes it | Good adds and removes keys and tables by name | Poor a package from PyPI | Good edits a parsed document, not raw text |
| One line edited, then checked Works from: the lines of the file | Good bump.py changed 1 line and kept both comments | Poor one simple key = value line at a time | Good tomllib and the standard library only | Mixed only when you parse again and compare, as bump.py did |
| Tomli-W, written from data Works from: the data alone | Poor a writer from data lost both comments and the list layout in bump.py | Good writes whatever data it is given | Poor a package from PyPI | Good writes TOML from data, so it always parses |
How the answers decide
- “Do people edit the file by hand and keep comments in it?” Yes, people edit it: rules out Tomli-W, written from data. A writer that works from data cannot keep comments or layout, because the data holds none: the rewrite in bump.py lost them.
- “Do you change a few values, or produce the whole file?” The whole file: rules out One line edited, then checked. A line edit changes one simple line; producing a whole file needs a writer or a document library.
- “May the program install a package from PyPI?” No, the standard library only: rules out TOML Kit (tomlkit) and Tomli-W, written from data. Both are packages from PyPI. The standard library reads TOML with tomllib and has no writer.
- “Do you change a few values, or produce the whole file?” A few values and “May the program install a package from PyPI?” No, the standard library only: adds a caveat to One line edited, then checked. Parse the file again after the edit and compare the data, so a wrong line never ships.
Key idea
Data has no comments, so a program that writes a file from its data loses them: to change a file that people edit, change only its text, with a style-preserving library or a checked line edit.
TOML to JSON and back: null, dates and inf need a decision
The two formats do not hold the same values. JSON has null, which TOML lacks. TOML has four date and time types and
the floats inf and nan, which JSON lacks: RFC 8259 does not permit “Infinity and NaN”. A converter has to decide
what happens to each, and this program makes every decision visible:
TOML · convert/sale.toml
starts = 2026-11-02T10:00:00+05:30
order_id = 9007199254740993
max_discount = infJSON · convert/export.json
{
"branch": "Kochi",
"manager": null,
"opens": "10:00:00",
"limits": { "max_orders": 120, "discount": null },
"printers": [
{ "name": "counter", "tray": null },
{ "name": "office" }
]
}"""TOML to JSON and JSON to TOML, with each value that has no form on the other side handled on purpose."""
import json
import math
import tomllib
# ---- TOML to JSON: dates have no JSON type, and inf has no JSON form.
with open("sale.toml", "rb") as f:
sale = tomllib.load(f)
try:
json.dumps(sale)
except TypeError as e:
print("json.dumps:", e)
print("default=str:", json.dumps(sale, default=str))
try:
json.dumps(sale, default=str, allow_nan=False)
except ValueError as e:
print("allow_nan=False:", e)
# ---- JSON to TOML: TOML has no null, and a document is a table.
def to_toml(data, nulls="drop"):
"""TOML text for a JSON object: tables, arrays of tables and values. nulls="drop" leaves out keys whose value is
null and lists them; nulls="error" refuses them. A null inside an array is always refused: an array cannot skip."""
if not isinstance(data, dict):
raise ValueError("a TOML document is a table: put the JSON array or value under a key first")
dropped = []
def value(v, path):
if v is None or (isinstance(v, float) and not math.isfinite(v)):
raise ValueError(f"{path}: TOML has no form for {v!r} in this place")
if isinstance(v, list):
return "[" + ", ".join(value(x, f"{path}[{i}]") for i, x in enumerate(v)) + "]"
if isinstance(v, dict):
return "{ " + ", ".join(f"{json.dumps(k)} = {value(x, f'{path}.{k}')}" for k, x in v.items()) + " }"
return json.dumps(v, ensure_ascii=False) if not isinstance(v, bool) else str(v).lower()
def table(t, path, header):
lines = [header] if header else []
subs = []
for k, v in t.items():
p = f"{path}.{k}" if path else k
if v is None:
if nulls == "error":
raise ValueError(f"{p}: TOML has no null")
dropped.append(p)
elif isinstance(v, dict):
subs.append(("table", k, v))
elif isinstance(v, list) and v and all(isinstance(x, dict) for x in v):
subs.append(("array", k, v))
else:
lines.append(f"{k} = {value(v, p)}")
for kind, k, v in subs:
p = f"{path}.{k}" if path else k
if kind == "table":
lines += ["", *table(v, p, f"[{p}]")]
else:
for i, item in enumerate(v):
lines += ["", *table(item, f"{p}[{i}]", f"[[{p}]]")]
return lines
return "\n".join(table(data, "", None)) + "\n", dropped
with open("export.json", encoding="utf-8") as f:
exported = json.load(f)
text, dropped = to_toml(exported)
print(text, end="")
print("left out (null):", ", ".join(dropped))
def without_nulls(v):
if isinstance(v, dict):
return {k: without_nulls(x) for k, x in v.items() if x is not None}
if isinstance(v, list):
return [without_nulls(x) for x in v]
return v
print("tomllib reads back the same data, minus the nulls:", tomllib.loads(text) == without_nulls(exported))
for nulls, data in [("error", exported), ("drop", [exported])]:
try:
to_toml(data, nulls)
except ValueError as e:
print(f"nulls={nulls!r}:", e)Output
json.dumps: Object of type datetime is not JSON serializable
default=str: {"starts": "2026-11-02 10:00:00+05:30", "order_id": 9007199254740993, "max_discount": Infinity}
allow_nan=False: Out of range float values are not JSON compliant: inf
branch = "Kochi"
opens = "10:00:00"
[limits]
max_orders = 120
[[printers]]
name = "counter"
[[printers]]
name = "office"
left out (null): manager, limits.discount, printers[0].tray
tomllib reads back the same data, minus the nulls: True
nulls='error': manager: TOML has no null
nulls='drop': a TOML document is a table: put the JSON array or value under a key first
Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 convert.py
Runs on this device, in your browser. The first run downloads Python (about 13.5 MB), which is kept for the next runs.
Your run, in this browser
Going to JSON, json.dumps refused the offset date-time. With default=str it wrote the date as text, kept the large
order id, and wrote Infinity, which is not JSON: a strict reader refuses the whole document. With allow_nan=False,
Python’s json module refused inf instead, which is the safer default, because the problem is then found where it
starts.
Going to TOML, the export had three nulls, and the converter left each out and listed it by path: manager,
limits.discount and printers[0].tray. Dropping is a choice, so it is printed; with nulls="error" the same export
was refused at manager. A null inside an array has no choice at all, because an array cannot skip an item, and a JSON
array at the top level was refused because a TOML document is a table. Last, opens stayed the string "10:00:00":
JSON had no time to give it, and turning such strings into TOML times is one more decision to make on purpose. Each
value that has no form on the other side needs one of these decisions:
| JSON or TOML value | Has no form in | A decision that keeps the data honest |
|---|---|---|
null |
TOML | Leave the key out and list it, or refuse |
| An offset date-time | JSON | An RFC 3339 string with its offset, from isoformat() |
inf or nan |
JSON | Refuse, or agree on a string with the reader |
| An integer past 2⁵³ | JavaScript’s JSON reader | A string of its digits |
Checkpoint
Check: Changing a file without losing its text · Converting between TOML and JSON
Practise: bump a version and change nothing else
The exercise is the release script of this lesson, done properly: raise the [project] version by a major, minor or
patch step and leave every other character as it was, a [tool] table’s own version key included. The sample tests
compare the whole text:
Exercise · Medium · Python
Bump the version of pyproject.toml and change nothing else
A release script must raise the version of a project before each release. Write bump_version(text, part) in bump.py. It takes the text of a pyproject.toml and returns the same text with the version of the [project] table raised, and with every other character, comments and layout included, exactly as it was:
partis"major","minor"or"patch":1.4.2becomes2.0.0,1.5.0or1.4.3. Any otherpartraisesValueError.- The version is a line
version = "X.Y.Z"inside the[project]table, in double or single quotes, possibly with a comment after it. Keep the quotes and the comment as they are. - Other tables may have a
versionkey of their own, such as a tool's settings: leave those lines alone. - When
[project]has no such line, because the version is listed indynamic, or when the version is not three whole numbers, raiseValueError.
You may use tomllib to read the current version, and you should: parse the result again to check that only the version changed.
Starter code · bump.py
def bump_version(text, part):
"""The text of a pyproject.toml with the [project] version raised by part: major, minor or patch."""
return textThe sample tests · test_bump.py
import tomllib
from bump import bump_version
PYPROJECT = '''# The stock service of the bookshop's branches
[build-system]
requires = ["hatchling>=1.27"]
build-backend = "hatchling.build"
[project]
name = "shelf"
version = "1.4.2" # keep in step with CHANGELOG.md
dependencies = [
"httpx>=0.27",
]
[tool.release]
version = "1.4.2"
'''
def test_patch():
"""patch raises the last number and nothing else changes"""
assert bump_version(PYPROJECT, "patch") == PYPROJECT.replace('version = "1.4.2" #', 'version = "1.4.3" #')
def test_minor_and_major():
"""minor and major set the numbers after them to 0"""
assert bump_version(PYPROJECT, "minor") == PYPROJECT.replace('version = "1.4.2" #', 'version = "1.5.0" #')
assert bump_version(PYPROJECT, "major") == PYPROJECT.replace('version = "1.4.2" #', 'version = "2.0.0" #')
def test_other_tables_untouched():
"""the version of [tool.release] stays 1.4.2"""
data = tomllib.loads(bump_version(PYPROJECT, "minor"))
assert data["project"]["version"] == "1.5.0"
assert data["tool"]["release"]["version"] == "1.4.2"
def test_single_quotes_and_two_digit_numbers():
"""single quotes stay single, and 1.9.10 becomes 1.10.0 for minor"""
text = "[project]\nname = 'shelf'\nversion = '1.9.10'\n"
assert bump_version(text, "minor") == "[project]\nname = 'shelf'\nversion = '1.10.0'\n"
def test_dynamic_version_is_refused():
"""a version listed in dynamic has no line to change"""
text = '[project]\nname = "shelf"\ndynamic = ["version"]\n'
try:
bump_version(text, "patch")
except ValueError:
return
raise AssertionError("a dynamic version was accepted")
def test_array_of_tables_header_ends_project():
"""a [[...]] header ends [project] too, so a version line below it is not the project's"""
text = '[project]\nname = "shelf"\ndynamic = ["version"]\n\n[[tool.release.targets]]\nversion = "1.4.2"\n'
try:
bump_version(text, "patch")
except ValueError:
return
raise AssertionError("the version of a [[tool.release.targets]] item was taken for the project's")
def test_unknown_part_is_refused():
"""only major, minor and patch are parts"""
try:
bump_version(PYPROJECT, "build")
except ValueError:
return
raise AssertionError("part 'build' was accepted")A hint
Walk through the lines with text.splitlines(keepends=True), so that joining them gives back the text exactly. Keep the name of the table you are in: a line that starts with [ is a header, [name] or [[name]], and either kind ends the table above it. In the [project] table, a regular expression such as ^(\s*version\s*=\s*)(["'])(\d+)\.(\d+)\.(\d+)\2(.*)$ with re.DOTALL finds the line and keeps everything around the number.
Results of the sample tests
| Test | Result | Details |
|---|
What your code printed
The sample tests run on this device, in your browser (Pyodide): nothing is sent to mysmartcopilot.com. The first run downloads Python (about 13.5 MB), which is kept for the next runs. A check in your browser is feedback for you, not proof that the code is right for every input.
Then review the four ideas, from the three tables of pyproject.toml to the converter’s decisions, one question at a time; a wrong answer explains itself, and the next question goes back to the weakest idea:
Practice that adapts to you
Practise this lesson
14 questions about 4 ideas of this lesson. Each answer is explained at once, and what you get right is kept in this browser.
Interview questions
A warm-up for freshers and 2–5 years: what goes in the three tables of pyproject.toml, and who reads each?
[build-system] says how to build the project: requires lists the packages the build needs and build-backend
names the back-end, such as hatchling.build; the installer reads it first. [project] holds the metadata that the
back-end turns into the package’s own metadata: name, which is required, version, given or listed in dynamic,
dependencies as an array of dependency specifiers, optional-dependencies as a table of such arrays, and a few more
keys from a fixed list. [tool] holds one sub-table per tool, such as [tool.ruff], and each tool reads only its own.
A file can be valid TOML and still break these rules, so a CI check reads it and tests the data.
Two harder questions start from this lesson: a release script whose diff is too large, and a JSON export with nulls that must become TOML. Sketch your answer to each before you open its first step:
Model answers open with any pass: ₹79 for 1 week ($2.99), one payment that never renews.
-
Mid to senior
Your release job raises the version in pyproject.toml. A teammate's version of the job reads the file with tomllib, changes the version and writes the file again with a TOML writer, and the release diff shows six lines gone and two comments lost. Design a bump that changes one line, and prove that it did.
-
Mid to senior
A settings export arrives as JSON, with null values in tables and inside arrays, and a tool needs it as TOML. What do you do with each null, and how do you show that the conversion lost nothing that matters?
Flashcards · 6 cards
Recall it, then check
Say the answer to yourself before you look: recalling it is what makes it stick.
-
Who reads each table of pyproject.toml?
The installer reads [build-system], the build back-end reads [project], and each tool reads only its own [tool.name] table.
-
Which [project] key of pyproject.toml is always required?
name. version is either given or listed in dynamic, and dependencies is an array of dependency specifiers.
-
What does Cargo require of a feature that names dep:x?
That the dependency x is optional. Cargo stops at the first such problem, so a check of your own can list them all.
-
Why does a TOML writer lose the comments of the file it rewrites?
It writes from parsed data, which has no comments or layout. To change a file people edit, use TOML Kit, which keeps the style, or a line edit checked by parsing again.
-
Can Python's tomllib write TOML?
No, it only reads. Writing needs a library such as Tomli-W, which works from data, or TOML Kit, which keeps the style.
-
Which values need a decision when converting between TOML and JSON?
JSON's null, which TOML lacks, and TOML's dates, inf and nan, which JSON lacks. Decide each on purpose, list what is left out, and refuse what cannot be carried.
Key takeaways
pyproject.tomlhas three tables with three readers: the installer reads[build-system], the back-end[project]and each tool its own[tool]sub-table; parse the file, then check its data against the specification’s rules.Cargo.tomlrequires only a name, and a feature that namesdep:xneedsxto be optional; Cargo stops at the first problem, so a check of your own can list them all.- A writer works from data and loses comments and layout; change a file that people edit by its text, with a style-preserving library such as TOML Kit or a line edit checked by parsing again.
- JSON has null and TOML does not; TOML has dates,
infandnanand JSON does not: decide each case on purpose, list what you leave out, and refuse what you cannot carry.
Terms in this lesson
- build back-end
- The library that builds a Python project into a wheel or a source distribution, such as hatchling or setuptools. pyproject.toml names it in the build-backend key of [build-system].
- dependency specifier
- A string that names a Python package and the versions a project accepts, such as httpx>=0.27. The dependencies array of pyproject.toml holds them.
- feature (Cargo)
- A named option of a Rust package, listed in the [features] table of Cargo.toml, that turns on code or optional dependencies, written as dep:name.
References
- pyproject.toml specification (Python Packaging Authority)
- Dependency specifiers (Python Packaging Authority)
- PEP 517: A build-system independent format for source trees (Python Software Foundation)
- The Manifest Format (The Cargo Book) (The Rust Project)
- Features (The Cargo Book) (The Rust Project)
- tomllib: Parse TOML files (Python 3.14) (Python Software Foundation)
- TOML Kit documentation (TOML Kit project)
- Tomli-W (Python Package Index) (Tomli-W project)
- json: JSON encoder and decoder (Python 3.14) (Python Software Foundation)
- RFC 8259: The JavaScript Object Notation (JSON) Data Interchange Format (IETF)
Related tools
Report a problem with this lesson
Kept only in this browser. Your Learn progress