Data Formats (JSON, YAML, TOML, XML, CSV) Module 4 – Querying JSON: JSONPath, jq, JMESPath and SQL
JSONPath (RFC 9535): selectors and segments
Write JSONPath queries the RFC 9535 way: names, wildcards, indexes, slices and descendants, read normalized paths, and see what changed from the 2007 article.
What you will learn
- Select values with name, wildcard, index and slice selectors
- Use child and descendant segments and read the normalized path of every node
- Predict the order, the duplicates and the empty results of a nodelist
- Explain how RFC 9535 differs from the 2007 article and from older libraries
Before you start
What you’ll learn · Before you start
What you will learn
- Select values with name, wildcard, index and slice selectors
- Use child and descendant segments and read the normalized path of every node
- Predict the order, the duplicates and the empty results of a nodelist
- Explain how RFC 9535 differs from the 2007 article and from older libraries
Before you start
On this page
The bookshop’s catalogue is one JSON document with 4 books, 1 gift card and 2 shelf sections, and a report needs
every title, the code of the last book and every price. In JSONPath each of those is one short query,
$.books[*].title, $.books[-1].sku and $..price_paise, and the last of them quietly returns 5 prices, not 4,
because it finds the gift card’s price too.
You will write queries the way RFC 9535, the JSONPath standard, defines them: the root and the segments that make a query, the name, wildcard, index and slice selectors, the descendant segment and the order of its results, and the normalized path that names every node a query returns. You will also see what the standard changed from the 2007 article that started JSONPath, which is why older libraries disagree with it. Filters are the next lesson.
Every query starts at the root and returns a nodelist
A JSONPath query is a path through a document, read from left to right. It starts with $, the root identifier,
which stands for the whole document, and goes on with segment: One step of a JSONPath query. A child segment such as [0] or .title picks children of every node found so far; a descendant segment such as ..title picks from those nodes and everything below them.. RFC 9535 calls
everything a query can select a node: a value together with its place in the document. Each segment takes the nodes
found so far and hands on a new list of nodes, and the list left at the end is the answer, a
nodelist: The result of a JSONPath query: a list of nodes, each a value together with its place in the document. An empty nodelist means that nothing matched, which is not the same as a match whose value is null.. Step through one query on the bookshop’s catalogue:
The drawing shows the whole scene; its 6 steps are listed below.
Every step as text
- Start at the root. The query begins with $, the root identifier. The nodelist so far holds one node: the whole catalogue, whose normalized path is $.
- .books picks one member. .books is shorthand for ['books']. Applied to the one node so far, it picks the member called books: one node, the array of four books.
- [*] takes every element. The wildcard picks every child of each node it is given. The array has four elements, so the nodelist grows to four nodes, $['books'][0] to $['books'][3].
- .series drops three books. Three of the four books have no member called series, so they add nothing, and nothing fails. Only BK-310 has one, and the nodelist shrinks to one node.
- .title finds one title. Applied to the series object, ['title'] picks its title. The query ends with one node, "Kahaniyan", at $['books'][3]['series']['title'].
- The nodelist is the answer. Had no book had a series, the query would have ended with an empty nodelist: a valid answer that says nothing matched, which is neither an error nor null.
Three of the four books had no series member, and they simply dropped out: a selector that finds nothing adds
nothing, and the query goes on with the rest. That holds for the whole language. RFC 9535 says that a valid query
must not produce errors when it runs, while a query that breaks the grammar is refused before the document is read.
So there are two outcomes to tell apart: an empty nodelist, where nothing matched, and a nodelist that holds a node
whose value happens to be null, which is a match like any other.
The programs of this lesson run their queries on this catalogue:
{
"books": [
{
"sku": "BK-101",
"title": "Monsoon Maths",
"lang": "en",
"price_paise": 34900,
"authors": ["Asha Rao"],
"stock": { "Pune": 4, "Kochi": 2 }
},
{
"sku": "BK-205",
"title": "बादल और बूँदें",
"lang": "hi",
"price_paise": 19900,
"authors": ["Ravi Kant"],
"stock": { "Pune": 0, "Indore": 6 }
},
{
"sku": "BK-233",
"title": "The Tea Garden Ledger",
"lang": "en",
"price_paise": 52500,
"authors": ["Meera Das", "Tenzin Lama"],
"stock": { "Guwahati": 3 }
},
{
"sku": "BK-310",
"title": "कोड की कहानियाँ",
"lang": "hi",
"price_paise": 27900,
"authors": ["Sana Iqbal"],
"stock": { "Mysuru": 1, "Pune": 7 },
"series": { "title": "Kahaniyan", "number": 2 }
}
],
"gift_cards": [{ "code": "GC-500", "price_paise": 50000 }],
"sections": {
"new arrivals": ["BK-310", "BK-233"],
"children's": ["BK-101"]
}
}The first program runs twelve queries and prints every node of each nodelist, with its normalized path, the exact place of the node in the document. Press Run, then Edit one of the queries and run it again:
"""Twelve JSONPath queries on the bookshop's catalogue, each printed with every node it selected."""
import json
from jsonpath_mini import query
with open("catalogue.json", encoding="utf-8") as f:
catalogue = json.load(f)
QUERIES = [
"$.books[0].title",
"$['books'][0]['title']",
"$.books[*].title",
"$.books[-1].sku",
"$.books[0, 2].sku",
"$.books[*].authors[0]",
"$.sections['new arrivals'][*]",
"$.sections[\"children's\"][0]",
"$.books[0].stock.*",
"$.books[*].publisher",
"$.books[9].title",
"$.books.title",
]
for q in QUERIES:
nodes = query(q, catalogue)
print(f"{q} {len(nodes)} node{'' if len(nodes) == 1 else 's'}")
for path, value in nodes:
print(f" {path} = {json.dumps(value, ensure_ascii=False)}")Output
$.books[0].title 1 node
$['books'][0]['title'] = "Monsoon Maths"
$['books'][0]['title'] 1 node
$['books'][0]['title'] = "Monsoon Maths"
$.books[*].title 4 nodes
$['books'][0]['title'] = "Monsoon Maths"
$['books'][1]['title'] = "बादल और बूँदें"
$['books'][2]['title'] = "The Tea Garden Ledger"
$['books'][3]['title'] = "कोड की कहानियाँ"
$.books[-1].sku 1 node
$['books'][3]['sku'] = "BK-310"
$.books[0, 2].sku 2 nodes
$['books'][0]['sku'] = "BK-101"
$['books'][2]['sku'] = "BK-233"
$.books[*].authors[0] 4 nodes
$['books'][0]['authors'][0] = "Asha Rao"
$['books'][1]['authors'][0] = "Ravi Kant"
$['books'][2]['authors'][0] = "Meera Das"
$['books'][3]['authors'][0] = "Sana Iqbal"
$.sections['new arrivals'][*] 2 nodes
$['sections']['new arrivals'][0] = "BK-310"
$['sections']['new arrivals'][1] = "BK-233"
$.sections["children's"][0] 1 node
$['sections']['children\'s'][0] = "BK-101"
$.books[0].stock.* 2 nodes
$['books'][0]['stock']['Pune'] = 4
$['books'][0]['stock']['Kochi'] = 2
$.books[*].publisher 0 nodes
$.books[9].title 0 nodes
$.books.title 0 nodes
Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 queries.py
Runs on this device, in your browser. The first run downloads Python (about 13.5 MB), which is kept for the next runs.
Your run, in this browser
The first two queries are one query in dot and bracket notation: .books is shorthand for ['books']. The last
three select nothing, for three different reasons: no book has a publisher, there is no tenth book, and books is
an array, which has no member called title. None of them is an error, and none of them returns null.
Key idea
A query is the root followed by segments; each segment turns the nodelist so far into a new one, and a step that finds nothing adds nothing to it.
Checkpoint
Check: Queries, segments and nodelists
Names, wildcards and indexes pick children
Every segment holds one or more selector: One item inside a JSONPath segment that picks children of a node: a name, the wildcard *, an index, a slice or a filter. A segment may hold several, separated by commas., and each selector picks children of the nodes it is given. Three selectors cover most queries:
| Selector | Selects | Given anything else |
|---|---|---|
Name: ['title'], ["title"] or .title |
the member of an object with that name | nothing: only objects have members |
Wildcard: [*] or .* |
every member value of an object, every element of an array | nothing: strings and numbers have no children |
Index: [0], [-1] |
one element of an array, counted from 0, or from the end when negative | nothing, nor does a position past the end |
The dot form of a name is shorthand that works only for names that start with a letter or _ and go on with
letters, digits and _. Letters beyond ASCII count, so a Devanagari name such as दाम works after a dot, but a name
with a space, an apostrophe or a hyphen needs brackets and quotes, as the catalogue’s sections do: $.sections['new arrivals'] and
$.sections["children's"]. Inside single quotes an apostrophe is written \'; inside double quotes it needs no
escape.
A segment may hold several selectors separated by commas. $.books[0, 2].sku returned BK-101 and BK-233, in the
order of the selectors, and a nodelist keeps duplicates, so [0, 0] would select the first book twice. The wildcard
on an object, $.books[0].stock.*, returned 4 and 2, the stock in Pune and in Kochi. RFC 9535 fixes the order of an
array’s elements but not of an object’s members, so another library may return 2 before 4.
A name on an array
Wrong: $.books.title, expecting the title of every book. It returned 0 nodes, because books is an array and a
name selects members of objects only. Right: $.books[*].title, which steps into every element first.
Checkpoint
Check: Name, wildcard and index selectors
Slices count like Python’s, inside the array’s bounds
An array slice, start:end:step, selects a run of elements: it begins at position start, moves by step, and
stops before it reaches end. A part left out takes a default: the step is 1, and with a positive step the start is
0 and the end is the length of the array. The program applies eleven slices to the six days the shop opens:
"""Array slices on the six days the shop opens: the positions each one selects, by RFC 9535's rules."""
from jsonpath_mini import query
week = {"days": ["Mon", "Tue", "Wed", "Thu", "Fri", "Sat"]}
SLICES = ["[1:4]", "[:2]", "[-2:]", "[::2]", "[::-1]", "[4:1:-1]", "[5:1:-2]", "[-10:2]", "[10:]", "[2:2]", "[1:5:0]"]
for s in SLICES:
nodes = query(f"$.days{s}", week)
positions = [path[path.rindex("[") + 1 : -1] for path, _ in nodes]
shown = " ".join(value for _, value in nodes) or "(nothing)"
print(f"{s:<10} positions {', '.join(positions) or '-':<18} {shown}")
try:
week["days"][::0]
except ValueError as e:
print(f"Python's own days[::0]: ValueError: {e}")Output
[1:4] positions 1, 2, 3 Tue Wed Thu [:2] positions 0, 1 Mon Tue [-2:] positions 4, 5 Fri Sat [::2] positions 0, 2, 4 Mon Wed Fri [::-1] positions 5, 4, 3, 2, 1, 0 Sat Fri Thu Wed Tue Mon [4:1:-1] positions 4, 3, 2 Fri Thu Wed [5:1:-2] positions 5, 3 Sat Thu [-10:2] positions 0, 1 Mon Tue [10:] positions - (nothing) [2:2] positions - (nothing) [1:5:0] positions - (nothing) Python's own days[::0]: ValueError: slice step cannot be zero
Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 slices.py
Runs on this device, in your browser. The first run downloads Python (about 13.5 MB), which is kept for the next runs.
Your run, in this browser
A negative start or end counts from the end, so [-2:] gave Fri and Sat. A negative step walks backwards: [4:1:-1]
began at Fri, position 4, and stopped before position 1, so it gave Fri, Thu and Wed. With a negative step the
defaults turn round too, the start becoming the last position and the end the place before the first, which is why
[::-1] returned all six days in reverse order.
Section 2.3.4 of RFC 9535 gives the exact rules. It adds the length of the array to a negative start or end, then
clamps both into the array, so [-10:2] became positions 0 and 1, Mon and Tue, with no complaint about the −10. A
start past the end, as in [10:], or an empty range, as in [2:2], selects nothing, and so does a step of 0.
Expecting an error for a step of 0
Python refuses a zero step: days[::0] raised ValueError: slice step cannot be zero in the last line of the run.
A JSONPath slice never fails, so $.days[1:5:0] quietly selected nothing. Check the length of the nodelist rather
than waiting for an error.
Checkpoint
Check: Array slices
A descendant segment searches every level, in a fixed order
Two dots make a descendant segment: ..title applies the name selector ['title'] to the node it starts from and to
every node below it, at any depth. It is the quickest way to find a name whose place you do not know, and the easiest
way to find more than you meant:
"""Descendant segments (..) on the catalogue: what they find, and in which order."""
import json
from jsonpath_mini import query
with open("catalogue.json", encoding="utf-8") as f:
catalogue = json.load(f)
def show(q):
nodes = query(q, catalogue)
print(f"{q} {len(nodes)} nodes")
for path, value in nodes:
print(f" {path} = {json.dumps(value, ensure_ascii=False)}")
show("$..title")
show("$..price_paise")
show("$.books[*].price_paise")
show("$.books[3]..title")
show("$.books[1]..*")
show("$.books[0]['sku', 'sku']")
print("$..* selects", len(query("$..*", catalogue)), "nodes: every member value and element in the catalogue")Output
$..title 5 nodes
$['books'][0]['title'] = "Monsoon Maths"
$['books'][1]['title'] = "बादल और बूँदें"
$['books'][2]['title'] = "The Tea Garden Ledger"
$['books'][3]['title'] = "कोड की कहानियाँ"
$['books'][3]['series']['title'] = "Kahaniyan"
$..price_paise 5 nodes
$['books'][0]['price_paise'] = 34900
$['books'][1]['price_paise'] = 19900
$['books'][2]['price_paise'] = 52500
$['books'][3]['price_paise'] = 27900
$['gift_cards'][0]['price_paise'] = 50000
$.books[*].price_paise 4 nodes
$['books'][0]['price_paise'] = 34900
$['books'][1]['price_paise'] = 19900
$['books'][2]['price_paise'] = 52500
$['books'][3]['price_paise'] = 27900
$.books[3]..title 2 nodes
$['books'][3]['title'] = "कोड की कहानियाँ"
$['books'][3]['series']['title'] = "Kahaniyan"
$.books[1]..* 9 nodes
$['books'][1]['sku'] = "BK-205"
$['books'][1]['title'] = "बादल और बूँदें"
$['books'][1]['lang'] = "hi"
$['books'][1]['price_paise'] = 19900
$['books'][1]['authors'] = ["Ravi Kant"]
$['books'][1]['stock'] = {"Pune": 0, "Indore": 6}
$['books'][1]['authors'][0] = "Ravi Kant"
$['books'][1]['stock']['Pune'] = 0
$['books'][1]['stock']['Indore'] = 6
$.books[0]['sku', 'sku'] 2 nodes
$['books'][0]['sku'] = "BK-101"
$['books'][0]['sku'] = "BK-101"
$..* selects 54 nodes: every member value and element in the catalogue
Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 descendants.py
Runs on this device, in your browser. The first run downloads Python (about 13.5 MB), which is kept for the next runs.
Your run, in this browser
$..title found 5 titles, not 4: the fifth is the title of the series that BK-310 belongs to. $..price_paise found
5 prices, the four books’ and the gift card’s 50000, so a report that adds them up counts a gift card as a book.
$.books[*].price_paise names the place and found exactly the four.
The order of the results is fixed by section 2.5.2 of RFC 9535: the segment visits the node it starts from, then the
nodes below it, each node before its descendants and the elements of an array in order, and joins what the selector
found at each visit. Step through $.books[3]..title, the last book searched from top to bottom:
The drawing shows the whole scene; its 6 steps are listed below.
Every step as text
- The book itself first. A descendant segment visits the node it starts from before anything below it, and applies its selector there. ['title'] finds the book's own title: result 1.
- Then its children, in order. Next come the book's member values, here in the document's order, which RFC 9535 leaves open for objects. sku, title, lang and price_paise are strings and numbers with no children, so ['title'] finds nothing in them.
- authors holds no title. authors is an array. A name selector finds nothing in an array, and its one element, "Sana Iqbal", is a string with no children of its own.
- stock holds no title. stock is an object with the members Mysuru and Pune, and neither is called title. The walk visits both numbers, which have no children.
- series has a title. series is an object with a member called title, so ['title'] selects it: result 2, "Kahaniyan".
- The walk ends. series.title and series.number have no children, so the walk is over. The nodelist holds the two titles in the order the walk met them: the book's, then the series'.
The walk met the book’s own title first and the series title last, which is also the order of the two titles in the
program’s run. The same rule explains $.books[1]..*: the book’s six member values come first, then the author
inside authors and the two numbers inside stock. Only the order of an object’s members is left open, as for the
wildcard. Duplicates stay here as everywhere: $.books[0]['sku', 'sku'] returned the same node twice.
Name the place when you know it
Write the path you mean, such as $.books[*].price_paise, and keep .. for documents whose shape you do not
control. When you do use it, print the normalized path of every node it returns before you trust a total.
Checkpoint
Check: Descendant segments and their order
Every node has exactly one normalized path
Every node of a nodelist comes with a normalized path: The one JSONPath form that names a single node exactly, such as $['books'][0]['title']: every name in single quotes and every array position counted from 0. RFC 9535 section 2.7 defines it., the one form of JSONPath that
names a single node exactly. Section 2.7 of RFC 9535 defines it: $, then one pair of brackets for each step, every
name in single quotes and every position as a number counted from 0. $.books[-1] and $.books[3] select the same
node, and its normalized path is $['books'][3] either way.
Inside a name, a normalized path escapes only what it must: an apostrophe as \', a backslash as \\, the control
characters backspace, form feed, line feed, carriage return and tab as \b, \f, \n, \r and \t, and every
other character below U+0020 as \u with four lowercase hexadecimal digits. Everything else stays as it is, a slash,
a tilde or Devanagari letters included:
"""Normalized paths for member names that a dot cannot reach: spaces, quotes, slashes, control characters."""
from jsonpath_mini import query
doc = {
"sections": {"new arrivals": 2, "children's": 1},
"codes": {"a/b": "slash", "m~n": "tilde", "line\nbreak": "newline", "bell\u0007": "control", "दाम": "Hindi"},
}
for q in ["$.sections.*", "$.codes.*", "$.codes['a/b']", "$.codes.दाम", "$['codes']['line\\nbreak']"]:
print(q)
for path, value in query(q, doc):
print(f" {path} ({value})")Output
$.sections.*
$['sections']['new arrivals'] (2)
$['sections']['children\'s'] (1)
$.codes.*
$['codes']['a/b'] (slash)
$['codes']['m~n'] (tilde)
$['codes']['line\nbreak'] (newline)
$['codes']['bell\u0007'] (control)
$['codes']['दाम'] (Hindi)
$.codes['a/b']
$['codes']['a/b'] (slash)
$.codes.दाम
$['codes']['दाम'] (Hindi)
$['codes']['line\nbreak']
$['codes']['line\nbreak'] (newline)
Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 paths.py
Runs on this device, in your browser. The first run downloads Python (about 13.5 MB), which is kept for the next runs.
Your run, in this browser
Because every node has one normalized path, and every normalized path names one node, the path is a safe label for a
value: in a log line, an error report, or the list of differences between two versions of a document. It is also
close to a JSON Pointer: A string that names one value inside a JSON document, such as /items/0/qty, defined by RFC 6901. Inside a name, ~ is written ~0 and / is written ~1. More in “JSON interop: duplicates, key order, I-JSON”, the form of RFC 6901 that JSON Patch uses:
$['codes']['a/b'] is /codes/a~1b there, with / written ~1 and ~ written ~0. Appendix C of RFC 9535 notes
that a normalized path converts to a pointer by its syntax alone, while the other direction needs the document,
because a pointer step such as 0 may be an array position or a member called "0". The second exercise of this
lesson builds that conversion, and the JSON Pointer lesson covers
pointers themselves.
Checkpoint
Check: Normalized paths
RFC 9535 is stricter than the 2007 article
JSONPath began as an article by Stefan Gössner in 2007 and spread through libraries in many languages before the IETF published RFC 9535 in 2024, as a Proposed Standard. The RFC says it is “based on Stefan Gössner’s popular JSONPath proposal”, and it turns that proposal into rules that every implementation must follow exactly. The differences are where older libraries still disagree with it:
| Topic | The 2007 article | RFC 9535 |
|---|---|---|
| Script expressions | ( ) is a “script expression, using the underlying script engine”, as in $..book[(@.length-1)] |
none; the last element is [-1] |
| Filters | ?( ) “applies a filter (script) expression” |
a filter grammar of its own (the next lesson), with optional parentheses |
| No match | the article’s implementation returns false |
an empty nodelist |
| Slices | “borrowed from ES4”, with no exact rules | the exact rules of section 2.3.4 |
| Order and duplicates | not stated | arrays in order, nodes before their descendants, duplicates kept |
| Result paths | paths such as $['store']['book'][0]['author'] |
the same form, defined exactly as normalized paths |
Here are queries written in the article’s style, read by an evaluator that follows the RFC:
"""Queries in the style of the 2007 article, read by an evaluator that follows RFC 9535."""
import json
from jsonpath_mini import JSONPathSyntaxError, query
with open("catalogue.json", encoding="utf-8") as f:
catalogue = json.load(f)
for q in ["$..books[(@.length-1)].sku", "$..books[-1:].sku", "$..books[-1].sku", "$..books[0,1].sku", "$.books[*].isbn"]:
try:
nodes = query(q, catalogue)
print(f"{q:<30} {[value for _, value in nodes]}")
except JSONPathSyntaxError as e:
print(f"{q:<30} refused: {e}")Output
$..books[(@.length-1)].sku refused: (…) is a script expression of the 2007 article, which RFC 9535 does not have (at position 9) $..books[-1:].sku ['BK-310'] $..books[-1].sku ['BK-310'] $..books[0,1].sku ['BK-101', 'BK-205'] $.books[*].isbn []
Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 old_syntax.py
Runs on this device, in your browser. The first run downloads Python (about 13.5 MB), which is kept for the next runs.
Your run, in this browser
The script expression was refused before the catalogue was read, while the article’s other forms, the slice [-1:],
the union [0,1] and the two dots, mean the same today. The last query found no isbn, and its answer is an empty
list, not false.
Libraries written before 2024 often follow the article, or their own reading of it, so one query can give different nodelists in two languages. The JSONPath Compliance Test Suite, a public set of queries, documents and the nodelists RFC 9535 expects, is the yardstick: python-jsonpath, for one, says that it follows RFC 9535 and tests against the suite. The evaluator that the programs on this page import reads every query with the RFC’s grammar before it looks at the document, and runs nothing as code. Read it to see each rule as a few lines of Python:
"""jsonpath_mini: the selectors and segments of JSONPath (RFC 9535), written with the standard library only.
from jsonpath_mini import query
for path, value in query("$.books[*].title", catalogue):
print(path, value)
query() reads the whole query before it looks at the document, so a query that RFC 9535 does not allow raises
JSONPathSyntaxError, with the position of the problem, before anything is selected. Nothing in a query is ever run as
code. The result is the nodelist, a list of (normalized path, value) pairs in the order RFC 9535 gives, with object
members in the order of the document. Filter selectors (?…) are left out here: the next lesson's evaluator adds them.
Section numbers in the comments are those of RFC 9535.
"""
import re
MAX_EXACT = 2**53 - 1 # integers in a query stay in the I-JSON range of exact values (section 2.1)
BLANK = " \t\n\r" # the blank space allowed between the parts of a query (section 2.1.1)
DIGITS = "0123456789"
class JSONPathSyntaxError(ValueError):
"""The text is not a JSONPath query of RFC 9535; .position is where the problem was found."""
def __init__(self, message, position):
super().__init__(f"{message} (at position {position})")
self.position = position
def query(text, document):
"""The nodelist that the query selects from the document: a list of (normalized path, value)."""
segments = _Parser(text).parse()
return [(normalized_path(path), value) for path, value in _apply(segments, [((), document)])]
# ---- running segments and selectors --------------------------------------------------------------------------------
# A node is (path, value): the path is a tuple of member names and array positions from the root.
def _apply(segments, nodes):
"""Applies each segment to every node of the nodelist so far, in order (section 2.1.2)."""
for descendant, selectors in segments:
out = []
for node in nodes:
for target in _self_and_descendants(node) if descendant else (node,):
for selector in selectors:
out.extend(selector(target))
nodes = out
return nodes
def _children(node):
"""The member values of an object, or the elements of an array, in order (sections 1.1, 2.3.2)."""
path, value = node
if isinstance(value, dict):
return [(path + (name,), item) for name, item in value.items()]
if isinstance(value, list):
return [(path + (i,), item) for i, item in enumerate(value)]
return []
def _self_and_descendants(node):
"""The node, then its descendants: every node before its descendants, arrays in order (section 2.5.2).
A list of nodes still to visit, not recursion, so a deeply nested document cannot overflow the stack."""
stack = [node]
while stack:
current = stack.pop()
yield current
stack.extend(reversed(_children(current)))
def _name_selector(name):
def select(node):
path, value = node
if isinstance(value, dict) and name in value:
return [(path + (name,), value[name])]
return [] # nothing is selected from a value that is not an object (section 2.3.1.2)
return select
def _wildcard_selector(node):
return _children(node)
def _index_selector(index):
def select(node):
path, value = node
if isinstance(value, list):
i = index + len(value) if index < 0 else index # -1 is the last element (section 2.3.3.2)
if 0 <= i < len(value):
return [(path + (i,), value[i])]
return []
return select
def slice_positions(length, start, end, step):
"""The positions that start:end:step selects in an array of this length (section 2.3.4.2.2)."""
step = 1 if step is None else step
if step == 0:
return []
if start is None:
start = 0 if step > 0 else length - 1
if end is None:
end = length if step > 0 else -length - 1
def normalize(i):
return i if i >= 0 else length + i
if step > 0:
lower = min(max(normalize(start), 0), length)
upper = min(max(normalize(end), 0), length)
return list(range(lower, upper, step))
upper = min(max(normalize(start), -1), length - 1)
lower = min(max(normalize(end), -1), length - 1)
return list(range(upper, lower, step))
def _slice_selector(start, end, step):
def select(node):
path, value = node
if not isinstance(value, list):
return []
return [(path + (i,), value[i]) for i in slice_positions(len(value), start, end, step)]
return select
# ---- normalized paths (section 2.7) --------------------------------------------------------------------------------
_PATH_ESCAPES = {"\b": "\\b", "\f": "\\f", "\n": "\\n", "\r": "\\r", "\t": "\\t", "'": "\\'", "\\": "\\\\"}
def normalized_path(path):
"""$['books'][0]['title']: single quotes, the escapes of section 2.7, non-negative positions."""
parts = ["$"]
for step in path:
if isinstance(step, int):
parts.append(f"[{step}]")
else:
name = "".join(_PATH_ESCAPES.get(c) or (f"\\u{ord(c):04x}" if ord(c) < 0x20 else c) for c in step)
parts.append(f"['{name}']")
return "".join(parts)
# ---- the parser (the ABNF of sections 2.1 to 2.5) -----------------------------------------------------------------
class _Parser:
def __init__(self, text):
self.text, self.pos = text, 0
def fail(self, message, at=None):
raise JSONPathSyntaxError(message, self.pos if at is None else at)
def peek(self, s):
return self.text.startswith(s, self.pos)
def eat(self, s):
if self.text.startswith(s, self.pos):
self.pos += len(s)
return True
return False
def blank(self):
while self.pos < len(self.text) and self.text[self.pos] in BLANK:
self.pos += 1
def parse(self):
if not self.eat("$"):
self.fail("a query starts with $")
segments = []
while True: # *(S segment): blank space is allowed before a segment, never after the last one
before = self.pos
self.blank()
if self.eat(".."): # a descendant segment: ..[…], ..* or ..name (section 2.5.2)
segments.append((True, self.bracketed() if self.peek("[") else [self.dotted()]))
elif self.eat("."): # .* or .name, shorthand for [*] and ['name'] (section 2.5.1)
segments.append((False, [self.dotted()]))
elif self.peek("["):
segments.append((False, self.bracketed()))
else:
self.pos = before
break
if self.pos != len(self.text):
self.fail("unexpected text")
return segments
def dotted(self):
"""What follows a dot: * or a member name of letters, digits and _ that does not start with a digit."""
if self.eat("*"):
return _wildcard_selector
start = self.pos
if self.pos < len(self.text) and _name_first(self.text[self.pos]):
self.pos += 1
while self.pos < len(self.text) and (_name_first(self.text[self.pos]) or self.text[self.pos] in DIGITS):
self.pos += 1
return _name_selector(self.text[start : self.pos])
self.fail("expected a member name or * after the dot")
def bracketed(self):
"""[ selector, selector … ]: one or more selectors, separated by commas (section 2.5.1.1)."""
self.pos += 1 # [
selectors = []
while True:
self.blank()
selectors.append(self.selector())
self.blank()
if self.eat("]"):
return selectors
if not self.eat(","):
self.fail("expected , or ]")
def selector(self):
c = self.text[self.pos : self.pos + 1]
if c in ("'", '"'):
return _name_selector(self.string())
if c == "*":
self.pos += 1
return _wildcard_selector
if c == "?":
self.fail("filter selectors (?…) are not part of this evaluator; the next lesson's evaluator runs them")
if c == "(":
self.fail("(…) is a script expression of the 2007 article, which RFC 9535 does not have")
start = self.integer()
after = self.pos
self.blank()
if not self.peek(":"):
self.pos = after
if start is None:
self.fail("expected a selector")
return _index_selector(start)
self.pos += 1 # the slice selector: [start S] ":" S [end S] [":" [S step]] (section 2.3.4.1)
self.blank()
end = self.integer()
if end is not None:
after = self.pos
self.blank()
if not self.peek(":"):
self.pos = after
step = None
if self.eat(":"):
after = self.pos
self.blank()
step = self.integer()
if step is None:
self.pos = after
return _slice_selector(start, end, step)
def integer(self):
"""An integer of the query (section 2.3.3.1): no leading zeros, no -0, inside the I-JSON range; or None."""
m = re.compile(r"-?[0-9]+").match(self.text, self.pos)
if not m:
return None
digits = m.group().lstrip("-")
if (digits.startswith("0") and len(digits) > 1) or m.group() == "-0":
self.fail("integers have no leading zeros, and -0 is not one")
value = int(m.group())
if abs(value) > MAX_EXACT:
self.fail("the integer is outside the range -(2^53 - 1) to 2^53 - 1")
self.pos = m.end()
return value
def string(self):
"""A name in single or double quotes, with the escapes of section 2.3.1.1."""
quote = self.text[self.pos]
self.pos += 1
out = []
while True:
if self.pos >= len(self.text):
self.fail("the name has no closing quote")
c = self.text[self.pos]
self.pos += 1
if c == quote:
return "".join(out)
if c == "\\":
out.append(self.string_escape(quote))
elif ord(c) < 0x20:
self.fail("a control character must be escaped", self.pos - 1)
else:
out.append(c)
def string_escape(self, quote):
c = self.text[self.pos : self.pos + 1]
self.pos += 1
simple = {"b": "\b", "f": "\f", "n": "\n", "r": "\r", "t": "\t", "/": "/", "\\": "\\", quote: quote}
if c in simple:
return simple[c]
if c != "u":
self.fail(f"\\{c} is not an escape in a {'single' if quote == chr(39) else 'double'}-quoted name", self.pos - 2)
code = self.hex4()
if 0xDC00 <= code <= 0xDFFF:
self.fail("a low surrogate needs a high surrogate before it")
if 0xD800 <= code <= 0xDBFF:
if not self.eat("\\u"):
self.fail("a high surrogate needs a low surrogate after it")
low = self.hex4()
if not 0xDC00 <= low <= 0xDFFF:
self.fail("a high surrogate needs a low surrogate after it")
return chr(0x10000 + ((code - 0xD800) << 10) + (low - 0xDC00))
return chr(code)
def hex4(self):
digits = self.text[self.pos : self.pos + 4]
if len(digits) != 4 or any(d not in "0123456789abcdefABCDEF" for d in digits):
self.fail("\\u needs four hexadecimal digits")
self.pos += 4
return int(digits, 16)
def _name_first(c):
"""ALPHA, _, or any character from U+0080 on (section 2.5.1.1)."""
return c == "_" or ("a" <= c <= "z") or ("A" <= c <= "Z") or ord(c) >= 0x80Checkpoint
Check: RFC 9535 and the 2007 article
Practise: query a timetable, then turn paths into pointers
The first exercise gives you a train timetable and eight questions to answer with one query each; its sample tests run your queries with this lesson’s evaluator. The second turns a normalized path into the JSON Pointer of the same node, with the escapes of both formats:
Code lab · 2 steps
Query a timetable, then name its nodes
Answer eight questions about a timetable with one JSONPath query each, then convert normalized paths into JSON Pointers.
- Eight questions about a timetable, eight queries To do
- From a normalized path to a JSON Pointer To do
Exercise · Easy · Python
Eight questions about a timetable, eight queries
Fill in QUERIES in timetable.py with one JSONPath query (RFC 9535) for each question. The sample tests run every query on this timetable with the lesson's evaluator and compare the values it selects, in order:
{
"trains": [
{ "number": "T-301", "name": "Monsoon Link",
"stops": [{ "station": "Mumbai", "dep": "06:10" }, { "station": "Ratnagiri", "arr": "11:05", "dep": "11:10" },
{ "station": "Madgaon", "arr": "15:40" }],
"fares": { "sleeper": 450, "ac3": 1180 } },
{ "number": "T-302", "name": "Deccan Dawn",
"stops": [{ "station": "Pune", "dep": "05:45" }, { "station": "Satara", "arr": "07:50", "dep": "07:55" },
{ "station": "Kolhapur", "arr": "10:30" }],
"fares": { "sleeper": 310, "ac3": 820, "ac2": 1150 } },
{ "number": "T-303", "name": "Coastal Night",
"stops": [{ "station": "Mangaluru", "dep": "21:00" }, { "station": "Udupi", "arr": "22:10", "dep": "22:12" },
{ "station": "Karwar", "arr": "01:35" }],
"fares": { "sleeper": 290 } }
],
"platforms": { "Madgaon Junction": 2, "Pune Junction": 4 }
}train_names: the name of every train.second_train_start: the first station of the second train.last_stations: the last station of every train.all_departures: every departure time (dep) in the timetable, in the document's order.first_two_numbers: the numbers of the first two trains, with a slice.first_train_backwards: the stations of the first train, from its last stop to its first.second_train_fares: every fare of the second train.pune_platform: the platform of Pune Junction.
Starter code · timetable.py
"""One JSONPath query for each question about the timetable (the timetable is in the prompt and the tests)."""
QUERIES = {
"train_names": "",
"second_train_start": "",
"last_stations": "",
"all_departures": "",
"first_two_numbers": "",
"first_train_backwards": "",
"second_train_fares": "",
"pune_platform": "",
}The sample tests · test_timetable.py
"""Sample tests: each one runs your query on the timetable and compares the values it selects, in order.
The evaluator at the end of this file is the one the lesson's examples import (jsonpath_mini.py), so a query that
RFC 9535 does not allow fails its test with the reason and the position."""
from timetable import QUERIES
TIMETABLE = {
"trains": [
{
"number": "T-301",
"name": "Monsoon Link",
"stops": [{"station": "Mumbai", "dep": "06:10"}, {"station": "Ratnagiri", "arr": "11:05", "dep": "11:10"}, {"station": "Madgaon", "arr": "15:40"}],
"fares": {"sleeper": 450, "ac3": 1180},
},
{
"number": "T-302",
"name": "Deccan Dawn",
"stops": [{"station": "Pune", "dep": "05:45"}, {"station": "Satara", "arr": "07:50", "dep": "07:55"}, {"station": "Kolhapur", "arr": "10:30"}],
"fares": {"sleeper": 310, "ac3": 820, "ac2": 1150},
},
{
"number": "T-303",
"name": "Coastal Night",
"stops": [{"station": "Mangaluru", "dep": "21:00"}, {"station": "Udupi", "arr": "22:10", "dep": "22:12"}, {"station": "Karwar", "arr": "01:35"}],
"fares": {"sleeper": 290},
},
],
"platforms": {"Madgaon Junction": 2, "Pune Junction": 4},
}
def values(name):
"""The values that QUERIES[name] selects from the timetable, in order."""
return [value for _, value in query(QUERIES[name], TIMETABLE)]
def test_train_names():
"""the name of every train"""
assert values("train_names") == ["Monsoon Link", "Deccan Dawn", "Coastal Night"]
def test_second_train_start():
"""the first station of the second train"""
assert values("second_train_start") == ["Pune"]
def test_last_stations():
"""the last station of every train"""
assert values("last_stations") == ["Madgaon", "Kolhapur", "Karwar"]
def test_all_departures():
"""every departure time, in the document's order"""
assert values("all_departures") == ["06:10", "11:10", "05:45", "07:55", "21:00", "22:12"]
def test_first_two_numbers():
"""the numbers of the first two trains, with a slice"""
assert values("first_two_numbers") == ["T-301", "T-302"]
assert ":" in QUERIES["first_two_numbers"], "use a slice, such as [start:end]"
def test_first_train_backwards():
"""the stations of the first train, from its last stop to its first"""
assert values("first_train_backwards") == ["Madgaon", "Ratnagiri", "Mumbai"]
def test_second_train_fares():
"""every fare of the second train"""
assert values("second_train_fares") == [310, 820, 1150]
def test_pune_platform():
"""the platform of Pune Junction"""
assert values("pune_platform") == [4]
# ---- the lesson's evaluator (examples/jsonpath/jsonpath_mini.py), which runs your queries ----------------------------
import re
MAX_EXACT = 2**53 - 1 # integers in a query stay in the I-JSON range of exact values (section 2.1)
BLANK = " \t\n\r" # the blank space allowed between the parts of a query (section 2.1.1)
DIGITS = "0123456789"
class JSONPathSyntaxError(ValueError):
"""The text is not a JSONPath query of RFC 9535; .position is where the problem was found."""
def __init__(self, message, position):
super().__init__(f"{message} (at position {position})")
self.position = position
def query(text, document):
"""The nodelist that the query selects from the document: a list of (normalized path, value)."""
segments = _Parser(text).parse()
return [(normalized_path(path), value) for path, value in _apply(segments, [((), document)])]
# ---- running segments and selectors --------------------------------------------------------------------------------
# A node is (path, value): the path is a tuple of member names and array positions from the root.
def _apply(segments, nodes):
"""Applies each segment to every node of the nodelist so far, in order (section 2.1.2)."""
for descendant, selectors in segments:
out = []
for node in nodes:
for target in _self_and_descendants(node) if descendant else (node,):
for selector in selectors:
out.extend(selector(target))
nodes = out
return nodes
def _children(node):
"""The member values of an object, or the elements of an array, in order (sections 1.1, 2.3.2)."""
path, value = node
if isinstance(value, dict):
return [(path + (name,), item) for name, item in value.items()]
if isinstance(value, list):
return [(path + (i,), item) for i, item in enumerate(value)]
return []
def _self_and_descendants(node):
"""The node, then its descendants: every node before its descendants, arrays in order (section 2.5.2).
A list of nodes still to visit, not recursion, so a deeply nested document cannot overflow the stack."""
stack = [node]
while stack:
current = stack.pop()
yield current
stack.extend(reversed(_children(current)))
def _name_selector(name):
def select(node):
path, value = node
if isinstance(value, dict) and name in value:
return [(path + (name,), value[name])]
return [] # nothing is selected from a value that is not an object (section 2.3.1.2)
return select
def _wildcard_selector(node):
return _children(node)
def _index_selector(index):
def select(node):
path, value = node
if isinstance(value, list):
i = index + len(value) if index < 0 else index # -1 is the last element (section 2.3.3.2)
if 0 <= i < len(value):
return [(path + (i,), value[i])]
return []
return select
def slice_positions(length, start, end, step):
"""The positions that start:end:step selects in an array of this length (section 2.3.4.2.2)."""
step = 1 if step is None else step
if step == 0:
return []
if start is None:
start = 0 if step > 0 else length - 1
if end is None:
end = length if step > 0 else -length - 1
def normalize(i):
return i if i >= 0 else length + i
if step > 0:
lower = min(max(normalize(start), 0), length)
upper = min(max(normalize(end), 0), length)
return list(range(lower, upper, step))
upper = min(max(normalize(start), -1), length - 1)
lower = min(max(normalize(end), -1), length - 1)
return list(range(upper, lower, step))
def _slice_selector(start, end, step):
def select(node):
path, value = node
if not isinstance(value, list):
return []
return [(path + (i,), value[i]) for i in slice_positions(len(value), start, end, step)]
return select
# ---- normalized paths (section 2.7) --------------------------------------------------------------------------------
_PATH_ESCAPES = {"\b": "\\b", "\f": "\\f", "\n": "\\n", "\r": "\\r", "\t": "\\t", "'": "\\'", "\\": "\\\\"}
def normalized_path(path):
"""$['books'][0]['title']: single quotes, the escapes of section 2.7, non-negative positions."""
parts = ["$"]
for step in path:
if isinstance(step, int):
parts.append(f"[{step}]")
else:
name = "".join(_PATH_ESCAPES.get(c) or (f"\\u{ord(c):04x}" if ord(c) < 0x20 else c) for c in step)
parts.append(f"['{name}']")
return "".join(parts)
# ---- the parser (the ABNF of sections 2.1 to 2.5) -----------------------------------------------------------------
class _Parser:
def __init__(self, text):
self.text, self.pos = text, 0
def fail(self, message, at=None):
raise JSONPathSyntaxError(message, self.pos if at is None else at)
def peek(self, s):
return self.text.startswith(s, self.pos)
def eat(self, s):
if self.text.startswith(s, self.pos):
self.pos += len(s)
return True
return False
def blank(self):
while self.pos < len(self.text) and self.text[self.pos] in BLANK:
self.pos += 1
def parse(self):
if not self.eat("$"):
self.fail("a query starts with $")
segments = []
while True: # *(S segment): blank space is allowed before a segment, never after the last one
before = self.pos
self.blank()
if self.eat(".."): # a descendant segment: ..[…], ..* or ..name (section 2.5.2)
segments.append((True, self.bracketed() if self.peek("[") else [self.dotted()]))
elif self.eat("."): # .* or .name, shorthand for [*] and ['name'] (section 2.5.1)
segments.append((False, [self.dotted()]))
elif self.peek("["):
segments.append((False, self.bracketed()))
else:
self.pos = before
break
if self.pos != len(self.text):
self.fail("unexpected text")
return segments
def dotted(self):
"""What follows a dot: * or a member name of letters, digits and _ that does not start with a digit."""
if self.eat("*"):
return _wildcard_selector
start = self.pos
if self.pos < len(self.text) and _name_first(self.text[self.pos]):
self.pos += 1
while self.pos < len(self.text) and (_name_first(self.text[self.pos]) or self.text[self.pos] in DIGITS):
self.pos += 1
return _name_selector(self.text[start : self.pos])
self.fail("expected a member name or * after the dot")
def bracketed(self):
"""[ selector, selector … ]: one or more selectors, separated by commas (section 2.5.1.1)."""
self.pos += 1 # [
selectors = []
while True:
self.blank()
selectors.append(self.selector())
self.blank()
if self.eat("]"):
return selectors
if not self.eat(","):
self.fail("expected , or ]")
def selector(self):
c = self.text[self.pos : self.pos + 1]
if c in ("'", '"'):
return _name_selector(self.string())
if c == "*":
self.pos += 1
return _wildcard_selector
if c == "?":
self.fail("filter selectors (?…) are not part of this evaluator; the next lesson's evaluator runs them")
if c == "(":
self.fail("(…) is a script expression of the 2007 article, which RFC 9535 does not have")
start = self.integer()
after = self.pos
self.blank()
if not self.peek(":"):
self.pos = after
if start is None:
self.fail("expected a selector")
return _index_selector(start)
self.pos += 1 # the slice selector: [start S] ":" S [end S] [":" [S step]] (section 2.3.4.1)
self.blank()
end = self.integer()
if end is not None:
after = self.pos
self.blank()
if not self.peek(":"):
self.pos = after
step = None
if self.eat(":"):
after = self.pos
self.blank()
step = self.integer()
if step is None:
self.pos = after
return _slice_selector(start, end, step)
def integer(self):
"""An integer of the query (section 2.3.3.1): no leading zeros, no -0, inside the I-JSON range; or None."""
m = re.compile(r"-?[0-9]+").match(self.text, self.pos)
if not m:
return None
digits = m.group().lstrip("-")
if (digits.startswith("0") and len(digits) > 1) or m.group() == "-0":
self.fail("integers have no leading zeros, and -0 is not one")
value = int(m.group())
if abs(value) > MAX_EXACT:
self.fail("the integer is outside the range -(2^53 - 1) to 2^53 - 1")
self.pos = m.end()
return value
def string(self):
"""A name in single or double quotes, with the escapes of section 2.3.1.1."""
quote = self.text[self.pos]
self.pos += 1
out = []
while True:
if self.pos >= len(self.text):
self.fail("the name has no closing quote")
c = self.text[self.pos]
self.pos += 1
if c == quote:
return "".join(out)
if c == "\\":
out.append(self.string_escape(quote))
elif ord(c) < 0x20:
self.fail("a control character must be escaped", self.pos - 1)
else:
out.append(c)
def string_escape(self, quote):
c = self.text[self.pos : self.pos + 1]
self.pos += 1
simple = {"b": "\b", "f": "\f", "n": "\n", "r": "\r", "t": "\t", "/": "/", "\\": "\\", quote: quote}
if c in simple:
return simple[c]
if c != "u":
self.fail(f"\\{c} is not an escape in a {'single' if quote == chr(39) else 'double'}-quoted name", self.pos - 2)
code = self.hex4()
if 0xDC00 <= code <= 0xDFFF:
self.fail("a low surrogate needs a high surrogate before it")
if 0xD800 <= code <= 0xDBFF:
if not self.eat("\\u"):
self.fail("a high surrogate needs a low surrogate after it")
low = self.hex4()
if not 0xDC00 <= low <= 0xDFFF:
self.fail("a high surrogate needs a low surrogate after it")
return chr(0x10000 + ((code - 0xD800) << 10) + (low - 0xDC00))
return chr(code)
def hex4(self):
digits = self.text[self.pos : self.pos + 4]
if len(digits) != 4 or any(d not in "0123456789abcdefABCDEF" for d in digits):
self.fail("\\u needs four hexadecimal digits")
self.pos += 4
return int(digits, 16)
def _name_first(c):
"""ALPHA, _, or any character from U+0080 on (section 2.5.1.1)."""
return c == "_" or ("a" <= c <= "z") or ("A" <= c <= "Z") or ord(c) >= 0x80A hint
Every query starts with $ and names the path step by step: $.trains[1] is the second train, [-1] counts from the end, [*] takes every element, ..dep searches every level, and a name with a space needs brackets and quotes, such as ['Madgaon Junction']. Change one query at a time and run the sample tests: each failing test names its question.
Results of the sample tests
| Test | Result | Details |
|---|
What your code printed
The sample tests run on this device, in your browser (Pyodide): nothing is sent to mysmartcopilot.com. The first run downloads Python (about 13.5 MB), which is kept for the next runs. A check in your browser is feedback for you, not proof that the code is right for every input.
Exercise · Medium · Python
From a normalized path to a JSON Pointer
Write to_json_pointer(path) in pointer.py. It takes a normalized path of RFC 9535, such as $['books'][0]['title'], and returns the JSON Pointer of RFC 6901 that names the same node, /books/0/title.
$alone is the whole document, whose pointer is the empty string"".- A step in brackets and single quotes is a member name. Undo the escapes of a normalized path:
\'is an apostrophe,\\a backslash,\b,\f,\n,\rand\tare those control characters, and\uwith four hexadecimal digits is the character with that code. - A step in brackets without quotes is an array position: its digits become the pointer's step.
- In the pointer, every step starts with
/, and inside a step~is written~0and/is written~1.
So $['codes']['a/b'] becomes /codes/a~1b, and $['sections']['children\'s'][0] becomes /sections/children's/0.
Starter code · pointer.py
def to_json_pointer(path):
"""The RFC 6901 JSON Pointer of the node that an RFC 9535 normalized path names."""
return pathThe sample tests · test_pointer.py
from pointer import to_json_pointer
def test_root():
"""$ alone is the whole document: the empty pointer"""
assert to_json_pointer("$") == ""
def test_names_and_positions():
"""names and array positions become pointer steps"""
assert to_json_pointer("$['books'][0]['title']") == "/books/0/title"
assert to_json_pointer("$['books'][3]['series']['title']") == "/books/3/series/title"
assert to_json_pointer("$[10]") == "/10"
def test_pointer_escapes():
"""writes ~ as ~0 and / as ~1, the tilde first"""
assert to_json_pointer("$['codes']['a/b']") == "/codes/a~1b"
assert to_json_pointer("$['codes']['m~n']") == "/codes/m~0n"
assert to_json_pointer("$['~1']") == "/~01"
def test_apostrophe_and_backslash():
"""undoes \\' and \\\\ in names"""
assert to_json_pointer("$['sections']['children\\'s'][0]") == "/sections/children's/0"
assert to_json_pointer("$['a\\\\b']") == "/a\\b"
def test_control_characters():
"""undoes \\n and the \\u escapes of control characters"""
assert to_json_pointer("$['codes']['line\\nbreak']") == "/codes/line\nbreak"
assert to_json_pointer("$['codes']['bell\\u0007']") == "/codes/bell\u0007"
def test_other_characters_stay():
"""keeps Devanagari letters, spaces and an empty name as they are"""
assert to_json_pointer("$['codes']['दाम']") == "/codes/दाम"
assert to_json_pointer("$['sections']['new arrivals'][1]") == "/sections/new arrivals/1"
assert to_json_pointer("$['']") == "/"A hint
Read the path from position 1, just after $. Each step starts with [: when the next character is ', collect the name up to the next ' that is not escaped, turning each backslash escape into its character as you go; otherwise the step is the digits up to ]. Replace ~ before /, or the ~1 you have just written would turn into ~01.
Results of the sample tests
| Test | Result | Details |
|---|
What your code printed
The sample tests run on this device, in your browser (Pyodide): nothing is sent to mysmartcopilot.com. The first run downloads Python (about 13.5 MB), which is kept for the next runs. A check in your browser is feedback for you, not proof that the code is right for every input.
Fifteen questions cover the six ideas, from $ to the 2007 article. Whichever of them your answers so far leave
weakest comes first, and any question you miss is asked once more before the end:
Practice that adapts to you
Practise this lesson
15 questions about 6 ideas of this lesson. Each answer is explained at once, and what you get right is kept in this browser.
Interview questions
Warm-up (fresher to 2–5 years): what does a JSONPath query return when nothing matches, and how do you tell that from a match whose value is null?
A query always returns a nodelist, and RFC 9535 says that a valid query must not produce errors when it runs. When
nothing matches, the nodelist is empty: $.books[*].publisher on the catalogue returned no nodes at all. A member
whose value is null is a match: on {"note": null}, $.note returns one node, at $['note'], whose value is null,
because JSON null is a value like any other (section 2.6). So test the length of the nodelist, not the truthiness of a
value: no nodes means no match, and a node whose value is null means the member exists and holds null. The 2007
article’s implementation returned false when nothing matched, which is why older code still checks for false.
Two harder questions start from this lesson: a total that grew when a partner added gift cards, and a test that compares the nodelists of two services and fails only for some keys. Decide what you would check first, then open each first step:
Model answers open with any pass: ₹79 for 1 week ($2.99), one payment that never renews.
-
Fresher to mid
A nightly job totals the values of $..price_paise in each partner's catalogue. One partner starts listing gift cards, and its total rises by exactly the gift cards' prices, with no error anywhere. Explain why, and how you would change the query and the job so that the next new kind of priced item cannot change a total unnoticed.
-
Mid to senior
A contract test compares the nodelist of $.stock.* from a Python service with the one a Node.js service returns for the same document. The values always match as a set, but the test fails when stock is keyed by PIN codes such as 560001 and passes when it is keyed by city names. Explain the failure and fix the test.
Flashcards · 7 cards
Recall it, then check
Say the answer to yourself before you look: recalling it is what makes it stick.
-
What does a JSONPath query start with, and what does it return?
The root, $, then segments. It returns a nodelist, empty when nothing matches; a valid query never fails while it runs.
-
When must a member name be written in brackets and quotes?
When it holds spaces, apostrophes, hyphens or other characters the dot form cannot take.
-
What does a JSONPath slice with a step of 0 select?
Nothing. Python raises an error for such a slice, but RFC 9535 selects no elements. A slice is also clamped to the array.
-
In which order does .. visit the nodes of a document?
Each node before its descendants, and array elements in order. It finds a name at every depth, including matches you did not mean.
-
What is a normalized path?
The single canonical path of a node, such as $['books'][3]. Its syntax alone converts it to a JSON Pointer.
-
How do you tell "no match" from "matched a null"?
Count the nodes. No nodes means no match; one node holding null means the member exists and is null. The 2007 article's code returned false for no match.
-
What did RFC 9535 remove from the 2007 JSONPath article?
Script expressions run by an outside engine, such as [(@.length-1)]; the last element is now [-1]. It also defined slices, order and empty results exactly.
Key takeaways
- A JSONPath query is
$followed by segments; each segment turns the nodelist so far into a new one, and a selector that finds nothing adds nothing, so a valid query answers with an empty nodelist instead of failing. - Names select members of objects, the wildcard every child, and an index one element counted from 0 or from the end; names with spaces, apostrophes or hyphens need brackets and quotes.
- A slice
start:end:stephas defaults that depend on the sign of the step, clamps to the array, and selects nothing for a step of 0, where Python raises an error. - A descendant segment searches every level in a fixed order, each node before its descendants and arrays in order,
so
..finds every match, including the ones you did not mean. - Every node has one normalized path, such as
$['books'][3], a safe label that converts to a JSON Pointer by its syntax alone. - RFC 9535 dropped the 2007 article’s script expressions and fixed slices, order and empty results; choose libraries that follow the RFC and test against its compliance suite.
Terms in this lesson
- nodelist
- The result of a JSONPath query: a list of nodes, each a value together with its place in the document. An empty nodelist means that nothing matched, which is not the same as a match whose value is null.
- normalized path
- The one JSONPath form that names a single node exactly, such as $['books'][0]['title']: every name in single quotes and every array position counted from 0. RFC 9535 section 2.7 defines it.
- segment
- One step of a JSONPath query. A child segment such as [0] or .title picks children of every node found so far; a descendant segment such as ..title picks from those nodes and everything below them.
- selector
- One item inside a JSONPath segment that picks children of a node: a name, the wildcard *, an index, a slice or a filter. A segment may hold several, separated by commas.
References
- RFC 9535: JSONPath: Query Expressions for JSON (IETF)
- JSONPath: XPath for JSON (the original article) (Stefan Gössner)
- RFC 6901: JavaScript Object Notation (JSON) Pointer (IETF)
- JSONPath Compliance Test Suite (jsonpath-standard)
- python-jsonpath documentation (python-jsonpath project)
- Built-in types: common sequence operations (slices) (Python Software Foundation)
Related tools
Report a problem with this lesson
Kept only in this browser. Your Learn progress