GIS & Geospatial Data (QGIS, GeoPandas, PostGIS) Module 12 – Professional practice
Data quality, metadata and reproducible workflows
How to state a dataset's quality by ISO 19157's elements, write ISO 19115 metadata with lineage, pin versions so results repeat, and check data before release.
What you will learn
- Assess a dataset's completeness, logical consistency, positional, thematic and temporal quality
- Write an ISO 19115-style metadata record with lineage, accuracy, CRS and licence
- Configure a pinned environment so that a geospatial analysis gives the same numbers next year
- Run automated quality checks before publishing data or maps
Before you start
What you’ll learn · Before you start
What you will learn
- Assess a dataset's completeness, logical consistency, positional, thematic and temporal quality
- Write an ISO 19115-style metadata record with lineage, accuracy, CRS and licence
- Configure a pinned environment so that a geospatial analysis gives the same numbers next year
- Run automated quality checks before publishing data or maps
Before you start
On this page
A ward layer passes from a partner to your team, and a year later nobody can say why ward W06 is missing, which CRS its areas were measured in, or why the same script now prints a slightly different area. Three habits prevent that: state the data’s quality, describe it in metadata, and make every run repeatable.
Here you check a layer by the elements of ISO 19157, write its ISO 19115-1 record the way Survey of India describes its boundaries, find why numbers change between machines, and build the checks that run before anything is published.
Quality is stated element by element, with how it was measured
“Good data” says nothing. ISO 19157-1 sets out “the principles for describing the quality of geographic data”, and leaves the acceptable levels to each product’s specification (ISO/TC 211, ISO 19157-1). Its schema names the elements a quality report uses (ISO 19157-2 schema):
| Element | The question it answers | For the ward layer |
|---|---|---|
| completeness (data quality): The ISO 19157 data quality element that asks whether features, attributes or relationships are missing (omission) or present in excess (commission), against what the data should hold. | Is anything missing or extra? | Are all 12 wards there, once each? |
| logical consistency: The ISO 19157 data quality element that asks whether data obeys the rules of its own structure: its concepts, value domains, format and topology, such as unique ids and closed rings. | Does the data obey its own rules? | Unique ids, no negative population, closed rings that do not cross themselves |
| positional accuracy: The ISO 19157 data quality element that states how close positions are to their true values, absolutely or relative to each other, for example as a horizontal RMSE at check points. | How far are positions from the truth? | The RMSE at surveyed check points |
| Thematic accuracy | Are the classes and values right? | Populations close to the census count |
| Temporal quality | Are dates right, in order and valid? | A survey year that has happened |
| Usability | Does it fit a stated purpose? | Fit for ward-level planning |
Each element splits further: completeness into commission and omission, logical consistency into conceptual, domain, format and topological consistency, positional accuracy into absolute, relative and gridded data. A good report names the element, the measure, the result and how it was measured. Survey of India’s boundary database is a model: its metadata reports an absolute external positional accuracy, “Horizontal RMSE ±12.5 m” (ABDB metadata). India’s own standard for data quality, IS 18565 Part 1, is the counterpart of ISO 19157-1 (Survey of India, standards report).
One accuracy figure for everything
“Accuracy: good” or a single number hides which element it measures. Report the element, the measure and the method: “absolute positional accuracy, horizontal RMSE 2.1 m at 24 check points”, not “accurate to 2 m”.
Checkpoint
Check: Quality, element by element
Metadata says what the data is, where it came from and how far to trust it
metadata (geospatial): Data that describes a dataset for its users: what it is, its extent, quality, CRS, lineage, constraints and distribution. ISO 19115-1 defines a schema for it. is the dataset’s description for the next user. ISO 19115-1 “defines the schema required for describing geographic information and services by means of metadata”: the identification, the extent, the quality, the spatial and temporal aspects, the content, the spatial reference, the portrayal and the distribution (ISO/TC 211, ISO 19115-1). India’s national metadata standard, IS 16439, is its equivalent. Survey of India writes its boundary database’s sheets in ISO 19115-1:2014: an identifier with the edition, an abstract and a purpose, a scale of 1:50,000, the reference system “LCC- WGS84; EPSG7755”, a lineage: The part of a metadata record that tells a dataset's history: the sources it came from and the steps applied to them, from which a user judges what it is fit for. from its 1:50,000 topographic database, the RMSE, copyright constraints and the formats.
Most layers arrive with far less. This program holds the record of the town’s ward layer as the partner sent it, checks it against ten elements a user needs, and then prints the completed record:
"""A metadata record for the town's ward layer, written with ISO 19115-1's elements, the way Survey of India's
boundary database describes itself, and a check that every element a user needs is filled in."""
REQUIRED = { # element (ISO 19115-1 class and property): what a user decides with it
"MD_DataIdentification.citation.title": "which dataset this is",
"MD_DataIdentification.abstract": "what it holds",
"MD_DataIdentification.purpose": "what it was made for",
"MD_ReferenceSystem.referenceSystemIdentifier": "the CRS to assign",
"MD_Resolution.equivalentScale": "the largest scale it holds at",
"LI_Lineage.statement": "where it came from and what was done to it",
"DQ_AbsoluteExternalPositionalAccuracy": "how far to trust a position",
"MD_LegalConstraints.useConstraints": "what you may do with it",
"MD_DataIdentification.credit": "the credit to show",
"MD_Distribution.distributionFormat": "how to read it",
}
received = {
"MD_DataIdentification.citation.title": "Wards of the town",
"MD_DataIdentification.abstract": "12 ward polygons with population",
"MD_ReferenceSystem.referenceSystemIdentifier": "EPSG:32643",
"MD_Distribution.distributionFormat": "GeoPackage",
}
completed = received | {
"MD_DataIdentification.purpose": "Ward-level planning of clinics and schools",
"MD_Resolution.equivalentScale": "1:10,000",
"LI_Lineage.statement": "Digitised from the town's ward map; corners checked by GNSS; the faults qa.py found corrected",
"DQ_AbsoluteExternalPositionalAccuracy": "Horizontal RMSE 2.1 m at 24 check points",
"MD_LegalConstraints.useConstraints": "CC BY 4.0",
"MD_DataIdentification.credit": "Town planning office",
}
for name, record in (("as received", received), ("completed", completed)):
missing = [element for element in REQUIRED if not record.get(element)]
print(f"{name}: {len(REQUIRED) - len(missing)} of {len(REQUIRED)} elements")
for element in missing:
print(f" missing {element}: {REQUIRED[element]}")
print()
for element, value in completed.items():
print(f"{element}: {value}")Output
as received: 4 of 10 elements missing MD_DataIdentification.purpose: what it was made for missing MD_Resolution.equivalentScale: the largest scale it holds at missing LI_Lineage.statement: where it came from and what was done to it missing DQ_AbsoluteExternalPositionalAccuracy: how far to trust a position missing MD_LegalConstraints.useConstraints: what you may do with it missing MD_DataIdentification.credit: the credit to show completed: 10 of 10 elements MD_DataIdentification.citation.title: Wards of the town MD_DataIdentification.abstract: 12 ward polygons with population MD_ReferenceSystem.referenceSystemIdentifier: EPSG:32643 MD_Distribution.distributionFormat: GeoPackage MD_DataIdentification.purpose: Ward-level planning of clinics and schools MD_Resolution.equivalentScale: 1:10,000 LI_Lineage.statement: Digitised from the town's ward map; corners checked by GNSS; the faults qa.py found corrected DQ_AbsoluteExternalPositionalAccuracy: Horizontal RMSE 2.1 m at 24 check points MD_LegalConstraints.useConstraints: CC BY 4.0 MD_DataIdentification.credit: Town planning office
Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 record.py
Runs on this device, in your browser. The first run downloads Python (about 13.5 MB), which is kept for the next runs.
Your run, in this browser
The received record fills 4 of 10: it says what the layer is and how to read it, but not what it was made for, how it was made, how accurate it is, or what you may do with it. Those six are the ones a careful user asks for first. The lineage matters most: “digitised from the town’s ward map; corners checked by GNSS” tells a reader both the source and the check, and so how far the RMSE can be believed.
JSON Validator Check that a metadata record kept as JSON is valid, and list every field it holds, before you publish it.Checkpoint
Check: An ISO 19115-1 record
The same script gives other numbers when the libraries change
A geospatial script is only partly your code. The geometry is done by GEOS, the coordinate operations by PROJ, with its own copy of the EPSG dataset, and the files by GDAL, and each environment carries its own versions. Take two environments of this track: the Python wheels behind its GeoPandas lessons, and QGIS 4.2.3 from conda-forge, which its QGIS lessons ran on. Their version reports differ:
"""Version reports of two environments of this track: the Python wheels behind its GeoPandas lessons and QGIS 4.2.3 from
conda-forge, which its QGIS lessons ran on: the geometry (GEOS), projection (PROJ, with its copy of the EPSG dataset)
and file (GDAL) libraries that each one carried."""
REPORTS = {
"wheels": {"GEOS": "3.13.1", "PROJ": "9.8.1", "EPSG": "v12.029", "GDAL": "3.12.4"}, # shapely 2.1.2, pyproj 3.8.0, pyogrio 0.13.0
"QGIS 4.2.3": {"GEOS": "3.14.1", "PROJ": "9.9.0", "EPSG": "v13.102", "GDAL": "3.13.3"}, # from conda-forge
}"""Which libraries differ between the environments of reports.py, and what a difference can change in the results."""
from reports import REPORTS
CHANGES = {
"GEOS": "overlay, buffers and validity: vertex order, extra vertices, last digits of areas",
"PROJ": "which transformation is chosen, and so the coordinates",
"EPSG": "which CRSs and transformations exist at all",
"GDAL": "how files are read and written: drivers, field types, defaults",
}
names = list(REPORTS)
print("library " + "".join(f"{n:>14s}" for n in names))
for lib in CHANGES:
values = [REPORTS[n][lib] for n in names]
print(f"{lib:8s}" + "".join(f"{v:>14s}" for v in values) + ("" if len(set(values)) == 1 else " differs"))
print()
for lib, effect in CHANGES.items():
if len({REPORTS[n][lib] for n in names}) > 1:
print(f"{lib}: {effect}")Output
library wheels QGIS 4.2.3 GEOS 3.13.1 3.14.1 differs PROJ 9.8.1 9.9.0 differs EPSG v12.029 v13.102 differs GDAL 3.12.4 3.13.3 differs GEOS: overlay, buffers and validity: vertex order, extra vertices, last digits of areas PROJ: which transformation is chosen, and so the coordinates EPSG: which CRSs and transformations exist at all GDAL: how files are read and written: drivers, field types, defaults
Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 drift.py
Runs on this device, in your browser. The first run downloads Python (about 13.5 MB), which is kept for the next runs.
Your run, in this browser
All four libraries differ between the two environments. Even their EPSG copies differ, v12.029 against v13.102, because each PROJ release ships its own version of the dataset: PROJ 9.6.0 carries v12.004, 9.8.1 went back to v12.029, and 9.9.0 carries v13.102 (PROJ news); pyproj 3.8.0’s wheels carry PROJ 9.8.1 (pyproj). A transformation or a CRS definition can differ between such copies, and a new GEOS can return the same polygon from another start vertex, in the other direction, or with an extra vertex on a straight edge.
Key idea
A geospatial result belongs to the libraries that made it: name GEOS, PROJ with its EPSG version, and GDAL beside every number you publish.
Checkpoint
Check: Why numbers change between environments
Pinned environments and tests of the shape make results repeat
Repeatable results take two habits. The first is to pin the environment, so next year’s run uses this year’s libraries:
- Python: lock the dependencies. uv writes a lockfile from your project’s dependencies, and
uv sync --lockedrefuses to run when the lockfile is out of date (uv); pip’s hash-checking mode installs only pinned requirements whose files match their recorded hashes, dependencies included (pip). - QGIS: keep heavy work in one pinned environment or a pinned container image, run its steps with
qgis_process, whose--jsonoption prints results in a structured form, and keep each workflow as a model in a.model3file (QGIS). - Every run: print the versions of GEOS, PROJ, EPSG and GDAL into its log, seed anything random, and sort outputs that have no order.
The second habit is to test what matters. A file’s bytes change with library versions even when the data does not, so compare the shape: counts, areas and corners. This program compares two exports of ward W01 that differ as text:
"""Two exports of ward W01 that a byte-for-byte test calls different and a test of the shape calls equal."""
A = "POLYGON ((500000 2004000, 502000 2004000, 502000 2006000, 500000 2006000, 500000 2004000))"
B = "POLYGON ((502000 2006000, 502000 2004000, 500000 2004000, 500000 2006000, 501000 2006000, 502000 2006000))"
def ring(wkt):
"""The ring's vertices as (x, y) floats, closing point dropped."""
body = wkt[wkt.index("((") + 2:wkt.index("))")]
pts = [tuple(float(v) for v in p.split()) for p in body.split(",")]
return pts[:-1]
def area(pts):
return abs(sum(x1 * y2 - x2 * y1 for (x1, y1), (x2, y2) in zip(pts, pts[1:] + pts[:1]))) / 2
def corners(pts):
"""Vertices where the boundary turns: a vertex in the middle of a straight edge adds nothing to the shape."""
keep = []
for i, (x, y) in enumerate(pts):
(ax, ay), (bx, by) = pts[i - 1], pts[(i + 1) % len(pts)]
if (x - ax) * (by - y) - (y - ay) * (bx - x) != 0:
keep.append((x, y))
return set(keep)
print(f"same text: {A == B}")
print(f"areas: {area(ring(A)):,.0f} and {area(ring(B)):,.0f} m²")
print(f"same corners, whatever the start, direction or extra vertices: {corners(ring(A)) == corners(ring(B))}")Output
same text: False areas: 4,000,000 and 4,000,000 m² same corners, whatever the start, direction or extra vertices: True
Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 same_shape.py
Runs on this device, in your browser. The first run downloads Python (about 13.5 MB), which is kept for the next runs.
Your run, in this browser
The second export starts at another corner, runs the other way and carries an extra vertex in the middle of an edge, so a byte-for-byte test fails. Both enclose 4,000,000 m² and have the same four corners, so a test of the shape passes, and would still catch a real change.
Checkpoint
Check: Pinning and testing the shape
Automated checks run before anything is published
Checks a person must remember get skipped. Write them as a program and run it on every release, element by element:
"""The ward layer of the track's town as a partner sent it: 12 wards of 2 km in UTM zone 43N, with five faults planted
for the quality checks to find. All values are made up."""
CRS = "EPSG:32643"
def square(col, row):
"""The ring of the ward in column col and row row, counter-clockwise, closed."""
x, y = 500_000 + col * 2_000, 2_000_000 + row * 2_000
return [(x, y), (x + 2_000, y), (x + 2_000, y + 2_000), (x, y + 2_000), (x, y)]
WARDS = []
for i in range(12):
col, row = i % 4, 2 - i // 4
WARDS.append({"id": f"W{i + 1:02d}", "ring": square(col, row), "population": 1_800 + 97 * i, "survey_year": 2024})
WARDS[5]["id"] = "W05" # a second W05: W06's id was typed wrong
WARDS[7]["ring"] = WARDS[7]["ring"][:-1] # W08: the ring is not closed
r = WARDS[9]["ring"]
WARDS[9]["ring"] = [r[0], r[2], r[1], r[3], r[0]] # W10: two corners swapped, a ring that crosses itself
WARDS[2]["population"] = -120 # W03: a value no ward can have
WARDS[10]["survey_year"] = 2031 # W11: a survey year that has not happened"""Quality checks of the ward layer before its release, by ISO 19157's elements: completeness, logical consistency
(conceptual, domain and topology) and temporal validity. The report is printed as JSON and as Markdown."""
import json
from wards import CRS, WARDS
EXPECTED = [f"W{i:02d}" for i in range(1, 13)]
def crosses(p, q, r, s):
"""True when segment p-q crosses segment r-s at a point inside both."""
def turn(a, b, c):
return (b[0] - a[0]) * (c[1] - a[1]) - (b[1] - a[1]) * (c[0] - a[0])
return turn(p, q, r) * turn(p, q, s) < 0 and turn(r, s, p) * turn(r, s, q) < 0
def ring_problem(ring):
if len(ring) < 4 or ring[0] != ring[-1]:
return "ring not closed"
edges = list(zip(ring, ring[1:]))
for i, a in enumerate(edges):
for b in edges[i + 2:]:
if a[0] != b[1] and crosses(*a, *b):
return "ring crosses itself"
return None
problems = []
ids = [w["id"] for w in WARDS]
problems += [{"element": "completeness", "id": i, "problem": "missing"} for i in EXPECTED if i not in ids]
problems += [{"element": "logical consistency", "id": i, "problem": "duplicate id"} for i in sorted(set(ids)) if ids.count(i) > 1]
if CRS != "EPSG:32643":
problems.append({"element": "logical consistency", "id": "layer", "problem": f"CRS {CRS}, expected EPSG:32643"})
for w in WARDS:
if (p := ring_problem(w["ring"])):
problems.append({"element": "logical consistency", "id": w["id"], "problem": p})
if w["population"] < 0:
problems.append({"element": "logical consistency", "id": w["id"], "problem": f"population {w['population']}"})
if not 2011 <= w["survey_year"] <= 2025:
problems.append({"element": "temporal validity", "id": w["id"], "problem": f"survey year {w['survey_year']}"})
report = {"layer": "wards", "crs": CRS, "features": len(WARDS), "problems": len(problems), "ok": not problems}
print(json.dumps(report, sort_keys=True))
print("\n| Element | Ward | Problem |\n| --- | --- | --- |")
for p in sorted(problems, key=lambda p: (p["element"], p["id"])):
print(f"| {p['element']} | {p['id']} | {p['problem']} |")Output
{"crs": "EPSG:32643", "features": 12, "layer": "wards", "ok": false, "problems": 6}
| Element | Ward | Problem |
| --- | --- | --- |
| completeness | W06 | missing |
| logical consistency | W03 | population -120 |
| logical consistency | W05 | duplicate id |
| logical consistency | W08 | ring not closed |
| logical consistency | W10 | ring crosses itself |
| temporal validity | W11 | survey year 2031 |
Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 qa.py
Runs on this device, in your browser. The first run downloads Python (about 13.5 MB), which is kept for the next runs.
Your run, in this browser
Five faults were planted, and the report lists six problems, because one slip, W06’s id typed as W05, makes both a
duplicate and a missing ward. The JSON line is for machines: a release script stops when ok is false. The table is
for people, sorted by element. Step through the whole path from a received layer to a published one:
The drawing shows the whole scene; its 7 steps are listed below.
Every step as text
- What arrived. A partner sends the ward layer of the town, 12 features in UTM zone 43N, with a metadata record that fills 4 of the 10 elements a user needs.
- Checks by element. The quality checks find 6 problems, one or more for each kind of fault planted in the layer, and file each under its ISO 19157 element, from completeness to temporal validity.
- Fix what the data proves. A duplicate W05 and a missing W06 are one typing slip; the open ring, the crossing ring, the negative population and the survey year of 2031 go back to the partner, who knows the truth.
- Check again. Every fix is followed by the same checks, so a correction that breaks something else is caught at once. The layer moves on only when the report is clean.
- Write the record. The metadata record gets its missing elements, among them the purpose, the scale, the lineage, the positional accuracy and the licence, so it fills 10 of 10.
- Note the versions. The run that made the layer prints its library versions. The two environments of reports.py differ in all four libraries, so the report says which one produced these numbers.
- Publish. The data goes out with its record and its version report, and a reader can now tell what the layer is, where it came from, how far to trust it and how to rebuild it.
Fix only what the data itself proves, such as the mistyped id, and send the rest back to the partner. Then run the same checks again, because a fix can break something else.
Checkpoint
Check: Automated checks before release
Practise: the checks of a ward layer
Write four of the checks yourself; the sample tests use three wards of the track’s town:
Exercise · Easy · Python
Write the quality checks for a ward layer
Write problems(wards, expected) in checks.py. wards is a list of dicts with an id, a ring (a list of (x, y) pairs) and a population; expected is the list of ids the layer should hold. Return a sorted list of (id, problem) pairs, one for each fault:
(id, "missing")for every expected id that no ward has (completeness);(id, "duplicate id")once for every id that two or more wards share (logical consistency);(id, "ring not closed")for a ring with fewer than 4 points, or whose last point is not its first;(id, "negative population")for a population below 0 (logical consistency: a value outside its domain).
A layer without faults gives []. The sample tests use three wards of the track's town.
Starter code · checks.py
def problems(wards, expected):
"""Sorted (id, problem) pairs: missing ids, duplicate ids, rings not closed, negative populations."""
# Replace this line: collect the faults of the four checks into a list, then sort it.
return []The sample tests · test_checks.py
from checks import problems
SQUARE = [(0, 0), (2_000, 0), (2_000, 2_000), (0, 2_000), (0, 0)]
def ward(id_, ring=SQUARE, population=1_000):
return {"id": id_, "ring": ring, "population": population}
def test_clean_layer():
"""three good wards give no problems"""
assert problems([ward("W01"), ward("W02"), ward("W03")], ["W01", "W02", "W03"]) == []
def test_duplicate_hides_a_missing_ward():
"""a mistyped id makes one duplicate and one missing ward"""
assert problems([ward("W01"), ward("W01"), ward("W03")], ["W01", "W02", "W03"]) == [
("W01", "duplicate id"), ("W02", "missing")]
def test_geometry_and_values():
"""an open ring and a negative population are reported with their ids"""
layer = [ward("W01", ring=SQUARE[:-1]), ward("W02", population=-120), ward("W03", ring=[(0, 0), (1, 0), (0, 0)])]
assert problems(layer, ["W01", "W02", "W03"]) == [
("W01", "ring not closed"), ("W02", "negative population"), ("W03", "ring not closed")]A hint
Count the ids first, for example with ids = [w["id"] for w in wards] and ids.count(i). Add a "missing" pair for each expected id not in ids, a "duplicate id" pair for each id in set(ids) that occurs more than once, then loop over the wards for the ring and the population, and return sorted(found).
Results of the sample tests
| Test | Result | Details |
|---|
What your code printed
The sample tests run on this device, in your browser (Pyodide): nothing is sent to mysmartcopilot.com. The first run downloads Python (about 13.5 MB), which is kept for the next runs. A check in your browser is feedback for you, not proof that the code is right for every input.
Then test yourself on the five ideas of this lesson, starting from the one you answered least well:
Practice that adapts to you
Practise this lesson
10 questions about 5 ideas of this lesson. Each answer is explained at once, and what you get right is kept in this browser.
Interview questions
Warm-up (fresher): What is lineage in metadata, and why does a user need it?
Lineage is the dataset’s history: the sources it was made from and the steps applied to them, such as Survey of India’s note that its boundaries come from its 1:50,000 topographic database and were converted to WGS 84. A user needs it to judge fitness: a layer digitised from a 1:50,000 map cannot support a 1:5,000 plan, whatever its coordinates’ decimals suggest. It also lets a later analyst repeat or audit the work.
Warm-up (fresher): Why compare results by shape rather than byte for byte?
Because file bytes change when libraries change, even when the data does not: a new GEOS may start a polygon at another vertex or add a vertex on a straight edge, and file writers change their defaults. Compare counts, areas and corners, within a tolerance, so tests fail on real changes only.
Two working situations test these habits, areas that moved after an upgrade and a layer without metadata; the bank holds their first steps:
Model answers open with any pass: ₹79 for 1 week ($2.99), one payment that never renews.
-
Mid to senior
A district land-use report is rerun a year later on a new laptop, and 7 of 40 class areas differ from last year's published table, by up to 0.4 %. Nobody changed the code. How do you find the cause, and how do you make next year's run give the same numbers?
-
Fresher to mid
A partner sends a village boundary layer for a survey, with no metadata, three days before field work starts. What do you check before you use it, and what do you write down?
Flashcards · 8 cards
Recall it, then check
Say the answer to yourself before you look: recalling it is what makes it stick.
-
Which data quality elements does ISO 19157 name?
Completeness (is anything missing or extra?), logical consistency (does the data obey its own rules?), positional accuracy, thematic accuracy, temporal quality and usability. A good report gives the element, the measure, the result and how it was measured.
-
How should a dataset's accuracy be reported?
By element, measure and method, as in "absolute positional accuracy, horizontal RMSE 2.1 m at 24 check points", never as "accurate to 2 m" or "accuracy: good".
-
What does an ISO 19115-1 metadata record describe?
A dataset's identification, extent, quality, spatial and temporal aspects, content, spatial reference, portrayal and distribution. India's national metadata standard, IS 16439, is its equivalent, and IS 18565 Part 1 is the counterpart of ISO 19157-1 for quality.
-
What is lineage in metadata, and why does a user need it?
The sources a dataset was made from and the steps applied to them. It shows what the data is fit for (a layer digitised from a 1:50,000 map cannot support a 1:5,000 plan, whatever its decimals suggest) and lets a later analyst repeat or audit the work.
-
Why can the same geospatial script print different numbers on another machine?
GEOS, PROJ with its own copy of the EPSG dataset, and GDAL do much of the work, and every environment carries its own versions: the track's two environments differed in all four, with EPSG v12.029 against v13.102. Name these versions beside every number you publish.
-
How do you make next year's run use this year's libraries?
Lock the Python dependencies (uv sync --locked refuses an out-of-date lockfile; pip's hash-checking mode installs only files that match their hashes), keep QGIS work in one pinned environment or container image run with qgis_process, and log the GEOS, PROJ, EPSG and GDAL versions in every run.
-
Why test exported geometry by its shape rather than byte for byte?
A new GEOS may start a polygon at another corner, run it the other way or add a vertex on a straight edge, so the bytes change while the data does not. Compare counts, areas and corners within a tolerance: two different texts of ward W01 both enclosed 4,000,000 m² with the same four corners.
-
Why run a layer's quality checks as a program on every release?
Checks a person must remember get skipped. A program reports every problem by element, gives a JSON result that stops the release when ok is false, and runs again after each fix, because a fix can break something else.
Key takeaways
- State quality element by element, completeness, logical consistency, positional, thematic and temporal accuracy, each with its measure and method.
- An ISO 19115-1 record says what the data is, where it came from, how accurate it is and what you may do with it; lineage and accuracy are the elements users need most.
- Results depend on GEOS, PROJ with its EPSG copy, and GDAL, whose versions differ between environments.
- Pin environments with lock files or images, print versions in every log, and test shapes and counts, not bytes.
- Run the quality checks as a program on every release, and stop the release when they fail.
Terms in this lesson
- completeness (data quality)
- The ISO 19157 data quality element that asks whether features, attributes or relationships are missing (omission) or present in excess (commission), against what the data should hold.
- lineage
- The part of a metadata record that tells a dataset's history: the sources it came from and the steps applied to them, from which a user judges what it is fit for.
- logical consistency
- The ISO 19157 data quality element that asks whether data obeys the rules of its own structure: its concepts, value domains, format and topology, such as unique ids and closed rings.
- metadata (geospatial)
- Data that describes a dataset for its users: what it is, its extent, quality, CRS, lineage, constraints and distribution. ISO 19115-1 defines a schema for it.
- positional accuracy
- The ISO 19157 data quality element that states how close positions are to their true values, absolutely or relative to each other, for example as a horizontal RMSE at check points.
References
- ISO 19157-1, Geographic information: Data quality, Part 1: General requirements (ISO/TC 211 Geographic information/Geomatics)
- ISO 19157-2 XML schema: the data quality elements (dataQualityElement.xsd) (ISO/TC 211 Geographic information/Geomatics)
- ISO 19115-1, Geographic information: Metadata, Part 1: Fundamentals (ISO/TC 211 Geographic information/Geomatics)
- ABDB metadata sheets (ISO 19115-1:2014), state, district, sub-district and village (Survey of India)
- Report on Geospatial Data Standards in India (Survey of India)
- PROJ news (release notes) (PROJ contributors (OSGeo))
- pyproj change log (the PROJ version its wheels carry) (pyproj contributors)
- Locking and syncing (uv documentation) (Astral)
- Secure installs, hash-checking mode (pip documentation) (Python Packaging Authority)
- QGIS user manual: using processing from the command line (QGIS Project)
- QGIS user manual: the model designer (QGIS Project)
Related tools
Report a problem with this lesson
Kept only in this browser. Your Learn progress