Coupling, cohesion and information hiding

A design is not judged when it is first written. It is judged by the changes it meets later: a new fare rule, a new input format, a second kind of report. Two measures from the 1970s, coupling and cohesion, and one criterion, information hiding, tell you before those changes arrive which of them will be cheap. Everything else in this module, the SOLID principles included, builds on these three ideas.

Coupling: what one module knows about another

In 1974 Wayne Stevens, Glenford Myers and Larry Constantine described coupling, in the paper “Structured design”, as the strength of the association that a connection sets up from one module to another. The more strongly two modules are coupled, the harder each one is to understand, change or fix on its own, and the further a change or a bug can travel. They named three things that decide how strong a connection is:

1. How complicated the connection is. A call with two plain arguments is simpler than one with a dozen, or one whose meaning you can only guess.
2. Whether it goes to the module’s interface or to something inside it. Using what a module offers is weaker coupling than reaching for its internal parts.
3. What travels along it. Passing data is the weakest kind. Passing control, a flag or switch that tells the other module what to do, is stronger. Changing another module’s code is the strongest of all.

Here is what each kind looks like in Python, from the loosest to the tightest:

• Data: you pass only the values the callee needs, such as fare_for(zones=3, peak=True).
• Control: you pass a flag that picks what the callee does, such as load(path, parse=True). The 1974 paper’s fix still works: two functions, load_text(path) and parse(text), and no flag.
• Common environment: several modules read and write the same module-level dictionary or global variable. Each one is then coupled to every other one, even to those it never calls.
• Internals: one module uses a part that another one never offered, such as stock._items. In Python a leading underscore marks a name as non-public: an implementation detail that may change without notice. The language does not stop you from using it; only the convention does.
• Changing code: one module replaces a function of another at run time (monkeypatching). The changed module now behaves in a way its own source no longer shows.

Reaching past the interface

StockV1 and StockV2 below keep the same public methods, codes() and units(), but StockV2 stores every delivery separately. Two reorder reports use them: one asks the public methods, the other reads the dictionary behind the underscore.

"""Two reorder reports: one reads Stock's non-public dict, one asks Stock's public methods.

StockV2 changes how units are stored (one entry per delivery) and keeps the same public methods.
"""


class StockV1:
    """Units on hand per product code, one number each."""

    def __init__(self):
        self._items = {}

    def receive(self, code, units):
        self._items[code] = self._items.get(code, 0) + units

    def units(self, code):
        return self._items.get(code, 0)

    def codes(self):
        return sorted(self._items)


class StockV2:
    """Keeps every delivery separately, so the oldest units can be sold first."""

    def __init__(self):
        self._items = {}

    def receive(self, code, units):
        self._items.setdefault(code, []).append(units)

    def units(self, code):
        return sum(self._items.get(code, []))

    def codes(self):
        return sorted(self._items)


def reorder_by_peeking(stock, minimum):
    """Coupled to how Stock stores its units: it reads the dict behind the underscore."""
    return [code for code, units in sorted(stock._items.items()) if units < minimum]


def reorder_by_asking(stock, minimum):
    """Coupled only to what Stock promises: codes() and units()."""
    return [code for code in stock.codes() if stock.units(code) < minimum]


for stock_class in (StockV1, StockV2):
    stock = stock_class()
    stock.receive("rice-5kg", 4)
    stock.receive("tea-250g", 30)
    stock.receive("rice-5kg", 3)
    print(f"{stock_class.__name__}: rice-5kg has {stock.units('rice-5kg')} units")
    print("  asking: ", reorder_by_asking(stock, 10))
    try:
        print("  peeking:", reorder_by_peeking(stock, 10))
    except TypeError as error:
        print("  peeking: TypeError:", error)

StockV1: rice-5kg has 7 units
  asking:  ['rice-5kg']
  peeking: ['rice-5kg']
StockV2: rice-5kg has 7 units
  asking:  ['rice-5kg']
  peeking: TypeError: '<' not supported between instances of 'list' and 'int'

Both classes answer units("rice-5kg") with 7, so any test of Stock itself passes before and after the change. The report that peeked breaks anyway, because it depended on a decision Stock never promised to keep: one number per product. Here the mistake is loud, a TypeError. It can be quiet instead. Suppose a later version keeps the shelf count in _items and subtracts units reserved for open orders in units(): the peeking report keeps running, reads the shelf count and reorders too late.

Cohesion: how well a module’s parts belong together

The same paper measured the other side, how strongly the parts inside one module belong together. It called this binding; today the word is cohesion. Its scale runs from weakest to strongest:

1. Coincidental: the parts only share a file, like a utils.py holding slugify(), send_sms() and tax_rate().
2. Logical: the parts do the same category of thing and a flag picks one, like an export(data, kind) that writes CSV, PDF or an SMS. Its callers are control-coupled to it.
3. Temporal: the parts run at the same time and are otherwise unrelated, like an on_startup() that opens the log file, warms a price cache and checks a licence key.
4. Communicational: the parts work on the same data, like functions that validate, total and archive one order.
5. Sequential: the output of one part is the input of the next, like parse, then validate, then price.
6. Functional: every part serves one well-defined job, like fare_for(trip).

The authors stress that the steps are not evenly spaced: functional cohesion is far ahead of every other kind, while coincidental and logical cohesion lag far behind the rest. High cohesion and low coupling are two views of one goal. Every relationship that you keep inside a module is one that does not have to cross between modules.

Information hiding: Parnas’s criterion

Coupling and cohesion tell you whether a split is good. They do not tell you where to cut. In 1972 David Parnas answered that question with a small program that builds a KWIC (keyword in context) index: it reads lines of text, makes every circular shift of each line and prints them all in alphabetical order. He compared two ways of dividing it into modules. He treated a module as a responsibility assignment, a piece of work that a separate team can own, rather than as a subroutine.

• The first made each major step of the processing a module: input, circular shift, alphabetizing and output, the boxes of a flowchart. The modules exchanged tables whose format all of them had to know.
• The second used information hiding as its criterion. Each module knew one design decision and kept it from all the others, behind an interface that gave away as little as it could.

Parnas then listed decisions that were likely to change, such as the input format, keeping every line in memory, packing four characters into each machine word, and keeping an index of the shifts instead of the shifts themselves. A change to the second or the third of these, where the lines are kept or how they are packed, would touch every module of the first design and only one module of the second. His conclusion is the criterion this track uses:

We propose instead that one begins with a list of difficult design decisions or design decisions which are likely to change. Each module is then designed to hide such a decision from the others.

The modules no longer match the steps of the processing, and that is the point.

Measuring a change in two designs

Here is the same idea on a program of today’s size. It prints a statement for each bus pass from a trip log. The first design has one module per processing step, and every step knows the layout of a row.

from pathlib import Path

from price import price
from read_log import read_log
from statement import statement

log = Path(__file__).with_name("log.txt").read_text()
print(statement(price(read_log(log))))

Card C17
  08:10 Market to Airport, 14 km: 25
  18:40 Airport to Museum, 11 km: 25
  Total: 50
Card C42
  09:05 Market to Museum, 4 km: 10
  17:15 Museum to Market, 4 km: 10
  Total: 20

def read_log(text):
    """Step 1: each line "card;from;to;start" becomes a row [card, from, to, start]."""
    return [line.split(";") for line in text.strip().splitlines()]

KM = {("Market", "Museum"): 4, ("Market", "Airport"): 14, ("Museum", "Airport"): 11}


def price(rows):
    """Step 2: append the distance and the fare, so each row is [card, from, to, start, km, fare]."""
    for row in rows:
        km = KM.get((row[1], row[2])) or KM[(row[2], row[1])]
        fare = 10 if km <= 5 else 18 if km <= 10 else 25
        row.extend([km, fare])
    return rows

def statement(rows):
    """Step 3: one block per card, read from the positions that steps 1 and 2 chose."""
    out = []
    for card in sorted({row[0] for row in rows}):
        mine = [row for row in rows if row[0] == card]
        out.append(f"Card {card}")
        for row in mine:
            out.append(f"  {row[3]} {row[1]} to {row[2]}, {row[4]} km: {row[5]}")
        out.append(f"  Total: {sum(row[5] for row in mine)}")
    return "\n".join(out)

C17;Market;Airport;08:10
C42;Market;Museum;09:05
C42;Museum;Market;17:15
C17;Airport;Museum;18:40

The second design prints the same statements. Its modules each hide one decision: trips.py the format of the log, fares.py the fare rule and statement.py the layout. The statement asks price(trip) for an amount and a short text that explains it, so it never learns how a fare is worked out.

from pathlib import Path

from fares import price
from statement import statement
from trips import read_trips

log = Path(__file__).with_name("log.txt").read_text()
print(statement(read_trips(log), price))

Card C17
  08:10 Market to Airport, 14 km: 25
  18:40 Airport to Museum, 11 km: 25
  Total: 50
Card C42
  09:05 Market to Museum, 4 km: 10
  17:15 Museum to Market, 4 km: 10
  Total: 20

"""Hides the format of the trip log. Other modules see Trip objects only."""
from dataclasses import dataclass


@dataclass(frozen=True)
class Trip:
    card: str
    origin: str
    destination: str
    start: str


def read_trips(text):
    trips = []
    for line in text.strip().splitlines():
        card, origin, destination, start = line.split(";")
        trips.append(Trip(card, origin, destination, start))
    return trips

"""Hides the fare rule. Other modules call price(trip) and get (amount, basis)."""

KM = {("Market", "Museum"): 4, ("Market", "Airport"): 14, ("Museum", "Airport"): 11}


def price(trip):
    km = KM.get((trip.origin, trip.destination)) or KM[(trip.destination, trip.origin)]
    amount = 10 if km <= 5 else 18 if km <= 10 else 25
    return amount, f"{km} km"

"""Hides the layout of a statement. It knows Trip's fields and what price() returns."""


def statement(trips, price):
    out = []
    for card in sorted({trip.card for trip in trips}):
        out.append(f"Card {card}")
        total = 0
        for trip in (t for t in trips if t.card == card):
            amount, basis = price(trip)
            total += amount
            out.append(f"  {trip.start} {trip.origin} to {trip.destination}, {basis}: {amount}")
        out.append(f"  Total: {total}")
    return "\n".join(out)

C17;Market;Airport;08:10
C42;Market;Museum;09:05
C42;Museum;Market;17:15
C17;Airport;Museum;18:40

Two change requests then arrive. First, the trip log will come as CSV with a header row. Second, fares will depend on the number of zones a trip travels in, not on kilometres. Each design was changed in the smallest way that keeps it correct, and every version was saved in its own folder. This program checks that both designs still print the same statements and uses difflib from the standard library to count what each change touched:

"""Measures how much of each design two change requests touch, and checks that both designs print the same.

Each design has three versions in its folder: v1, v2 (the log arrives as CSV) and v3 (fares by zones).
"""
import contextlib
import difflib
import io
import runpy
import sys
from pathlib import Path

HERE = Path(__file__).parent
DESIGNS = {"steps": "by processing steps", "decisions": "by hidden decisions"}
CHANGES = [
    ("v1", "v2", "Change 1: the trip log arrives as CSV with a header row"),
    ("v2", "v3", "Change 2: fares by zones travelled instead of kilometres"),
]


def run(folder):
    """What folder/main.py prints. Its modules are imported fresh, because every version has the same names."""
    for module in folder.glob("*.py"):
        sys.modules.pop(module.stem, None)
    sys.path.insert(0, str(folder))
    try:
        printed = io.StringIO()
        with contextlib.redirect_stdout(printed):
            runpy.run_path(str(folder / "main.py"), run_name="__main__")
        return printed.getvalue()
    finally:
        sys.path.remove(str(folder))


def edits(old, new):
    """{file name: (lines added, lines removed, diff)} of the .py files that differ (blank lines not counted)."""
    out = {}
    for path in sorted(old.glob("*.py")):
        before = path.read_text().splitlines()
        after = (new / path.name).read_text().splitlines()
        names = (f"{old.name}/{path.name}", f"{new.name}/{path.name}")
        diff = list(difflib.unified_diff(before, after, *names, n=0, lineterm=""))
        if diff:
            changed = [line for line in diff[2:] if not line.startswith("@@") and line[1:].strip()]
            added = sum(line.startswith("+") for line in changed)
            out[path.name] = (added, len(changed) - added, diff)
    return out


sys.dont_write_bytecode = True
for version in ("v1", "v2", "v3"):
    same = run(HERE / "steps" / version) == run(HERE / "decisions" / version)
    print(f"{version}: both designs print the same statements: {'yes' if same else 'NO'}")

for old, new, title in CHANGES:
    print(f"\n{title}")
    for design, label in DESIGNS.items():
        changed = edits(HERE / design / old, HERE / design / new)
        added = sum(a for a, _, _ in changed.values())
        removed = sum(r for _, r, _ in changed.values())
        count = f"{len(changed)} file" + ("s" if len(changed) != 1 else "")
        print(f"  {label}: {count} ({', '.join(changed)}), +{added} -{removed} lines")

print("\nThe edits of change 2:")
for design, label in DESIGNS.items():
    print(f"\n[{label}]")
    for _, _, diff in edits(HERE / design / "v2", HERE / design / "v3").values():
        print("\n".join(diff))

v1: both designs print the same statements: yes
v2: both designs print the same statements: yes
v3: both designs print the same statements: yes

Change 1: the trip log arrives as CSV with a header row
  by processing steps: 1 file (read_log.py), +5 -2 lines
  by hidden decisions: 1 file (trips.py), +4 -5 lines

Change 2: fares by zones travelled instead of kilometres
  by processing steps: 2 files (price.py, statement.py), +7 -6 lines
  by hidden decisions: 1 file (fares.py), +5 -4 lines

The edits of change 2:

[by processing steps]
--- v2/price.py
+++ v3/price.py
@@ -1 +1,2 @@
-KM = {("Market", "Museum"): 4, ("Market", "Airport"): 14, ("Museum", "Airport"): 11}
+ZONE = {"Market": 1, "Museum": 1, "Airport": 3}
+FARE = {1: 10, 2: 16, 3: 22}
@@ -5 +6 @@
-    """Step 2: append the distance and the fare, so each row is [card, from, to, start, km, fare]."""
+    """Step 2: append the zones travelled and the fare, so each row is [card, from, to, start, zones, fare]."""
@@ -7,3 +8,2 @@
-        km = KM.get((row[1], row[2])) or KM[(row[2], row[1])]
-        fare = 10 if km <= 5 else 18 if km <= 10 else 25
-        row.extend([km, fare])
+        zones = abs(ZONE[row[1]] - ZONE[row[2]]) + 1
+        row.extend([zones, FARE[zones]])
--- v2/statement.py
+++ v3/statement.py
@@ -8 +8,2 @@
-            out.append(f"  {row[3]} {row[1]} to {row[2]}, {row[4]} km: {row[5]}")
+            unit = "zone" if row[4] == 1 else "zones"
+            out.append(f"  {row[3]} {row[1]} to {row[2]}, {row[4]} {unit}: {row[5]}")

[by hidden decisions]
--- v2/fares.py
+++ v3/fares.py
@@ -3 +3,2 @@
-KM = {("Market", "Museum"): 4, ("Market", "Airport"): 14, ("Museum", "Airport"): 11}
+ZONE = {"Market": 1, "Museum": 1, "Airport": 3}
+FARE = {1: 10, 2: 16, 3: 22}
@@ -7,3 +8,3 @@
-    km = KM.get((trip.origin, trip.destination)) or KM[(trip.destination, trip.origin)]
-    amount = 10 if km <= 5 else 18 if km <= 10 else 25
-    return amount, f"{km} km"
+    zones = abs(ZONE[trip.origin] - ZONE[trip.destination]) + 1
+    basis = "1 zone" if zones == 1 else f"{zones} zones"
+    return FARE[zones], basis

The first change cost each design one file, because both had kept the reading of the log in one place. The second cost the step design two files and the other design one. The step design had told statement.py that the fifth field of a row is a distance in kilometres, so a new fare rule had to edit the layout code as well; even the docstring of price.py had to change, because it describes the row. In the second design the fare rule was one module’s secret, and the statement printed whatever explanation price() gave it.

The difference here is one file, because the program has three modules. It grows with every module that shares a decision: a monthly summary, a refund report and an export would each have read the same row layout. That is what Parnas showed, too. Information hiding does not make every change cheaper, only changes to the decisions you hid.

Separation of concerns

Edsger Dijkstra wrote about “the separation of concerns” in an essay of 1974: give your full attention to one aspect of a problem at a time, on its own terms, while remembering that it is only one aspect. In a design, the usual concerns are reading the input, the business rules, storage and presentation. Give each its own module, and a change in one, such as CSV instead of semicolons or a new statement layout, does not need you to read the others.

Fewer connections, not more classes

Lower coupling is about the connections between modules, not about how many classes you have. Splitting a class in three while all three still read and write one shared dictionary leaves the coupling where it was, now spread over three files. What lowers it:

• Narrower interfaces: a method that answers the caller’s question, such as units(code), instead of the data needed to work the answer out.
• Fewer shared data structures: a value object such as Trip, passed along, instead of a row whose layout every step must know.
• Data instead of control: two functions instead of one function and a flag.
• One home per decision: a module of plain functions hides a decision as well as a class does. fares.py above has no class at all.

Key takeaways

• Coupling is how strongly a connection ties two modules together. It grows with a connection’s complexity, with reaching past the interface, and from data to control to changing code.
• Cohesion (the 1974 paper’s binding) runs from coincidental to functional; aim for modules whose parts serve one job, and avoid grab-bag modules.
• Parnas’s criterion: list the difficult decisions and those likely to change, and give each module one of them to hide. A Python name with a leading underscore is a decision the class has not promised to keep.
• Measure a design by the changes it meets: the same change cost the step design two files and the hiding design one, and the gap widens as more modules share a decision.