When patterns hurt: cost, YAGNI and overuse

Every design pattern buys flexibility along one axis and pays for it with structure: more types, more indirection, more places to look when something goes wrong. When that axis really varies, the trade is a good one. When it never varies, the pattern is pure cost, paid by everyone who reads, tests or changes the code. “You aren’t gonna need it” (YAGNI), a rule from Extreme Programming, says not to build for changes you only imagine, and Joshua Kerievsky’s Refactoring to Patterns (2004) argues that refactoring toward a pattern when the need shows up beats putting patterns in early.

This lesson measures what three needless patterns add to a ten-line feature, lists the signs of overuse with a small checker that finds some of them, gives a rule for deciding between now, later and never, and shows how to write the consequences of a decision in two sentences.

What a pattern costs

The costs come in a few kinds, and most of them are paid on every read rather than once:

• Indirection. A call goes through an interface, a factory or a wrapper before it reaches the code that does the work, so a reader follows several hops to find one formula.
• More types and files. Each participant of a pattern is a name to learn, a file to open and a test to keep.
• Harder navigation. “Go to definition” on a call through an interface lands on the interface, not on the code that runs; the reader has to find which implementation is wired in.
• Runtime overhead. A virtual call, an extra allocation, a lookup in a registry. It is usually too small to matter, but it adds up in a loop that runs millions of times.
• A vocabulary to share. A team that does not know the pattern pays to learn it before the code reads easily.

Here is one small feature, a delivery fee, written plainly and then behind three patterns stacked the way they often are: a Strategy that never varies, a Factory that makes it, and a Singleton that holds it. A third program runs both on the same eight orders and measures them:

"""Runs both designs on the same cases, then measures what the layers cost: classes, functions, lines and call depth."""
import ast
import sys

import layered
import plain

CASES = [(499, 3), (500, 9), (120, 0), (120, 3), (120, 4), (120, 10), (999, 1), (0, 25)]


def code_lines(module):
    """Lines that hold code: not blank, not a comment, not part of a docstring."""
    source = open(module.__file__).read()
    tree = ast.parse(source)
    docstrings = set()
    for node in ast.walk(tree):
        if isinstance(node, (ast.Module, ast.ClassDef, ast.FunctionDef)) and ast.get_docstring(node) is not None:
            first = node.body[0]
            docstrings.update(range(first.lineno, first.end_lineno + 1))
    lines = source.splitlines()
    return sum(1 for n, line in enumerate(lines, 1) if line.strip() and not line.strip().startswith("#") and n not in docstrings)


def counts(module):
    tree = ast.parse(open(module.__file__).read())
    classes = sum(isinstance(node, ast.ClassDef) for node in ast.walk(tree))
    functions = sum(isinstance(node, ast.FunctionDef) for node in ast.walk(tree))
    return classes, functions


def call_depth(function, *args):
    """How many Python functions are running at once at the deepest point of one call."""
    depth = deepest = 0

    def profile(frame, event, arg):
        nonlocal depth, deepest
        if event == "call":
            depth += 1
            deepest = max(deepest, depth)
        elif event == "return":
            depth -= 1

    sys.setprofile(profile)
    try:
        function(*args)
    finally:
        sys.setprofile(None)
    return deepest


same = [plain.delivery_fee(s, km) for s, km in CASES] == [layered.delivery_fee(s, km) for s, km in CASES]
print("fees:", [plain.delivery_fee(s, km) for s, km in CASES])
print(f"{len(CASES)} cases, same answers from both designs: {same}")
print(f"{'design':<8} {'classes':>7} {'functions':>9} {'code lines':>10} {'call depth':>10}")
for module in (plain, layered):
    classes, functions = counts(module)
    print(f"{module.__name__:<8} {classes:>7} {functions:>9} {code_lines(module):>10} {call_depth(module.delivery_fee, 120, 10):>10}")

fees: [30, 0, 30, 30, 35, 65, 0, 140]
8 cases, same answers from both designs: True
design   classes functions code lines call depth
plain          0         1          4          1
layered        4         7         28          3

"""The delivery fee, written plainly."""


def delivery_fee(subtotal, km):
    """Free from a subtotal of 500; otherwise 30, plus 5 for every km beyond the first 3."""
    if subtotal >= 500:
        return 0
    return 30 + 5 * max(0, km - 3)

"""The same fee behind three patterns: a Strategy that never varies, made by a Factory, held by a Singleton."""
from abc import ABC, abstractmethod


class FeeStrategy(ABC):
    @abstractmethod
    def compute(self, subtotal, km):
        """The fee for an order."""


class StandardFeeStrategy(FeeStrategy):
    def compute(self, subtotal, km):
        if subtotal >= 500:
            return 0
        return 30 + 5 * max(0, km - 3)


class FeeStrategyFactory:
    @staticmethod
    def create(kind="standard"):
        if kind == "standard":
            return StandardFeeStrategy()
        raise ValueError(f"unknown fee strategy: {kind}")


class FeeCalculator:
    _instance = None

    def __init__(self):
        self._strategy = FeeStrategyFactory.create()

    @classmethod
    def instance(cls):
        if cls._instance is None:
            cls._instance = cls()
        return cls._instance

    def fee(self, subtotal, km):
        return self._strategy.compute(subtotal, km)


def delivery_fee(subtotal, km):
    """Free from a subtotal of 500; otherwise 30, plus 5 for every km beyond the first 3."""
    return FeeCalculator.instance().fee(subtotal, km)

Both designs give the same eight answers. The layered one has 4 classes where the plain one has none, 7 functions instead of 1 and 28 lines of code instead of 4, and one fee goes three Python functions deep instead of one. The extra structure would pay for itself if a second fee rule arrived and had to be chosen per order; until then it buys nothing, and every reader pays for it. Python’s own design rules say it in two lines of PEP 20, which you can read from the this module that ships with Python:

"""Two lines of the Zen of Python (PEP 20), read from the this module that ships with every Python."""
import codecs
import contextlib
import io

with contextlib.redirect_stdout(io.StringIO()):
    import this  # importing it prints the Zen; the module keeps the text, ROT13-encoded, in this.s

aphorisms = codecs.decode(this.s, "rot13").splitlines()[2:]  # after the title and a blank line
for number in (3, 17):
    print(f"{number:>2}. {aphorisms[number - 1]}")

 3. Simple is better than complex.
17. If the implementation is hard to explain, it's a bad idea.

The signs of overuse

Kent Beck named the idea of a code smell while helping Martin Fowler with his book Refactoring, and Fowler’s definition is short enough to quote:

A code smell is a surface indication that usually corresponds to a deeper problem in the system.

Over-engineering has smells of its own:

• A factory with one product. It always returns the same class, so calling the class directly says the same thing with less code.
• An interface with one implementation and no test double. Nothing is substituted, so the interface is only an extra file to open. With a test double, it is a real second variant.
• A Strategy that is always the same strategy. If the program never chooses, there is no family of algorithms to choose from.
• A Singleton used to avoid passing a parameter. It turns a visible dependency into hidden global state that tests share and that every function quietly depends on.
• A Visitor over a hierarchy that changes every sprint. Visitor makes new operations cheap and new classes expensive, so a growing hierarchy turns every new class into an edit to every visitor.
• Layers that only forward calls. A service that calls a manager that calls a helper, each adding nothing.

The general form of these smells is what Refactoring, by Martin Fowler with Kent Beck, calls speculative generality: hooks, parameters and abstract classes added for cases that nobody has asked for. Some of these smells can be found by reading code mechanically. This checker parses two pull requests with Python’s ast module and reports abstract classes with a single implementation, factories that only ever make one class, and Singletons:

"""A small checker for three signs of pattern overuse, run on both pull requests.

It reads the code with the ast module and reports an abstract class with only one implementation, a factory that only
ever makes one class, and a Singleton. A finding is a question to ask, not a verdict: a test double can justify an
interface, and the checker cannot see the tests.
"""
import ast

import pr_notifier
import pr_shipping


def is_abstract(cls):
    return any(isinstance(b, ast.Name) and b.id == "ABC" for b in cls.bases)


def findings(path):
    tree = ast.parse(open(path).read())
    classes = [node for node in tree.body if isinstance(node, ast.ClassDef)]
    found = []
    for cls in classes:
        if is_abstract(cls):
            subclasses = [c.name for c in classes if any(isinstance(b, ast.Name) and b.id == cls.name for b in c.bases)]
            if len(subclasses) == 1:
                found.append(f"{cls.name} is abstract with one implementation, {subclasses[0]}")
        if cls.name.endswith("Factory"):
            made = {n.value.func.id for n in ast.walk(cls) if isinstance(n, ast.Return) and isinstance(n.value, ast.Call) and isinstance(n.value.func, ast.Name)}
            if len(made) == 1:
                found.append(f"{cls.name} only ever makes {made.pop()}")
        names = {t.id for n in cls.body if isinstance(n, ast.Assign) for t in n.targets if isinstance(t, ast.Name)}
        if "_instance" in names:
            found.append(f"{cls.name} is a Singleton: one global instance")
    return found


for path in ("pr_shipping.py", "pr_notifier.py"):
    result = findings(path)
    print(f"{path}: {len(result)} finding(s)")
    for line in result:
        print(f"  - {line}")

print("pr_shipping.quote:", pr_shipping.quote("domestic", 1500), pr_shipping.quote("international", 1500))
print("pr_notifier.send_welcome:", pr_notifier.send_welcome("asha@example.com"))

pr_shipping.py: 0 finding(s)
pr_notifier.py: 2 finding(s)
  - Notifier is abstract with one implementation, EmailNotifier
  - NotifierFactory only ever makes EmailNotifier
pr_shipping.quote: 60 1650
pr_notifier.send_welcome: email to asha@example.com: Welcome!

"""Pull request 1: shipping rules for the two markets the shop sells to today, chosen per order."""
from abc import ABC, abstractmethod


class ShippingRule(ABC):
    @abstractmethod
    def cost(self, grams):
        """The cost of shipping a parcel of this many grams."""


class DomesticRule(ShippingRule):
    def cost(self, grams):
        return 40 + 10 * ((grams + 999) // 1000)


class InternationalRule(ShippingRule):
    def cost(self, grams):
        return 900 + 250 * ((grams + 499) // 500)


RULES = {"domestic": DomesticRule(), "international": InternationalRule()}


def quote(market, grams):
    return RULES[market].cost(grams)

"""Pull request 2: notifications, written to be "ready for any channel"."""
from abc import ABC, abstractmethod


class Notifier(ABC):
    @abstractmethod
    def notify(self, user, text):
        """Send text to user."""


class EmailNotifier(Notifier):
    def notify(self, user, text):
        return f"email to {user}: {text}"


class NotifierFactory:
    def create(self):
        return EmailNotifier()


def send_welcome(user):
    return NotifierFactory().create().notify(user, "Welcome!")

The first pull request has a Strategy with two shipping rules that the shop uses today, chosen per order, so nothing is flagged: the interface earns its place. The second has a notifier interface with one implementation and a factory that only ever makes it, both “ready for any channel”. A finding is a question to ask the author, not a verdict. The checker cannot see the tests, and a test double that needs the interface would justify it, so the review asks “which second channel, and when?” before asking for the layers to go.

Now, later or never

The decision turns on the variation the pattern would protect, and whether it is real:

• Now, when the variation exists today: two variants are in use, or a test needs a double through the same seam.
• Later, when the variation is likely but not here yet: someone has named the second case, or it is planned. Write the plain version, keep it covered by tests and easy to change, and refactor toward the pattern when the second case arrives. Kerievsky’s book is a catalogue of exactly those refactorings, toward patterns and away from them.
• Never, when the variation is imagined, with no user and no date. Leave the pattern out until it becomes real.
• Away, when a pattern is in place but its variation has died, and only one variant is left. Inline it.

“Later” only works if changing the code later is cheap. Fowler’s note on YAGNI makes the point: the rule is about capabilities built for presumed needs, and it does not apply to work that makes the software easier to modify, such as refactoring and self-testing code. Skipping the tests is not the same as following YAGNI.

The consequences in two sentences

A pattern you cannot justify briefly is usually one you do not need. Write each design decision down as two sentences: what the pattern makes easy, and what it costs, with the condition under which you would undo it.

• “We use Strategy so that a new shipping market needs no change to quote(). It costs one class per market, and we will remove it if the shop goes back to selling in one market.”
• “We put clock access behind a Clock interface so that tests of expiring offers can run at any moment. It costs one interface and a fixed clock in the tests, and nothing in production.”

Try the same with the second pull request: “We use a factory so that … needs no change to send_welcome().” No variation fits the gap, and that is the finding. The two sentences force you to name the axis of change, which is the part of a pattern people most often forget.

In a design interview this habit is worth as much as knowing the patterns. “The delivery fee is one function, because there is one fee rule; if a second rule had to be chosen per order, I’d turn the rules into strategies picked by the order’s market” is a strong answer: it names the variation you looked for, why it is not there yet, and what you would do when it appears.

Practise: remove three layers that never vary

The loyalty points module hides a one-line rule behind an interface with one implementation, a factory and a Singleton. Simplify it without changing what points_for returns. The sample tests pin down today’s answers, the way characterization tests do before any refactoring, and confirm that no class is left.

"""Loyalty points behind three layers that never vary: a Strategy, its Factory and a Singleton engine."""
from abc import ABC, abstractmethod


class PointsStrategy(ABC):
    @abstractmethod
    def points(self, amount):
        """Points earned for an order of this amount."""


class StandardPoints(PointsStrategy):
    def points(self, amount):
        earned = amount // 100
        if amount >= 2000:
            earned *= 2
        return min(earned, 500)


class PointsStrategyFactory:
    @staticmethod
    def create():
        return StandardPoints()


class PointsEngine:
    _instance = None

    def __init__(self):
        self._strategy = PointsStrategyFactory.create()

    @classmethod
    def instance(cls):
        if cls._instance is None:
            cls._instance = cls()
        return cls._instance

    def award(self, amount):
        return self._strategy.points(amount)


def points_for(amount):
    """Loyalty points for an order: 1 per full 100 spent, doubled from 2,000, at most 500."""
    return PointsEngine.instance().award(amount)

import ast
import inspect

import points
from points import points_for

# What points_for returns today, written down before anything changes (characterization tests).
TODAY = [
    (0, 0), (99, 0), (100, 1), (199, 1), (1999, 19), (2000, 40), (2550, 50),
    (24999, 498), (25000, 500), (25100, 500), (1_000_000, 500),
]


def test_small_orders():
    """one point per full 100"""
    for amount, expected in TODAY[:5]:
        assert points_for(amount) == expected, f"points_for({amount})"


def test_doubling_from_2000():
    """orders of 2,000 or more earn double"""
    for amount, expected in TODAY[5:8]:
        assert points_for(amount) == expected, f"points_for({amount})"


def test_cap_of_500():
    """never more than 500 points"""
    for amount, expected in TODAY[8:]:
        assert points_for(amount) == expected, f"points_for({amount})"


def test_the_layers_are_gone():
    """no interface, factory or Singleton is left: the module defines no class"""
    tree = ast.parse(inspect.getsource(points))
    classes = [node.name for node in ast.walk(tree) if isinstance(node, ast.ClassDef)]
    assert classes == [], f"classes left in points.py: {classes}"
    assert inspect.isfunction(points_for)

Interview questions

Fresher · What is YAGNI, and how does it apply to design patterns?

YAGNI stands for “you aren’t gonna need it”, a rule from Extreme Programming: do not build a capability until a real need for it exists. For patterns it means not adding a seam, such as an interface, a factory or a plugin point, for a variation you only imagine. The extra structure costs something on every read and every change, the variation may never come, and when it does come it often looks different from the guess. The plain version is easy to refactor toward the pattern once a second case appears, provided the code has tests. YAGNI does not argue against that kind of work: refactoring and self-testing code make the software easier to change, which is what makes waiting safe. So apply a pattern when its variation exists, and not before.

Key takeaways

• A pattern buys flexibility along one axis with extra structure; if the axis never varies, the structure is pure cost.
• The layered fee had 4 classes, 7 functions and 28 lines where the plain one had 1 function and 4 lines, for the same answers.
• Smells of overuse: a factory with one product, an interface with one implementation and no test double, a Strategy that never changes, a Singleton instead of a parameter, a Visitor over a churning hierarchy.
• Apply a pattern now when its variation exists, later by refactoring when it arrives, never while it is imagined, and refactor away from it when the variation dies.
• Justify each pattern in two sentences: what it makes easy and what it costs, with the condition for undoing it.