Python Module 4 – Functions, scope, closures and decorators
Lambdas and higher-order functions
Write small lambdas for sort keys and callbacks, use map, filter and sorted with key, and see when itemgetter, attrgetter or a comprehension reads better.
What you will learn
- Pass functions as arguments by name, without calling them
- Write small lambdas for sort keys and callbacks
- Sort on several keys with a key function and with two stable sorts
- Choose between map(), filter(), a comprehension and operator.itemgetter()
- Decide when functools.reduce() helps and when a built-in such as sum or max is clearer
Before you start
What you’ll learn · Before you start
What you will learn
- Pass functions as arguments by name, without calling them
- Write small lambdas for sort keys and callbacks
- Sort on several keys with a key function and with two stable sorts
- Choose between map(), filter(), a comprehension and operator.itemgetter()
- Decide when functools.reduce() helps and when a built-in such as sum or max is clearer
Before you start
On this page
A contest leaderboard of six rows must be ordered by three things at once: most problems solved, then the lowest
penalty, then the name. sorted() does it in one line, as long as you hand it a function that turns each row into
something to compare. A function that takes another function, as sorted() takes its key, or returns one, is a
higher-order function: A function that takes another function as an argument, returns one, or both. sorted() is one: it calls the key function you pass once for every item., and a lambda: An expression that makes a small function without a name: lambda r: r[1] takes r and returns r[1]. Its body is one expression, never a statement, and its name is always <lambda>. is the shortest
way to write the small function you hand over: lambda r: r["solved"] takes r and returns r["solved"], written in
place, without a def.
Below you will pass functions without calling them, step through a sort that calls its key once per row, sort in two
directions with two stable sorts, watch a filter() result run out, swap lambdas for itemgetter(), follow a
callback, and see when functools.reduce() helps and when a built-in says it better.
A function is a value you can pass without calling it
In Python a function is an object, just like a number or a list. Its name refers to the object, and you can pass the
object to another function without calling it, by leaving out the parentheses. sorted(words, key=str.lower) hands
over the method str.lower itself; sorted() then calls it once for every word and compares the results:
# A function is a value: it can be passed to another function without calling it.
words = ["kiwi", "Banana", "fig", "apple", "Cherry"]
print(sorted(words)) # capital letters sort before small ones
print(sorted(words, key=str.lower)) # compare the lower-case forms instead
print(sorted(words, key=len)) # shortest first; equal lengths keep their order
def shout(text):
return text.upper() + "!"
def apply_twice(func, value):
"""A higher-order function: it takes another function as an argument."""
return func(func(value))
print(apply_twice(shout, "hey"))
print(type(shout).__name__, shout.__name__)Output
['Banana', 'Cherry', 'apple', 'fig', 'kiwi'] ['apple', 'Banana', 'Cherry', 'fig', 'kiwi'] ['fig', 'kiwi', 'apple', 'Banana', 'Cherry'] HEY!! function shout
Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 pass_a_function.py
Runs on this device, in your browser. The first run downloads Python (about 13.5 MB), which is kept for the next runs.
Your run, in this browser
- Without a key, capital letters sort before small ones, because the comparison uses each character’s code point and
"B"comes before"a". Withkey=str.lower, the words are compared in lower case but returned unchanged. key=lensorts by length."Banana"and"Cherry"both have six letters and stay in their original order, because Python’s sort keeps items with equal keys in the order they came in.apply_twice()is a higher-order function you wrote yourself. It receivedshout, a plain function, and called it twice;type(shout)shows that a function’s type is simplyfunction.
Step through the call apply_twice(shout, "hey") and watch which names refer to which object. The parentheses inside
apply_twice() are what make the function run:
The drawing shows the whole scene; its 5 steps are listed below.
Every step as text
- def shout runs. Running def shout(text) makes a function object and binds the name shout to it, the way an assignment binds a name to a number. The body has not run yet.
- Passed, not called. apply_twice(shout, "hey") writes shout without parentheses, so it passes the function object itself. The parameter func now refers to that same object.
- The first call. Inside apply_twice(), func(value) calls the object with 'hey'. The parentheses are what make it run, and shout returns 'HEY!'.
- The second call. func(func(value)) calls the object again on that result, which gives 'HEY!!'. apply_twice() returns it, and the program prints HEY!!.
- One object, two names. type(shout) is function and shout.__name__ is still 'shout'. Passing a function hands over the object; it is never copied or renamed, and sorted(words, key=str.lower) works the same way.
shout and func are two names for one function object. Passing a function hands the object over; nothing is
called, copied or renamed until the receiving code writes the parentheses.
Calling the function instead of passing it
sorted(words, key=str.lower()) calls str.lower at once, with no string to work on, and fails with
TypeError: unbound method str.lower() needs an argument before sorted() starts. The key is the function,
key=str.lower, never its result.
lambda writes a one-expression function in place
lambda parameters: expression creates a function whose body is that single expression; calling it returns the
value of the expression. lambda x: x * x does the same work as a def that returns x * x, and its parameters
follow the same rules, so a default value: The value after = in a def line, as in exponent=2, which a parameter takes when a call leaves it out. Python evaluates it once, when the def runs. More in “Parameters: defaults, keyword-only, positional-only” and *args work too. What cannot go inside
is a statement: no assignment with =, no return, no for loop, no try. A conditional expression is allowed,
because it is an expression: lambda n: "even" if n % 2 == 0 else "odd". A lambda also cannot carry type
annotations. The program squares 7 both ways, then shows the name each kind of function carries:
import traceback
def square(x):
return x * x
print(square(7), (lambda x: x * x)(7)) # the same work, with and without a name
parity = lambda n: "even" if n % 2 == 0 else "odd" # works, but PEP 8 asks for def
print(parity(10), parity(7))
print(square.__name__, parity.__name__)
# Tracebacks name the function that failed: here it is only "<lambda>".
try:
(lambda x: 10 / x)(0)
except ZeroDivisionError as err:
frames = traceback.extract_tb(err.__traceback__)
print("failed in:", [frame.name for frame in frames])Output
49 49 even odd square <lambda> failed in: ['<module>', '<lambda>']
Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 lambda_vs_def.py
Runs on this device, in your browser. The first run downloads Python (about 13.5 MB), which is kept for the next runs.
Your run, in this browser
The difference that matters in practice is the name. A function made with def square is called square in
tracebacks, profiles and its repr; every lambda is called <lambda>, so when one fails, the traceback cannot tell
you which of your lambdas it was, as the last line of the output shows. That is why PEP 8: Python's style guide: the naming, layout and comment conventions most Python code follows, such as snake_case names for functions, four-space indents and lines of at most 79 characters. More in “Docstrings, comments and PEP 8 style”,
Python’s style guide, asks for a def instead of an assignment such as parity = lambda n: …. The table sums the two
up:
| Criterion | lambda | def |
|---|---|---|
| Body | One expression, whose value is returned | Any statements, with return for the result |
| Name | Always <lambda>, in tracebacks and repr() | The name after def, such as square |
| Docstring and annotations | Neither | Both |
| Parameters | The rules of def: defaults, *args, **kwargs | The same rules |
| When to choose | A small function passed straight into a call, such as a sort key | Anything you name, reuse, document or test |
Key idea
A lambda is an ordinary function with one expression for a body and no name of its own: use it inline, and give a
function that deserves a name a def.
Checkpoint
Check: Passing a function as a value · Writing a lambda
sorted() calls the key once per item, then compares the keys
sorted(iterable, key=None, reverse=False) returns a new sorted list and leaves its input alone, and key and
reverse can only be passed by keyword; list.sort() takes the same two arguments, sorts the list in place and
returns None. The key function: A function that sorted(), list.sort(), min() and max() call once for every item; they then compare its results instead of the items. key=len compares the lengths. is called exactly once for each item, and the sort
then compares those keys instead of the items. A key can be any value that compares, and tuples compare item by
item, which is what makes sorting on several criteria a one-liner:
# A contest leaderboard: most problems solved first, then the lower penalty, then the name.
results = [
{"name": "Meera", "solved": 4, "penalty": 95},
{"name": "Arjun", "solved": 5, "penalty": 130},
{"name": "Kabir", "solved": 4, "penalty": 95},
{"name": "Divya", "solved": 5, "penalty": 102},
{"name": "Sana", "solved": 3, "penalty": 40},
{"name": "Rohan", "solved": 4, "penalty": 88},
]
board = sorted(results, key=lambda r: (-r["solved"], r["penalty"], r["name"]))
for place, r in enumerate(board, start=1):
print(place, r["name"], r["solved"], r["penalty"])
# How often does sorted() call the key? A key function that keeps a log shows it.
log = []
def penalty_key(row):
log.append(row["name"])
return row["penalty"]
sorted(results, key=penalty_key)
print("key calls:", len(log), "for", len(results), "rows")
# Text cannot be negated, so names from Z to A need two stable sorts:
# first by the minor key, then by the major key.
by_name = sorted(results, key=lambda r: r["name"], reverse=True)
by_solved = sorted(by_name, key=lambda r: r["solved"], reverse=True)
print([r["name"] for r in by_solved])Output
1 Divya 5 102 2 Arjun 5 130 3 Rohan 4 88 4 Kabir 4 95 5 Meera 4 95 6 Sana 3 40 key calls: 6 for 6 rows ['Divya', 'Arjun', 'Rohan', 'Meera', 'Kabir', 'Sana']
Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 leaderboard.py
Runs on this device, in your browser. The first run downloads Python (about 13.5 MB), which is kept for the next runs.
Your run, in this browser
sorted() compares the keys, not the rows
Text description of the diagram
The diagram has three boxes, one above the other.
- The first box holds three rows of the leaderboard in the order they were given: Meera with 4 problems solved and a penalty of 95 minutes, Arjun with 5 solved and 130 minutes, and Rohan with 4 solved and 88 minutes.
- An arrow labelled "key(row) = (-solved, penalty, name)" leads to the second box. sorted() calls the key function once for each row, and the keys are (-4, 95, 'Meera'), (-5, 130, 'Arjun') and (-4, 88, 'Rohan').
- An arrow labelled "compare the tuples" leads to the third box, the result. Tuples are compared item by item, so (-5, …) is smallest and Arjun is first. Rohan and Meera both start with -4, so their penalties decide: 88 is less than 95, and Rohan comes second, Meera third.
Negating the number solved turns "most solved first" into an ordinary smallest-first sort.
The key (-r["solved"], r["penalty"], r["name"]) puts the most problems solved first by negating them, because the
sort is always smallest first; then the lower penalty; then the name, so Kabir comes before Meera, who solved as many
problems with the same penalty. The diagram shows three of the rows. Step through the same sort on all six, one key
call per row and then the comparisons that decide the places:
The drawing shows the whole scene; its 8 steps are listed below.
Every step as text
- Six rows. Each row is a name, the number of problems solved and the penalty in minutes. Before it compares anything, sorted() calls the key function once on every row, in the order the rows came.
- Key call 1. The lambda gets Meera's row and returns the tuple (-4, 95, 'Meera'): the number solved, negated, then the penalty, then the name.
- Key call 2. Arjun solved 5, so his key starts with -5: negating the number solved makes more solved a smaller key, and the sort puts smaller keys first.
- Key calls 3 and 4. Kabir's key is (-4, 95, 'Kabir'), the same as Meera's apart from the name. Divya's is (-5, 102, 'Divya').
- Key calls 5 and 6. Sana's key is (-3, 40, 'Sana') and Rohan's (-4, 88, 'Rohan'). That makes six calls for six rows, and the lambda will not be called again.
- Compare the first items. Now the sort compares keys only, item by item. -5 is the smallest first item, so Divya and Arjun come first, and Divya's penalty of 102 is below Arjun's 130.
- Ties on the number solved. Rohan, Kabir and Meera all start with -4, so the penalties decide: 88 comes before 95. Kabir and Meera tie on 95 as well, and the names decide: 'Kabir' comes before 'Meera'.
- The order. Sana's -3 is the largest first item, so she is last. The rows come out in the order of their keys, Divya, Arjun, Rohan, Kabir, Meera and Sana, which is the leaderboard the program printed.
Six rows, six key calls, which is what the logging key function of the program counted. A key function that takes a
while, such as one that parses a timestamp, therefore costs one parse per row, however many comparisons the sort
makes. Two things to check in your own keys: every key must compare with every other, so a missing value such as
None among numbers raises TypeError, and a key that builds big objects for millions of rows uses memory for all
of them at once.
Key idea
sorted() calls the key once per item and then compares only the keys, so a tuple key sorts on several criteria at
once, and negating a number turns “highest first” into an ordinary smallest-first sort.
Two stable sorts order keys that go different ways
Negation only works for numbers. To sort names from Z to A while the number solved stays the main order, sort twice:
first by the minor key, the name, then by the major key, the number solved. This works because Python’s sort is a
stable sort: A sort that keeps items with equal keys in the order they came in. Python's sorted() and list.sort() are stable, so a second sort keeps the order of the first among ties.: items with equal keys keep the order they came in, and reverse=True keeps
that promise too. Step through the two sorts on the six rows, as name and number pairs, and watch the three rows that
solved 4:
Program · two_stable_sorts.py
rows = [("Meera", 4), ("Arjun", 5), ("Kabir", 4),
("Divya", 5), ("Sana", 3), ("Rohan", 4)]
by_name = sorted(rows, key=lambda r: r[0], reverse=True)
board = sorted(by_name, key=lambda r: r[1], reverse=True)
print(board)Variables
- changed in this step
Printed so far
Nothing printed yet[('Divya', 5), ('Arjun', 5), ('Rohan', 4), ('Meera', 4), ('Kabir', 4), ('Sana', 3)]
Every step was recorded from a run with Python 3.14.8 on macOS 26 arm64.
The drawing shows the variables when the program ends, and the program's 5 steps are listed below.
Change the program and run it here. Runs on this device, in your browser. The first run downloads Python (about 13.5 MB), which is kept for the next runs.
Your run, in this browser
Every step as text
- Start. No line has run yet, so there are no variables. Python starts with line 1.
- Line 1.
rows = [("Meera", 4), ("Arjun", 5), ("Kabir", 4), ("Divya", 5), ("Sana", 3), ("Rohan", 4)]makesrowsa list of 6 items. - Line 3.
by_name = sorted(rows, key=lambda r: r[0], reverse=True)setsby_nameto[('Sana', 3), ('Rohan', 4), ('Meera', 4), ('Kabir', 4), ('Divya', 5), ('Arjun', 5)]. The first sort orders by the minor key, the name, from Z to A.reverse=Truestill keeps rows with equal keys in their original order. - Line 4.
board = sorted(by_name, key=lambda r: r[1], reverse=True)setsboardto[('Divya', 5), ('Arjun', 5), ('Rohan', 4), ('Meera', 4), ('Kabir', 4), ('Sana', 3)]. Key point The second sort orders by the major key, the number solved. Rohan, Meera and Kabir all solved 4, so they keep the Z-to-A order the first sort gave them. - Line 5.
print(board)prints[('Divya', 5), ('Arjun', 5), ('Rohan', 4), ('Meera', 4), ('Kabir', 4), ('Sana', 3)]. That was the last line, so the program ends. Most solved first, and names from Z to A among equal numbers solved. A negated name is impossible, so two stable sorts do what one key cannot.
The second sort moved Divya and Arjun to the front, and it left Rohan, Meera and Kabir in the Z-to-A order the first
sort gave them. The names read Divya, Arjun, Rohan, Meera, Kabir, Sana, the last line leaderboard.py printed. Press
Edit, remove reverse=True from the first sort and run it: Kabir, Meera and Rohan now come out from A to Z.
Sorting by the main key first
Sorting by the number solved first and by the name second orders the rows by name, and the number solved only breaks ties between equal names, because the last sort decides the main order. Sort by the minor key first and the major key last.
Checkpoint
Check: Sorting with a key function · Two stable sorts
map() and filter() hand out their items only once
map(function, iterable) applies the function to every item of an iterable: A value a for loop can go through one item at a time, such as a string, a list, a tuple, a dict, a set or a range. A dict gives its keys. More in “for loops with range, enumerate and zip”, and
filter(function, iterable) keeps the items for which the function returns a true value. Both return an
An object that hands out the items of a collection one at a time, when asked for the next one. Once it has handed out the last item it is used up and gives nothing more.:
they produce their items one at a time, when something asks for them, and only once. The program converts five
temperatures with map(), keeps the hot ones with filter(), then does both again with comprehensions:
celsius = [31.5, 24.0, 38.2, 29.9, 35.0]
# map and filter take a function and an iterable, and return lazy iterators.
fahrenheit = map(lambda c: round(c * 9 / 5 + 32, 1), celsius)
print(fahrenheit.__class__.__name__)
print(list(fahrenheit))
print(list(fahrenheit)) # the iterator is used up: a second pass gives nothing
hot = filter(lambda c: c >= 30, celsius)
print(list(hot))
# The same results as comprehensions, which give a list you can reuse.
print([round(c * 9 / 5 + 32, 1) for c in celsius])
print([c for c in celsius if c >= 30])
# map shines when the function already exists: no lambda needed.
print(list(map(int, "12 7 30".split())))
print(list(filter(None, ["a", "", "b", None, 0, "c"]))) # None keeps truthy itemsOutput
map [88.7, 75.2, 100.8, 85.8, 95.0] [] [31.5, 38.2, 35.0] [88.7, 75.2, 100.8, 85.8, 95.0] [31.5, 38.2, 35.0] [12, 7, 30] ['a', 'b', 'c']
Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 map_filter_comprehension.py
Runs on this device, in your browser. The first run downloads Python (about 13.5 MB), which is kept for the next runs.
Your run, in this browser
The second list(fahrenheit) is empty, because the first one used the iterator up. Step through the same thing with
filter() and watch first and second:
Program · used_up.py
celsius = [31.5, 24.0, 38.2, 29.9, 35.0]
hot = filter(lambda c: c >= 30, celsius)
first = list(hot)
second = list(hot)
print(first, second)Variables
- changed in this step
- read next
Printed so far
Nothing printed yet[31.5, 38.2, 35.0] []
Every step was recorded from a run with Python 3.14.8 on macOS 26 arm64.
The drawing shows the variables when the program ends, and the program's 6 steps are listed below.
Change the program and run it here. Runs on this device, in your browser. The first run downloads Python (about 13.5 MB), which is kept for the next runs.
Your run, in this browser
Every step as text
- Start. No line has run yet, so there are no variables. Python starts with line 1.
- Line 1.
celsius = [31.5, 24.0, 38.2, 29.9, 35.0]makescelsiusa list of 5 items. - Line 2.
hot = filter(lambda c: c >= 30, celsius)setshotto<filter object>. filter() has not tested a single temperature yet. It returns an object that will test each one when something asks for the next item. - Line 3.
first = list(hot)setsfirstto[31.5, 38.2, 35.0]. list() asks for every item, so the filter tests all five temperatures and hands out the three that are at least 30. - Line 4.
second = list(hot)setssecondto[]. The bug The filter object has nothing left to give, so the second list() gets no items. No error says so. - Line 5.
print(first, second)prints[31.5, 38.2, 35.0] []. That was the last line, so the program ends. To read the values twice, keep them in a list, first here, or build a new filter. A comprehension gives a list from the start.
That surprises people who replace a list comprehension with map() and then loop over the result twice: no error,
just a second pass with nothing in it. The comprehensions give the same values as real lists, and many readers find
them clearer as soon as a lambda would be needed. map() is at its best when the function already exists, as in
map(int, "12 7 30".split()), a common way to read numbers from one line of input. filter(None, items) is a
shortcut that keeps only the true items, dropping "", None and 0. The table sets the three ways to apply a
function side by side; its second row is what the traced run above showed:
| Criterion | map() and filter() | List comprehension | Generator expression |
|---|---|---|---|
| Gives you | a lazy iterator | a list | a lazy iterator |
| Use the result twice | No: the second pass is empty | Yes | No: the second pass is empty |
| Filter and transform at once | Needs both, nested | One expression: [f(x) for x in xs if test(x)] | The same, in parentheses |
| Reads best | With an existing function: map(int, parts) | With an expression you would write as a lambda | Fed straight into sum(), max(), any() or a loop |
| When to choose | The function exists already and one pass is enough | You need the values more than once, or len() and indexing | Very many items, one pass, little memory |
itemgetter() and attrgetter() replace lambdas that only fetch
Lambdas that only fetch something, lambda s: s[1] or lambda p: p.rating, are so common that the operator
module has ready-made versions. itemgetter(1) returns a function that gives back item 1 of whatever it receives, and
attrgetter("rating") returns a function that reads the attribute rating. Given several positions or names, they
return a tuple, which is again a ready-made multi-level key:
from collections import namedtuple
from operator import attrgetter, itemgetter
scores = [("Asha", 82), ("Vikram", 91), ("Leela", 82), ("Imran", 77)]
print(sorted(scores, key=itemgetter(1), reverse=True))
print(max(scores, key=itemgetter(1)))
# With several items, itemgetter returns a tuple, which makes a sort key.
pick = itemgetter(1, 0)
print(pick(("Asha", 82)))
print(sorted(scores, key=pick))
Player = namedtuple("Player", "name rating")
players = [Player("Asha", 1520), Player("Ishaan", 1610), Player("Bela", 1475)]
print([p.name for p in sorted(players, key=attrgetter("rating"))])
# They describe themselves; a lambda only says it is a lambda.
print(itemgetter(1), attrgetter("rating"))
print((lambda s: s[1]).__name__)Output
[('Vikram', 91), ('Asha', 82), ('Leela', 82), ('Imran', 77)]
('Vikram', 91)
(82, 'Asha')
[('Imran', 77), ('Asha', 82), ('Leela', 82), ('Vikram', 91)]
['Bela', 'Asha', 'Ishaan']
operator.itemgetter(1) operator.attrgetter('rating')
<lambda>
Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 itemgetter_attrgetter.py
Runs on this device, in your browser. The first run downloads Python (about 13.5 MB), which is kept for the next runs.
Your run, in this browser
sorted(scores, key=itemgetter(1), reverse=True)sorts by the score, highest first; Asha and Leela both have 82 and keep their original order.itemgetter(1, 0)builds the tuple(score, name), so ties are broken by name.attrgetterworks on objects, here a named tuple, and accepts dotted names such as"address.city"too.- Printed, they say exactly what they do:
operator.itemgetter(1). A lambda only says<lambda>.
Each fetching lambda has an operator function that does the same job:
| Criterion | The lambda | The operator function |
|---|---|---|
| Item 1 | lambda s: s[1] | itemgetter(1) |
| Item 1, then item 0 | lambda s: (s[1], s[0]) | itemgetter(1, 0) |
| An attribute | lambda p: p.rating | attrgetter("rating") |
| A dotted attribute | lambda p: p.address.city | attrgetter("address.city") |
| When to choose | The key computes something: a negation, a length, a lower-case form | The key only fetches items or attributes |
The Sorting HOWTO calls these functions easier to use and faster than the lambdas they replace. Still reach for a lambda when the key has to compute something: a negation, a length, a lower-case version of a string or a value from two fields.
Checkpoint
Check: map() and filter() hand out items once · itemgetter() and attrgetter()
A callback is a function that other code calls later
A callback: A function handed to other code, which calls it at the right moment, such as the function re.sub() calls for every match or the factory a defaultdict calls for a missing key. is a function you hand to some code so that it can call it at the right moment. Python’s standard library takes callbacks in many places, and a lambda keeps a short one next to the call that uses it:
import re
from collections import defaultdict
# re.sub calls the function back once for every match and uses what it returns.
text = "order 12 shipped in 3 boxes"
print(re.sub(r"\d+", lambda m: str(int(m.group()) * 2), text))
# defaultdict calls its factory whenever a missing key is read.
stock = defaultdict(lambda: "not stocked")
stock["tea"] = "12 packs"
print(stock["tea"], "|", stock["coffee"])
# min and max take the same key functions as sorted.
cities = {"Pune": 18.5, "Leh": -2.0, "Chennai": 29.1}
print(min(cities, key=cities.get), max(cities.items(), key=lambda kv: kv[1]))Output
order 24 shipped in 6 boxes
12 packs | not stocked
Leh ('Chennai', 29.1)
Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 callbacks.py
Runs on this device, in your browser. The first run downloads Python (about 13.5 MB), which is kept for the next runs.
Your run, in this browser
re.sub()accepts a function instead of replacement text, calls it once for every match with the match object, and puts its return value in place of the match: every number in the sentence was doubled.defaultdictcalls its factory, herelambda: "not stocked", whenever a missing key is read with square brackets, asstock["coffee"]is, and stores the result;stock.get("coffee")never calls it.min(cities, key=cities.get)passes a bound method as the key: the city with the smallest value wins. Withmax(cities.items(), key=lambda kv: kv[1])you get the key and the value back together.
Step through the re.sub() call as messages between your code, re.sub() and the lambda it calls back:
The drawing shows the whole scene; its 6 steps are listed below.
Every step as text
- Hand it over. Your code passes re.sub() three things: the pattern for a run of digits, the lambda itself, not called, and the text 'order 12 shipped in 3 boxes'.
- The first match. re.sub() finds the first run of digits, 12, and calls the lambda with its match object. Your code is not involved in this call.
- The replacement. The lambda reads the match with m.group(), doubles it and returns the text '24'. re.sub() puts '24' where '12' was.
- The second match. The next run of digits is 3, so re.sub() calls the lambda again, with a new match object.
- Another replacement. The lambda returns '6'. There are no more digits in the text, so re.sub() makes no more calls: two matches, two calls.
- The result. re.sub() returns the new string, and the program prints it. Your code handed over the lambda but never called it; re.sub() decided when and how often, which is what makes it a callback.
Your code never called the lambda: re.sub() did, twice, once for each number it found. Functions can also be
returned. def adder(n): return lambda x: x + n hands back a new function on every call; adder(5)(10) is 15. How
the returned lambda still knows n after adder has finished is the subject of the next lesson, on closures.
reduce() folds a sequence into one value
functools.reduce(function, iterable, initial) produces a fold: Combining a sequence into one value two items at a time: reduce(add, [3, 8, 1]) works out (3 + 8) + 1. sum(), max() and math.prod() are folds with names of their own.: it calls the two-argument
function on the first two items, then on that result and the third item, and so on to the end. Given an initial
value, it starts from that instead, which is also the answer for an empty sequence; without one, an empty sequence
raises TypeError:
import math
import operator
from functools import reduce
nums = [3, 8, 1, 9, 4]
# reduce folds a sequence: ((((3 + 8) + 1) + 9) + 4)
print(reduce(operator.add, nums), sum(nums))
print(reduce(lambda a, b: a if a > b else b, nums), max(nums))
print(reduce(operator.mul, nums), math.prod(nums))
# The initial value is the result for an empty sequence (since 3.14 also by keyword).
print(reduce(operator.add, [], 0), reduce(operator.add, [], initial=0))
try:
reduce(operator.add, [])
except TypeError as err:
print("TypeError:", err)
# any and all stop early; reduce always reads everything.
print(any(n > 5 for n in nums), all(n > 0 for n in nums))Output
25 25 9 9 864 864 0 0 TypeError: reduce() of empty iterable with no initial value True True
Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 reduce_and_builtins.py
Runs on this device, in your browser. The first run downloads Python (about 13.5 MB), which is kept for the next runs.
Your run, in this browser
Step through the first fold of the program, reduce(operator.add, nums) on [3, 8, 1, 9, 4]:
The drawing shows the whole scene; its 5 steps are listed below.
Every step as text
- The first call. Without an initial value, reduce() starts with the first two items and calls the function on them. operator.add(3, 8) returns 11.
- The next item. The result so far becomes the first argument of the next call, with the next item, 1, as the second. 11 + 1 is 12.
- And the next. The same again with 9. reduce() keeps only the result so far, never a list of partial results.
- The last item. 21 + 4 is 25. The list has no items left, so that was the last call, the fourth for five items.
- One value. reduce() returns 25, ((((3 + 8) + 1) + 9) + 4). sum(nums) gives the same answer and says what it does in its name, which is why it is the better choice here.
Four calls fold five items into 25, and every fold in that program has a clearer built-in: sum(), max() or
math.prod(), while min() and, for strings, "".join() name two more common folds. any() and all() also stop
at the first item that settles the answer, which reduce() never does. Use reduce() for a fold that has no
built-in and reads naturally as “combine two things into one”, such as merging a list of dictionaries or composing
functions.
Version note
Since Python 3.14, initial can also be passed by keyword, as in reduce(operator.add, [], initial=0), which the
program does on its fourth line of output.
Checkpoint
Check: Callbacks · Folding with reduce()
Practise: rank records and compose functions
The exercise asks for two functions: top_n() picks the best records by any key function, with ties ordered by name,
and compose() builds one function out of several, applied from right to left. Write both in the editor; the sample
tests check ties, empty results and built-in functions such as str and abs:
Exercise · Easy · Python
Pick the top records and compose functions
Write two functions in ranking.py:
top_n(records, n, key)returns a new list with thenrecords whosekey(record)is highest, best first. Records with the same key are ordered by their"name"value, from A to Z. Whennis larger than the number of records, return all of them in that order; whennis 0 or negative, return an empty list. The list passed in must not change.keycan be any function, such aslambda r: r["points"].compose(*funcs)returns a new function that applies the functions from right to left:compose(f, g, h)(x)isf(g(h(x))). With no functions at all,compose()returns a function that gives back its argument unchanged.
The tests sort players by their points, by the length of their names and with records that tie, and they compose built-in functions such as str and abs as well as lambdas.
Starter code · ranking.py
def top_n(records, n, key):
"""Return the n records with the highest key(record), best first; ties by "name"."""
return records[:n]
def compose(*funcs):
"""Return a function that applies funcs from right to left."""
def composed(value):
return value
return composedThe sample tests · test_ranking.py
from ranking import compose, top_n
PLAYERS = [
{"name": "Meera", "points": 40},
{"name": "Arjun", "points": 55},
{"name": "Kabir", "points": 40},
{"name": "Divya", "points": 55},
{"name": "Sana", "points": 12},
]
def names(records):
return [r["name"] for r in records]
def test_top_three():
"""returns the three best players, best first, ties from A to Z"""
assert names(top_n(PLAYERS, 3, key=lambda r: r["points"])) == ["Arjun", "Divya", "Kabir"]
def test_n_larger_than_the_list():
"""returns every record when n is larger than the list"""
assert names(top_n(PLAYERS, 10, key=lambda r: r["points"])) == ["Arjun", "Divya", "Kabir", "Meera", "Sana"]
def test_zero_or_negative_n():
"""returns an empty list when n is 0 or negative"""
assert top_n(PLAYERS, 0, key=lambda r: r["points"]) == []
assert top_n(PLAYERS, -2, key=lambda r: r["points"]) == []
def test_any_key_function():
"""works with any key, such as the length of the name"""
assert names(top_n(PLAYERS, 2, key=lambda r: len(r["name"]))) == ["Arjun", "Divya"]
assert names(top_n(PLAYERS, 1, key=lambda r: -r["points"])) == ["Sana"]
def test_input_unchanged():
"""does not change the list it was given"""
before = [dict(p) for p in PLAYERS]
top_n(PLAYERS, 2, key=lambda r: r["points"])
assert PLAYERS == before
def test_compose_right_to_left():
"""applies the functions from right to left"""
inc = lambda x: x + 1
double = lambda x: x * 2
assert compose(inc, double)(5) == 11
assert compose(double, inc)(5) == 12
assert compose(str, abs, int)("-42") == "42"
def test_compose_nothing():
"""compose() gives back its argument unchanged"""
same = compose()
assert same(7) == 7
assert same("text") == "text"A hint
sorted() returns a new list and never changes the one you give it, and its sorts are stable. So sort by name first, then sort that result by key with reverse=True: records with the same key keep their A-to-Z order. A slice such as [:n] returns at most n items. For compose, the inner function can loop over reversed(funcs), passing each result on to the next function.
Results of the sample tests
| Test | Result | Details |
|---|
What your code printed
The sample tests run on this device, in your browser (Pyodide): nothing is sent to mysmartcopilot.com. The first run downloads Python (about 13.5 MB), which is kept for the next runs. A check in your browser is feedback for you, not proof that the code is right for every input.
Then test yourself on keys, iterators, callbacks and folds. The session picks the idea you are least sure of, asks a harder question once you get one right, and explains every choice you make:
Practice that adapts to you
Practise this lesson
20 questions about 8 ideas of this lesson. Each answer is explained at once, and what you get right is kept in this browser.
Interview questions
Fresher · What is the difference between a lambda and a function defined with def?
Both create ordinary function objects: a lambda can be passed, stored and called exactly like a function made with
def, and it sees the variables around it in the same way. The difference is in what you can write and in the name.
A lambda’s body is one expression whose value is returned, so it cannot hold statements such as assignments, loops,
return, raise or try, and its parameters cannot have annotations. And a lambda has no name of its own: its
__name__ is <lambda>, which is all a traceback or a profiler can show. So a lambda suits a short function passed
straight into a call, such as a sort key or a callback, and def suits anything you name, reuse, document or test.
PEP 8 asks for def whenever a lambda would be assigned to a name.
Model answers open with any pass: ₹79 for 1 week ($2.99), one payment that never renews.
-
Fresher to mid
In a code review, valid = [parse(r) for r in rows if r] became valid = map(parse, filter(None, rows)). The tests pass, but a report that loops over valid twice now prints an empty second section. Explain what happened and how you would review such a change.
-
Mid to senior
You design run_pipeline(records, steps), where steps is a list of functions applied to each record in turn, some of them lambdas. A step fails on record 41,207 of 2 million and the traceback only says <lambda>. How do you make failures name the step and the record, and keep the steps easy to test?
Key takeaways
- A function is a value: pass it by its name, without parentheses, and the code you pass it to decides when to call it.
lambda params: expressionmakes a function with one expression for a body and the name<lambda>; use it inline, and usedeffor anything that deserves a name.sorted()calls its key once per item and compares the keys, so a tuple key sorts on several criteria and negation reverses a number.- Python’s sort is stable, so sorting by the minor key and then by the major key orders text in both directions.
map()andfilter()return iterators that are used up after one pass; a comprehension gives a list, anditemgetter()andattrgetter()replace lambdas that only fetch.- A callback is a function other code calls for you, and
functools.reduce()folds a sequence, wheresum(),max(),math.prod(),any()andall()usually say the same thing more clearly.
Terms in this lesson
- callback
- A function handed to other code, which calls it at the right moment, such as the function re.sub() calls for every match or the factory a defaultdict calls for a missing key.
- fold
- Combining a sequence into one value two items at a time: reduce(add, [3, 8, 1]) works out (3 + 8) + 1. sum(), max() and math.prod() are folds with names of their own.
- higher-order function
- A function that takes another function as an argument, returns one, or both. sorted() is one: it calls the key function you pass once for every item.
- key function
- A function that sorted(), list.sort(), min() and max() call once for every item; they then compare its results instead of the items. key=len compares the lengths.
- lambda
- An expression that makes a small function without a name: lambda r: r[1] takes r and returns r[1]. Its body is one expression, never a statement, and its name is always <lambda>.
- stable sort
- A sort that keeps items with equal keys in the order they came in. Python's sorted() and list.sort() are stable, so a second sort keeps the order of the first among ties.
References
- The Python Tutorial: lambda expressions (Python Software Foundation)
- The Python Language Reference: lambdas (Python Software Foundation)
- Sorting Techniques (Python HOWTO) (Python Software Foundation)
- Built-in Functions: sorted(), map() and filter() (Python Software Foundation)
- operator: standard operators as functions (Python Software Foundation)
- Functional Programming HOWTO (Python Software Foundation)
- functools.reduce() (Python Software Foundation)
- What's New in Python 3.14: functools (Python Software Foundation)
- PEP 8: Style Guide for Python Code, programming recommendations (Python Software Foundation)
- re.sub() (Python Software Foundation)
- collections.defaultdict (Python Software Foundation)
Related tools
Report a problem with this lesson
Kept only in this browser. Your Learn progress