Python Module 10 – Iterators, generators and functional tools

itertools: the iteration toolkit

Slice, chain, group, batch and pair iterators with itertools, generate combinations and permutations, and estimate how fast those search spaces grow.

Intermediate
20 minutes
Examples run with Python 3.14.8 and Pyodide 314.0.7
By MySmartCoPilot

Free: Every itertools function returns a lazy iterator · groupby() groups runs of equal keys, so sort first · batched() cuts fixed-size groups and pairwise() pairs neighbours · 2 more sections. With a pass: 2 in-depth parts and 3 interview answers, about 12 minutes.

What you will learn

Pick the itertools function for a job, from islice and chain to groupby and batched
Group sorted records with groupby and process them in batches with batched
Generate combinations, permutations and products, and estimate their size first
Avoid the traps of groupby, tee and accumulate
Limit a search space by pruning partial results instead of filtering every result

Before you start

What you’ll learn · Before you start

What you will learn

Pick the itertools function for a job, from islice and chain to groupby and batched
Group sorted records with groupby and process them in batches with batched
Generate combinations, permutations and products, and estimate their size first
Avoid the traps of groupby, tee and accumulate
Limit a search space by pruning partial results instead of filtering every result

Before you start

On this page

Cutting 120 message recipients into requests of 50, pairing five meter readings with their neighbours and listing the 15 pairs a team of six can form are three loops, and each one is a single call from itertools. The module holds the standard library’s iterator building blocks: every function in it returns Said of an iterator that makes each item only when a loop or a call asks for the next one, so nothing is computed ahead and an endless one costs nothing until it is read. iterator: An object that hands out items one at a time: next() returns its next item, or raises StopIteration when none are left. It keeps one position, so it is used up after one pass, and iter() returns the iterator itself. More in “How iteration works: iterables and iterators”, so its tools combine like generator stages and work on inputs of any size.

Below you will group attendance records, batch messages and count pairings before generating them, stepping through real runs of groupby() and pairwise().

Every itertools function returns a lazy iterator

count(1001) can number orders for ever, because it makes each number only when something asks for the next one, and islice() takes as many as you need. Every tool of the module works this way. They fall into four families: endless sources, tools that slice or join, tools that group, and combinatorics.

Function	What it gives
`count(start, step)`	Numbers for ever
`cycle(it)`	The items of `it` again and again
`repeat(x, n)`	The same value, `n` times or for ever
`islice(it, start, stop, step)`	A slice of any iterator
`chain(a, b)`, `chain.from_iterable(xs)`	One iterable after another
`takewhile(f, it)`, `dropwhile(f, it)`	The front while a test holds, or what follows it
`compress(it, flags)`	The items whose flag is true
`groupby(it, key)`	Each run of equal keys, with an iterator over it
`batched(it, n)`	Tuples of `n` items; the last may be shorter
`pairwise(it)`	Overlapping pairs of neighbours
`accumulate(it, f, initial=)`	Running totals, or running results of any two-argument function
`tee(it, n)`	`n` independent iterators over one input
`product`, `permutations`, `combinations`, `combinations_with_replacement`	Every tuple of a search space

Version note

pairwise() arrived in Python 3.10, batched() in 3.12 and its strict=True option, which raises ValueError for a short last batch, in 3.13.

The program below runs one call of each tool, in the order of the table. Press Edit and change any call:

One call of every tool, run Python · toolkit.py

from itertools import (accumulate, batched, chain, combinations, compress, count, cycle,
                       groupby, islice, pairwise, repeat, takewhile, tee)


def show(name, items):
    """Reads an iterator to the end and prints what it gave, tuples as joined letters."""
    words = ["".join(map(str, x)) if isinstance(x, tuple) else str(x) for x in items]
    print(f"{name:13}", " ".join(words))


show("count", islice(count(5), 3))          # endless: islice() takes the first 3
show("cycle", islice(cycle("ab"), 4))       # endless too
show("repeat", repeat(0, 3))
show("islice", islice("abcdef", 1, 4))
show("chain", chain("ab", "cd"))
show("takewhile", takewhile(lambda x: x < 3, [1, 2, 5, 1]))
show("compress", compress("abc", [1, 0, 1]))
show("groupby", (key for key, run in groupby("aabca")))
show("batched", batched("abcde", 2))
show("pairwise", pairwise("abc"))
show("accumulate", accumulate([1, 2, 3]))
first, second = tee("ab")
show("tee", chain(first, "|", second))
show("combinations", combinations("abc", 2))

Output

count         5 6 7
cycle         a b a b
repeat        0 0 0
islice        b c d
chain         a b c d
takewhile     1 2
compress      a c
groupby       a b c a
batched       ab cd e
pairwise      ab bc
accumulate    1 3 6
tee           a b | a b
combinations  ab ac bc

Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 toolkit.py

The two endless tools get a limit from islice(), and chain() read "ab" to its end before it started on "cd", without building a new list. zip_longest(), starmap() and filterfalse() complete the module.

Key idea

An itertools tool makes nothing until a loop or a call such as list() asks for an item, so an endless one needs islice() or a break.

Checkpoint

Check: Lazy iterators of itertools

Question 1 of 2 ids = islice(count(1001), 3) has just run. What has it made so far?
Choose one answer.

The list [1001, 1002, 1003] An error, because count() never ends Nothing yet: an iterator that will make 1001, 1002 and 1003 when asked Every number from 1001 upwards, until memory runs out

Show the answer: Question 1 of 2

Answer: Nothing yet: an iterator that will make 1001, 1002 and 1003 when asked

Nothing is computed when the line runs: ids is an iterator. list(ids) or a for loop would then make 1001, 1002 and 1003, and count() would never be asked for a fourth.
Question 2 of 2 lines is a generator over the lines of a large log. Which expression gives lines 10 to 19 without reading the rest of the file?
Choose one answer.

list(lines)[10:20] lines[10:20] islice(lines, 10, 20) takewhile(lambda line: line < 20, lines)

Show the answer: Question 2 of 2

Answer: islice(lines, 10, 20)

islice(it, start, stop) is the slice of an iterator: it skips start items, yields items up to stop and then stops asking, so the rest of the log is never read.

groupby() groups runs of equal keys, so sort first

Six attendance records arrive in the order they were marked, Monday and Tuesday in turn, and the report needs the names present on each day:

Attendance grouped by day Python · attendance.py

from itertools import groupby
from operator import itemgetter

records = [
    ("Mon", "Asha", "present"),
    ("Tue", "Ravi", "absent"),
    ("Mon", "Ravi", "present"),
    ("Tue", "Asha", "present"),
    ("Mon", "Meera", "absent"),
    ("Tue", "Meera", "present"),
]
day = itemgetter(0)

print("unsorted:", [(d, len(list(g))) for d, g in groupby(records, key=day)])

records.sort(key=day)                    # equal days next to each other
for d, group in groupby(records, key=day):
    present = [name for _, name, status in group if status == "present"]
    print(d, "present:", ", ".join(present))

Output

unsorted: [('Mon', 1), ('Tue', 1), ('Mon', 1), ('Tue', 1), ('Mon', 1), ('Tue', 1)]
Mon present: Asha, Ravi
Tue present: Asha, Meera

Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 attendance.py

groupby() groups runs of equal keys, not equal keys

Text description of the diagram

The diagram has two columns of the six attendance records, read from top to bottom.

Left, as recorded: Mon Asha, Tue Ravi, Mon Ravi, Tue Asha, Mon Meera, Tue Meera. The day changes at every record, so groupby() starts a new group each time and yields six groups of one record.
Right, sorted by day first: Mon Asha, Mon Ravi, Mon Meera, then Tue Ravi, Tue Asha, Tue Meera. Equal days are now next to each other, so groupby() yields two groups, Mon with three records and Tue with three. The sort keeps the records of each day in the order they arrived.

On the records as they arrived, the first line shows six groups of one, the left column of the diagram. groupby() works like the Unix uniq command: it starts a new run: A stretch of neighbouring items that share a key, such as three Mon records in a row. itertools.groupby() yields one group per run, so a key whose items are apart forms several runs. every time the key changes, so it groups neighbours, not equal keys. Step through the same six days, as they arrived and sorted. The flag under days is the position groupby() reads next:

Interactive · line by line groupby() on six days, as they arrived and sorted Step through both loops. The flag under days is the position groupby() reads next; watch how many days each group takes.

Program · day_runs.py

from itertools import groupby

days = ["Mon", "Tue", "Mon",
        "Tue", "Mon", "Tue"]
runs = []
for day, run in groupby(days):
    runs.append((day, len(list(run))))
print(runs)

days.sort()
runs = []
for day, run in groupby(days):
    runs.append((day, len(list(run))))
print(runs)

Variables

changed in this step
passed by the loop
read next

Printed so far

Nothing printed yet[('Mon', 1), ('Tue', 1), ('Mon', 1), ('Tue', 1), ('Mon', 1), ('Tue', 1)]
[('Mon', 3), ('Tue', 3)]

Every step was recorded from a run with Python 3.14.8 on macOS 26 arm64.

The drawing shows the variables when the program ends, and the program's 26 steps are listed below.

Line run 14

Next line end

Loop pass –

Every step as text

Start. No line has run yet, so there are no variables. Python starts with line 1.
Line 1. from itertools import groupby loads groupby.
Line 3. days = ["Mon", "Tue", "Mon", "Tue", "Mon", "Tue"] makes days a list of 6 items.
Line 5. runs = [] makes runs a list of 0 items.
Line 6. for day, run in groupby(days): starts: day is 'Mon' and run is <_grouper object>. groupby() has read 'Mon' at position 0 and hands it out as the key, with run, an iterator over that group.
Line 7. runs.append((day, len(list(run)))) adds ('Mon', 1) to the end of runs. Key point To find the end of the group, groupby() read 'Tue' at position 1. The key changed, so the Mon group holds one day, and 'Tue' starts the next group.
Line 6. for day, run in groupby(days): goes round again: day is 'Tue' and run is <_grouper object>.
Line 7. runs.append((day, len(list(run)))) adds ('Tue', 1) to the end of runs.
Line 6. for day, run in groupby(days): goes round again: day is 'Mon' and run is <_grouper object>. The bug Mon again, as a third group. groupby() compares each key only with the one before it, so the Mon days at positions 0 and 2 never meet.
Line 7. runs.append((day, len(list(run)))) adds ('Mon', 1) to the end of runs.
Line 6. for day, run in groupby(days): goes round again: day is 'Tue' and run is <_grouper object>.
Line 7. runs.append((day, len(list(run)))) adds ('Tue', 1) to the end of runs.
Line 6. for day, run in groupby(days): goes round again: day is 'Mon' and run is <_grouper object>.
Line 7. runs.append((day, len(list(run)))) adds ('Mon', 1) to the end of runs.
Line 6. for day, run in groupby(days): goes round again: day is 'Tue' and run is <_grouper object>.
Line 7. runs.append((day, len(list(run)))) adds ('Tue', 1) to the end of runs.
Line 6. for day, run in groupby(days): has no item left, so the loop ends.
Line 8. print(runs) prints [('Mon', 1), ('Tue', 1), ('Mon', 1), ('Tue', 1), ('Mon', 1), ('Tue', 1)].
Line 10. days.sort() puts the items of days in a new order: ['Mon', 'Mon', 'Mon', 'Tue', 'Tue', 'Tue']. The fix Sorting with the same key puts the three Mon days side by side, so they form one run.
Line 11. runs = [] sets runs to [] (it was [('Mon', 1), ('Tue', 1), ('Mon', 1), ('Tue', 1), ('Mon', 1), ('Tue', 1)]).
Line 12. for day, run in groupby(days): starts: day is 'Mon' and run is <_grouper object>.
Line 13. runs.append((day, len(list(run)))) adds ('Mon', 3) to the end of runs. This time the Mon group took positions 0 to 2. groupby() read 'Tue' at position 3 to see it end, so the group holds three days.
Line 12. for day, run in groupby(days): goes round again: day is 'Tue' and run is <_grouper object>.
Line 13. runs.append((day, len(list(run)))) adds ('Tue', 3) to the end of runs.
Line 12. for day, run in groupby(days): has no item left, so the loop ends.
Line 14. print(runs) prints [('Mon', 3), ('Tue', 3)]. That was the last line, so the program ends. Six groups as the days arrived, two once they were sorted. groupby() groups neighbours with equal keys, never equal keys that are apart.

To see where a group ends, groupby() reads one item past it, so a key that differs from the one before starts a new group. Sorting by the same key put each day’s records side by side, in the order they came. A sort costs O(n log n) and needs a list; data that already arrives in key order, such as a log sorted by time, needs neither.

Grouping records that are not sorted by the key

On records in arrival order, groupby(records, key=day) gave six groups for two days. Sort with the same key first, records.sort(key=day).

Checkpoint

Check: groupby() and runs of keys

Question 1 of 2 groupby() reads the days Mon, Mon, Tue, Mon. How many groups does it make?
Type a number.
Your answer groups

A hint

A new group starts every time the day differs from the one before it.

Show the answer: Question 1 of 2

Answer: 3 groups

Mon, Mon form one run, Tue a second and the last Mon a third. groupby() never joins that Mon to the first group, because Tue stood between them.
Question 2 of 2 records holds tuples (day, name, status), and you group them with groupby(records, key=status), where status = itemgetter(2). Which line before the loop makes each status one group?
Choose one answer.

No line: groupby() collects equal keys by itself records.sort(key=status) sorted(records, key=status) records.sort()

Show the answer: Question 2 of 2

Answer: records.sort(key=status)

groupby() groups runs, so sort by exactly the key it groups by: records.sort(key=status), or loop over sorted(records, key=status) instead of records.

batched() cuts fixed-size groups and pairwise() pairs neighbours

An API that accepts at most 50 recipients per request needs a list of 120 cut into pieces, and a meter read once a day needs each reading paired with the next:

Requests of 50 messages, and daily meter use Python · batches_and_pairs.py

from itertools import batched, pairwise

recipients = [f"user{n:03d}" for n in range(120)]
sizes = [len(batch) for batch in batched(recipients, 50)]
print("messages in each request:", sizes)

meter = [1520, 1528, 1541, 1541, 1560]        # electricity meter, one reading a day
used = [b - a for a, b in pairwise(meter)]
print("units used each day:", used)

Output

messages in each request: [50, 50, 20]
units used each day: [8, 13, 0, 19]

Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 batches_and_pairs.py

batched() cut the 120 recipients lazily into requests of 50, 50 and 20: each batch is a tuple of up to n items, and the last holds whatever is left. pairwise() yields each item with the one after it, so a day’s use is one subtraction per pair. Step through the meter loop and watch a, b and the flag under meter:

Interactive · line by line pairwise() over five meter readings Step through the loop. The flag under meter is the reading pairwise() reads next; a and b are the pair it gave.

Program · meter_pairs.py

from itertools import pairwise

meter = [1520, 1528, 1541, 1541, 1560]
used = []
for a, b in pairwise(meter):
    used.append(b - a)
print(used)

Variables

changed in this step
passed by the loop
read next

Printed so far

Nothing printed yet[8, 13, 0, 19]

Every step was recorded from a run with Python 3.14.8 on macOS 26 arm64.

The drawing shows the variables when the program ends, and the program's 14 steps are listed below.

Line run 7

Next line end

Loop pass –

Every step as text

Start. No line has run yet, so there are no variables. Python starts with line 1.
Line 1. from itertools import pairwise loads pairwise.
Line 3. meter = [1520, 1528, 1541, 1541, 1560] makes meter a list of 5 items.
Line 4. used = [] makes used a list of 0 items.
Line 5. for a, b in pairwise(meter): starts: a is 1520 and b is 1528. pairwise() read positions 0 and 1 and gives them as one pair. The flag shows it reads position 2 next.
Line 6. used.append(b - a) adds 8 to the end of used.
Line 5. for a, b in pairwise(meter): goes round again: a is 1528 and b is 1541. Key point The new pair starts with 1528, the end of the last one. pairwise() reads each reading once and keeps only the last, to pair it with the next.
Line 6. used.append(b - a) adds 13 to the end of used.
Line 5. for a, b in pairwise(meter): goes round again: a is 1541 and b is 1541.
Line 6. used.append(b - a) adds 0 to the end of used. Two equal readings are a pair too, so the day's use is 0.
Line 5. for a, b in pairwise(meter): goes round again: a is 1541 and b is 1560.
Line 6. used.append(b - a) adds 19 to the end of used.
Line 5. for a, b in pairwise(meter): has no item left, so the loop ends.
Line 7. print(used) prints [8, 13, 0, 19]. That was the last line, so the program ends. Five readings gave four pairs, so used holds four days of use.

Five readings gave four pairs, and pairwise() read each reading once. Before Python 3.10 the usual way to pair neighbours was zip(xs, xs[1:]), which needs a list; pairwise() works on any iterator, the lines of a file included.

Checkpoint

Check: batched() and pairwise()

Question 1 of 2 How many recipients are in the last request of batched(range(130), 50)?
Type a number.
Your answer recipients

Show the answer: Question 1 of 2

Answer: 30 recipients

130 recipients make two full batches of 50 and a third batch of the 30 that are left. batched() yields the shorter last batch without complaint unless strict=True is given.
Question 2 of 2 A meter is read once a day for 30 days. How many pairs does pairwise() give for the 30 readings?
Type a number.
Your answer pairs

A hint

Each pair is one reading and the next one. Which reading has no next one?

Show the answer: Question 2 of 2

Answer: 29 pairs

Every reading but the last starts a pair, so 30 readings give 29 pairs, one day's use for each day after the first, as five readings gave four in the program.

combinations() ignores order and permutations() keeps it: count before you loop

Six people can form 15 pairs for pair programming and fill the roles of captain and vice-captain in 30 ways:

Pairings and roles in a team of six Python · team_pairings.py

import math
from itertools import combinations, permutations

team = ["Asha", "Ravi", "Meera", "Arjun", "Zoya", "Dev"]

pairs = list(combinations(team, 2))       # order does not matter
print(len(pairs), math.comb(6, 2), pairs[:3])

roles = list(permutations(team, 2))       # (captain, vice-captain): order matters
print(len(roles), math.perm(6, 2), roles[:3])

print("all orders of the whole team:", math.factorial(6))

Output

15 15 [('Asha', 'Ravi'), ('Asha', 'Meera'), ('Asha', 'Arjun')]
30 30 [('Asha', 'Ravi'), ('Asha', 'Meera'), ('Asha', 'Arjun')]
all orders of the whole team: 720

Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 team_pairings.py

combinations(team, 2) yields each pair once, in the order of the input, and ignores order inside the pair: each tuple is a combination: A choice of items in which order does not matter: (Asha, Ravi) and (Ravi, Asha) are one combination. itertools.combinations() generates them, and math.comb(n, k) counts them without generating any.. permutations(team, 2) treats (“Asha”, “Ravi”) and (“Ravi”, “Asha”) as different permutation: An arrangement of items in which order matters: (Asha, Ravi) and (Ravi, Asha) are two permutations. itertools.permutations() generates them, and math.perm(n, k) counts them without generating any., which fits roles. math.comb() and math.perm() give the same counts without generating anything, and that is the habit to build: count a search space: Every candidate a brute-force search would try, such as the 720 orders of six people. Counting it before the loop, for example with math.comb() or math.perm(), shows whether the loop can finish. before you loop over it. The last line is a warning: six people have 720 orders, a factorial: n!, the product 1 × 2 × … × n: the number of orders of n different items. 4! is 24 and 6! is 720, and math.factorial(n) computes it., so every person added multiplies the count again. Pruning, which drops a partial order the moment it breaks a rule, can skip most of them. The two generators side by side:

combinations() or permutations()?
Criterion	combinations(team, 2)	permutations(team, 2)
Order inside a tuple	ignored: each pair once	kept: both orders
Tuples for 6 people	15	30
Counted first with	math.comb(6, 2)	math.perm(6, 2)
When to choose	Pairs, teams, committees	Roles, rankings, routes

Count with your real sizes

A test with six people runs in an instant even when the real input would take hours, so print math.comb() or math.perm() of the real sizes before you write the loop.

Permutation & Combination Calculator Check counts of combinations and permutations, with or without repetition, before you generate them. List Combiner & Keyword Mixer Combine two lists line by line, like zip(), or in every combination, like product(), and compare with your Python results.

Checkpoint

Check: Combinations, permutations and counts

One question How many tuples does combinations(team, 2) give for a team of five people?
Type a number.
Your answer pairs

A hint

Count the ordered pairs, 5 × 4, then remember that combinations ignore order.

Show the answer: One question

Answer: 10 pairs

math.comb(5, 2) is 5 × 4 ÷ 2 = 10. permutations(team, 2) would give 20, because it counts each pair in both orders.

Check yourself

Each question goes to the idea you have answered least well so far, and its answer is explained as soon as you choose.

Practice that adapts to you

Practise this lesson

13 questions about 12 ideas of this lesson. Each answer is explained at once, and what you get right is kept in this browser.

Question 1 of 13 Which of these never stop on their own, so that a loop over them needs a break or islice()? Choose every right answer.
Choose every answer that is right.

repeat(0, 3) pairwise(range(5)) repeat(0) cycle("ab") count(1)
Show the answer: Question 1 of 13
Answer:
- repeat(0)
- cycle("ab")
- count(1)
count() counts for ever, cycle() repeats its input for ever, and repeat() without a count repeats its value for ever. repeat(0, 3) stops after three zeros, and pairwise() ends with its input.
Question 2 of 13 What does this print?
Read the code, then choose one answer.
```
from itertools import chain
print(list(chain("ab", [1, 2])))
```
['a', 'b', 1, 2] ['a', 1, 'b', 2] ['ab', [1, 2]] TypeError, because a string and a list cannot be joined
Show the answer: Question 2 of 13

Answer: ['a', 'b', 1, 2]

chain(a, b) is one iterable after another: 'a' and 'b' from the string, then 1 and 2 from the list, each made only when it is asked for.
Question 3 of 13 Which of these finish when they are given the endless count(1)? Choose every right answer.
Choose every answer that is right.

sorted(count(1)) list(takewhile(lambda n: n < 10, count(1))) list(islice(count(1), 5)) max(count(1))
Show the answer: Question 3 of 13
Answer:
- list(takewhile(lambda n: n < 10, count(1)))
- list(islice(count(1), 5))
A tool that stops asking, such as islice() or takewhile(), makes an endless source safe. A call that must see every item, such as sorted(), max() or list() on its own, never returns.

Question 4 of 13 What does attendance.py print?

What does this program print? Choose one answer.

from itertools import groupby
from operator import itemgetter

records = [
    ("Mon", "Asha", "present"),
    ("Tue", "Ravi", "absent"),
    ("Mon", "Ravi", "present"),
    ("Tue", "Asha", "present"),
    ("Mon", "Meera", "absent"),
    ("Tue", "Meera", "present"),
]
day = itemgetter(0)

print("unsorted:", [(d, len(list(g))) for d, g in groupby(records, key=day)])

records.sort(key=day)                    # equal days next to each other
for d, group in groupby(records, key=day):
    present = [name for _, name, status in group if status == "present"]
    print(d, "present:", ", ".join(present))

unsorted: [('Mon', 3), ('Tue', 3)] Mon present: Asha, Ravi Tue present: Asha, Meera unsorted: [('Mon', 1), ('Tue', 1), ('Mon', 1), ('Tue', 1), ('Mon', 1), ('Tue', 1)] Mon present: Asha, Ravi, Meera Tue present: Ravi, Asha, Meera unsorted: [('Mon', 1), ('Tue', 1), ('Mon', 1), ('Tue', 1), ('Mon', 1), ('Tue', 1)] Mon present: Asha, Ravi Tue present: Asha, Meera

Show the answer

Answer: it prints

unsorted: [('Mon', 1), ('Tue', 1), ('Mon', 1), ('Tue', 1), ('Mon', 1), ('Tue', 1)]
Mon present: Asha, Ravi
Tue present: Asha, Meera

Unsorted, the day changes at every record, so groupby() starts six groups of one. After sorting by day, each day is one run. Sorting is stable, so the records of a day keep their original order, and the list comprehension keeps only the names marked present.

Question 5 of 13 How many groups does groupby("aabca") make?
Type a number.
Your answer groups

Show the answer: Question 5 of 13

Answer: 4 groups

The runs are aa, b, c and a. The last a is a group of its own, because b and c stand between it and the first run, so groupby() yields the keys a, b, c and a, as toolkit.py printed.
Question 6 of 13 A web server's log is written in time order, and you count its errors per hour with groupby(lines, key=hour). Do you sort the lines first?
Choose one answer.

No. The hours already arrive in order, so each hour is one run, and a sort would only cost time and memory No, because groupby() sorts its input itself Yes, because groupby() always needs sorted input Yes, by the text of each line, so that equal lines are together

Show the answer: Question 6 of 13

Answer: No. The hours already arrive in order, so each hour is one run, and a sort would only cost time and memory

Sort only when equal keys may be apart. Time-ordered data grouped by hour streams through groupby() with no sort and no memory beyond the current hour.
Question 7 of 13 What does this print?
Read the code, then choose one answer.
```
from itertools import batched
print([len(b) for b in batched(range(7), 3)])
```
[3, 3] [3, 3, 1] [3, 3, 3] ValueError: incomplete batch
Show the answer: Question 7 of 13

Answer: [3, 3, 1]

batched() yields tuples of up to 3 items, and the last one holds whatever is left, here one item. Only batched(..., strict=True) raises ValueError for a short last batch.
Question 8 of 13 What does this print?
Read the code, then choose one answer.
```
from itertools import pairwise
print(list(pairwise([4, 7, 9])))
```
[(4, 7), (7, 9)] [(4, 9)] [(4, 7, 9)] [(4, 7), (9,)]
Show the answer: Question 8 of 13

Answer: [(4, 7), (7, 9)]

pairwise() yields every item with the one after it, so n items give n − 1 overlapping pairs.
Question 9 of 13 times holds sorted login times in seconds. Which expression gives the longest gap between two logins in a row?
Choose one answer.

max(times[1:]) max(b - a for a, b in batched(times, 2)) max(b - a for a, b in pairwise(times)) max(times) - min(times)

Show the answer: Question 9 of 13

Answer: max(b - a for a, b in pairwise(times))

Gaps lie between neighbours, which is exactly what pairwise() pairs. Batches of two measure only every other gap.
Question 10 of 13 How many tuples does itertools.combinations(range(10), 3) produce?
Type a number.
Your answer tuples

Show the answer: Question 10 of 13

Answer: 120 tuples

Combinations ignore order, so the count is 10 choose 3: 10 × 9 × 8 ÷ (3 × 2 × 1) = 120, which math.comb(10, 3) also returns. permutations(range(10), 3) would give 720, because each choice of three appears in 6 orders.
Question 11 of 13 A meal plan picks 3 different dishes out of 8, and the order of the dishes does not matter. Which call counts the choices?
Choose one answer.

math.perm(8, 3) math.factorial(8) 8 ** 3 math.comb(8, 3)

Show the answer: Question 11 of 13

Answer: math.comb(8, 3)

When order does not matter, count with math.comb(n, k), and generate with combinations(); when order matters, math.perm(n, k) and permutations().
Question 12 of 13 A captain, a vice-captain and a wicket-keeper, three different players, are chosen from a squad of 11. In how many ways?
Type a number.
Your answer ways

A hint

The roles differ, so order matters.

Show the answer: Question 12 of 13

Answer: 990 ways

The roles are different, so it is a permutation: math.perm(11, 3) = 11 × 10 × 9 = 990, the length of permutations(squad, 3).
Question 13 of 13 permutations(team) lists every order of the whole team. How many times more orders does a team of 8 have than a team of 6?
Type a number.
Your answer times

A hint

Compare 8! with 6!.

Show the answer: Question 13 of 13

Answer: 56 times

8! ÷ 6! = 8 × 7 = 56. Each person added multiplies the orders by the new team size, which is why a factorial count outgrows any test data quickly.

groupby, tee and accumulate in depth, with the traps reproduced Pro: opens with a Pro, Premium or Ultimate pass

Three tools of the module keep state between items, and their state explains their traps. This part reproduces the groups that vanish, measures the memory that tee() holds, and turns accumulate() into a tool for balances and drawdowns.

In this part:

groupby’s groups share one iterator: groups that vanish, reproduced
tee buffers what one branch has read and the other has not, measured
accumulate beyond running totals: balances, lows and the largest fall
islice, takewhile, dropwhile and compress, compared
Ragged data: flattening one level and filling the gaps

5 runnable examples (Python) · about 9 minutes

Opening this part with your pass…

This part opens with any MySmartCoPilot pass.

Pro pass · 1 week₹79$2.99
30 days₹179$5.99
1 year₹1,699$45

One payment; a pass never renews. The price shown is the total you pay: no fees or charges are added at checkout.

Choose a pass I have a licence key

It also opens every other in-depth part in Learn and every Pro tool. Sales are final, except where the law gives you a right to withdraw, for example in the EU and the UK (refund policy).

Search spaces with numbers: product, permutations and pruning Pro: opens with a Pro, Premium or Ultimate pass

Combinatoric iterators are lazy, but laziness does not shrink the space they walk through. This part puts numbers on how fast each one grows, shows pruning cutting 3,628,800 orders to about 111,000 steps, builds a test matrix with product() and ends with an exercise.

In this part:

How fast each generator grows, with the hours worked out
The four counting rules side by side, with their formulas
Seat plans: 3,628,800 orders checked against 111,461 partial rows with pruning
product for test matrices, and cutting the matrix down
Exercise: attendance streaks and seat plans
- Attendance streaks and seat plans with itertools

1 diagram · 3 runnable examples (Python) · 1 exercise with sample tests · about 9 minutes

Opening this part with your pass…

This part opens with any MySmartCoPilot pass.

Pro pass · 1 week₹79$2.99
30 days₹179$5.99
1 year₹1,699$45

One payment; a pass never renews. The price shown is the total you pay: no fees or charges are added at checkout.

Choose a pass I have a licence key

It also opens every other in-depth part in Learn and every Pro tool. Sales are final, except where the law gives you a right to withdraw, for example in the EU and the UK (refund policy).

Interview questions

Fresher to mid · Warm-up

Why do you usually sort the data before calling itertools.groupby()?

groupby() does not collect equal keys from the whole input. It walks through the iterable once and starts a new group every time the key changes, like the Unix uniq command, so it groups runs of neighbouring items with the same key. On unsorted data the same key appears in several separate groups: records for Mon, Tue and Mon give three groups, not two. Sorting by the same key function first puts equal keys next to each other, so each key forms one group. Sorting costs O(n log n) and needs the data in a list, so when the data already arrives in key order, such as a log sorted by time, skip the sort. When it cannot be sorted, group with a dictionary of lists or a Counter instead.
Mid to senior

A service splits one stream of events with itertools.tee() into two branches: one counts events per minute, the other writes them to an archive in batches whenever a timer fires. Memory climbs for hours until the process is killed. Explain why, and restructure the code.

Model answer, Pro. Model answers open with any pass; prices are in the notice on this page. First step and follow-ups
Fresher to senior

A checkout tries every combination of the 30 coupons in a customer's wallet, using itertools.combinations for every size, to find the cheapest valid set. It passed its tests with 8 coupons and times out in production. Estimate the work, and redesign it.

Model answer, Pro. First step and follow-ups
Mid to senior

You must send 1.2 million notifications through an API that accepts at most 500 recipients per request and 20 requests a second. Write the batching loop with itertools, estimate the run time, and say what happens to the last batch and to a request that fails halfway.

Model answer, Pro. First step and follow-ups

Key takeaways

Every itertools function returns a lazy iterator, so the tools combine like generator stages and work on endless or huge inputs.
groupby() groups runs of equal keys, so sort by the same key first unless the data already arrives in that order.
batched() cuts tuples of up to n items and pairwise() yields overlapping neighbours, and both read their input once, lazily.
combinations() ignores order and permutations() keeps it; count a search space with math.comb() or math.perm() before you generate it.

Terms in this lesson

combination: A choice of items in which order does not matter: (Asha, Ravi) and (Ravi, Asha) are one combination. itertools.combinations() generates them, and math.comb(n, k) counts them without generating any.
factorial: n!, the product 1 × 2 × … × n: the number of orders of n different items. 4! is 24 and 6! is 720, and math.factorial(n) computes it.
permutation: An arrangement of items in which order matters: (Asha, Ravi) and (Ravi, Asha) are two permutations. itertools.permutations() generates them, and math.perm(n, k) counts them without generating any.
run: A stretch of neighbouring items that share a key, such as three Mon records in a row. itertools.groupby() yields one group per run, so a key whose items are apart forms several runs.
search space: Every candidate a brute-force search would try, such as the 720 orders of six people. Counting it before the loop, for example with math.comb() or math.perm(), shows whether the loop can finish.

References

itertools: Functions creating iterators for efficient looping (Python Software Foundation)
itertools.groupby() (Python Software Foundation)
itertools.batched() (Python Software Foundation)
itertools.pairwise() (Python Software Foundation)
itertools.tee() (Python Software Foundation)
itertools.accumulate() (Python Software Foundation)
math.comb() and math.perm() (Python Software Foundation)
Sorting Techniques: sort stability (Python Software Foundation)
Functional Programming HOWTO: The itertools module (Python Software Foundation)

Report a problem with this lesson

In this module: Iterators, generators and functional tools

How iteration works: iterables and iterators Free lesson, Pro answers: 20 minutes
Generators and generator expressions Free core, Pro depth: 9 of 25 minutes free
Generator pipelines, yield from and send Free core, Pro depth: 9 of 25 minutes free
itertools: the iteration toolkit Free core, Pro depth: 8 of 20 minutes free
Built-in functions every Python coder should know Free core, Pro depth: 8 of 20 minutes free

Your country

itertools: the iteration toolkit

What you will learn

Before you start

What you will learn

Before you start

Every itertools function returns a lazy iterator

groupby() groups runs of equal keys, so sort first

batched() cuts fixed-size groups and pairwise() pairs neighbours

combinations() ignores order and permutations() keeps it: count before you loop

Check yourself

groupby, tee and accumulate in depth, with the traps reproduced Pro: opens with a Pro, Premium or Ultimate pass

Search spaces with numbers: product, permutations and pruning Pro: opens with a Pro, Premium or Ultimate pass

Interview questions

Why do you usually sort the data before calling itertools.groupby()?

A service splits one stream of events with itertools.tee() into two branches: one counts events per minute, the other writes them to an archive in batches whenever a timer fires. Memory climbs for hours until the process is killed. Explain why, and restructure the code.

A checkout tries every combination of the 30 coupons in a customer's wallet, using itertools.combinations for every size, to find the cheapest valid set. It passed its tests with 8 coupons and times out in production. Estimate the work, and redesign it.

You must send 1.2 million notifications through an API that accepts at most 500 recipients per request and 20 requests a second. Write the batching loop with itertools, estimate the run time, and say what happens to the last batch and to a request that fails halfway.

Key takeaways

References

Related tools