Python Module 15 – Async programming with asyncio

Queues, semaphores and rate limits in asyncio

Cap concurrent requests with asyncio.Semaphore, connect producers and workers with asyncio.Queue, shut queues down cleanly and limit requests per second.

Advanced
20 minutes
Examples run with Python 3.14.8 and Pyodide 314.0.7
By MySmartCoPilot

Free: A semaphore lets n tasks in at a time, and the rest wait cheaply · A bounded queue makes a fast producer wait for its workers · Semaphore, queue or token bucket? · 2 more sections. With a pass: 2 in-depth parts and 3 interview answers, about 13 minutes.

What you will learn

Limit concurrency with asyncio.Semaphore
Build producer-consumer pipelines with asyncio.Queue and shut them down cleanly
Apply back-pressure with a bounded queue
Implement a token bucket that limits requests per second
Distinguish a concurrency limit from a rate limit

Before you start

Timeouts, cancellation and clean-up in asyncio

What you’ll learn · Before you start

What you will learn

Limit concurrency with asyncio.Semaphore
Build producer-consumer pipelines with asyncio.Queue and shut them down cleanly
Apply back-pressure with a bounded queue
Implement a token bucket that limits requests per second
Distinguish a concurrency limit from a rate limit

Before you start

Timeouts, cancellation and clean-up in asyncio

On this page

Twelve downloads started at once can swamp a server that accepts three at a time, and a producer that finds work faster than its workers finish it fills memory. asyncio gives you three ways to keep concurrent work within bounds. An asyncio.Semaphore(n) lets at most n tasks into a block at the same time, which is the standard way to cap concurrent requests to one service. An asyncio.Queue connects producers and consumers: producers put() work items, a fixed number of worker tasks get() them, and a queue with a maxsize makes a fast producer wait, which is called back-pressure: Slowing a producer down when its consumers fall behind, instead of letting work pile up in memory. A bounded asyncio.Queue does it: put() waits while the queue already holds maxsize items..

A token bucket: A rate limit in which each request spends a token and tokens come back at a fixed rate, so requests start no faster than that rate however many tasks are asking. limits how many requests start per second, however many tasks there are: each request spends a token, and tokens come back at a fixed rate. Semaphores bound how much runs at once, queues bound how much work waits, and rate limits bound how fast work starts. All of them coordinate the tasks of one event loop and are not thread-safe. Since Python 3.13, Queue.shutdown() tells waiting workers that no more work is coming.

A semaphore lets n tasks in at a time, and the rest wait cheaply

The program counts the downloads in flight and times the batch:

Twelve downloads, three at a time Python · semaphore_cap.py

"""asyncio.Semaphore(3) lets three downloads run at a time; the other nine wait their turn."""
import asyncio
import time

in_flight = 0
most = 0


async def download(gate, n):
    global in_flight, most
    async with gate:  # waits here while three downloads are running
        in_flight += 1
        most = max(most, in_flight)
        await asyncio.sleep(0.05)  # stands in for the transfer
        in_flight -= 1
    return f"file-{n:02}"


async def main():
    gate = asyncio.Semaphore(3)
    start = time.perf_counter()
    files = await asyncio.gather(*(download(gate, n) for n in range(1, 13)))
    took = time.perf_counter() - start
    print(f"{len(files)} files, at most {most} downloads at once, results in order: {files[:3]} ...")
    print(f"took about {took:.1f} s: 12 files in waves of 3, 0.05 s each, is 0.2 s")


asyncio.run(main())

Output

12 files, at most 3 downloads at once, results in order: ['file-01', 'file-02', 'file-03'] ...
took about 0.2 s: 12 files in waves of 3, 0.05 s each, is 0.2 s

This output changes from run to run: the time is measured on each run, and the browser's event loop is slower

Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 semaphore_cap.py

Step through the run on a clock:

Interactive · step by step Twelve downloads through three permits Step through semaphore_cap.py on a clock. The semaphore hands out three permits; a download needs one to start.

The drawing shows the whole scene; its 6 steps are listed below.

Every step as text

Twelve tasks start. gather() starts all twelve downloads at once. Each one reaches async with gate before it may download.
Three get in. The semaphore has three permits, so file-01, file-02 and file-03 take one each and start. The other nine wait at async with gate, which costs no thread and blocks nothing.
0.05 s, the first wave ends. The three transfers finish after 0.05 s, and leaving the block gives each permit back, also for a download that raised.
The second wave. In this run the next three waiting downloads, file-04 to file-06, take the permits at once. At no moment do more than three run.
Waves three and four. The second wave ends at 0.10 s. The third runs from 0.10 to 0.15 s and the fourth from 0.15 to 0.20 s.
0.2 s, all twelve. Twelve files in waves of three, 0.05 s each, take 4 × 0.05 = 0.2 s. gather() returns the names in the order the downloads were given, and the most that ran at once was 3.

async with gate: took one of the semaphore’s three places, and the block gave it back at its end, also when the download raised. The other downloads waited at async with gate: without blocking anything, so the twelve ran in four waves of three. Waiting is cheap, so it is fine to start all twelve tasks at once and let the semaphore pace them. Semaphores fit any shared limit: connections to one database, calls to one partner API, files open at once. Take the number from the other side’s limit, such as the connections a database allows your service or the parallel requests a partner documents, not from a guess.

Key idea

A semaphore caps how many tasks are inside its block at once, and the others wait at async with without a thread, so starting every task at once is fine.

Checkpoint

Check: A semaphore caps calls at once

Question 1 of 2 Twenty downloads of 0.1 s each run through asyncio.Semaphore(4). At best, how many seconds does the batch take?
Type a number.
Your answer seconds

A hint

Count the waves.

Show the answer: Question 1 of 2

Answer: 0.5 seconds (anything from 0.45 to 0.55 counts)

Four permits make waves of four: 20 ÷ 4 = 5 waves of 0.1 s, so about 0.5 s, the same arithmetic as 12 files in waves of 3, 0.05 s each, giving 0.2 s.
Question 2 of 2 A download inside async with gate: raises an exception. What happens to its permit?
Choose one answer.

It is lost, and the semaphore has one permit fewer from then on It goes back only if the task catches the exception The semaphore refills itself once the batch is over The block gives it back, as async with does on every way out

Show the answer: Question 2 of 2

Answer: The block gives it back, as async with does on every way out

async with gate: took one of the three places and gave it back at the end of the block, also when the download raised, so a failure never shrinks the limit.

A bounded queue makes a fast producer wait for its workers

Queues decouple the code that finds work from the code that does it. Here a producer puts eight orders into a queue that holds at most three, and two workers take them out:

A producer, a bounded queue and two workers

Text description of the diagram

The diagram runs from top to bottom.

The producer puts orders into the queue with await queue.put(order).
The queue holds at most 3 orders. While it is full, put() waits, so the producer can never get far ahead of the workers.
Two workers each take the next order with await queue.get() and handle it.
When the producer has no more orders, it calls queue.shutdown(). The workers finish the orders still in the queue, and once it is empty, get() raises QueueShutDown and each worker stops.

Eight orders, a queue of three, two workers Python · queue_workers.py

"""A producer and two workers joined by a bounded asyncio.Queue; shutdown() (Python 3.13+) tells the workers that
no more work is coming."""
import asyncio

log = []


async def producer(queue):
    for n in range(1, 9):
        await queue.put(f"order-{n}")  # waits while the queue already holds 3 orders
        log.append(f"put order-{n} (queue holds {queue.qsize()})")
    queue.shutdown()  # no more orders: workers finish what is queued, then stop
    log.append("producer: shut the queue down")


async def worker(name, queue, done):
    while True:
        try:
            order = await queue.get()
        except asyncio.QueueShutDown:
            log.append(f"{name}: queue shut down and empty, stopping")
            return
        await asyncio.sleep(0.01)  # stands in for packing the order
        done.append((name, order))
        queue.task_done()


async def main():
    queue = asyncio.Queue(maxsize=3)
    done = []
    async with asyncio.TaskGroup() as tg:
        tg.create_task(producer(queue))
        tg.create_task(worker("worker-1", queue, done))
        tg.create_task(worker("worker-2", queue, done))
    print("\n".join(log))
    print("orders packed:", len(done), "| by worker-1:", sum(1 for w, _ in done if w == "worker-1"))


asyncio.run(main())

Output

put order-1 (queue holds 1)
put order-2 (queue holds 2)
put order-3 (queue holds 3)
put order-4 (queue holds 2)
put order-5 (queue holds 3)
put order-6 (queue holds 2)
put order-7 (queue holds 3)
put order-8 (queue holds 2)
producer: shut the queue down
worker-1: queue shut down and empty, stopping
worker-2: queue shut down and empty, stopping
orders packed: 8 | by worker-1: 4

Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 queue_workers.py

Whenever the queue was full, as at the fourth order, await queue.put() waited until a worker had taken one, so the queue never held more than three orders. The two workers took orders with await queue.get() and called task_done() after each. When the producer had no more orders it called queue.shutdown(). The workers still packed everything in the queue, and once it was empty, get() raised asyncio.QueueShutDown and each worker returned. The task group then saw all three tasks finish. Step through the same run on a clock:

Interactive · step by step Eight orders through a queue of three Step through queue_workers.py on a clock. Watch the producer wait whenever the queue holds three orders.

The drawing shows the whole scene; its 9 steps are listed below.

Every step as text

Three orders fit. The producer runs first. put() returns at once for order-1, order-2 and order-3, and the queue is full, at its maxsize of 3.
The fourth put() waits. put() of order-4 finds the queue full and waits. The producer can never get more than three orders ahead of the workers.
worker-1 takes order-1. worker-1 takes order-1 with get(), which also wakes the waiting producer, and starts packing it, which takes 0.01 s.
worker-2 takes order-2. worker-2 takes order-2 in the same way, and the queue holds only order-3.
The producer catches up. The producer puts order-4 and order-5, so the log says the queue holds 2 and then 3. Its put() of order-6 waits again.
The same rhythm. At 0.01 s the workers finish order-1 and order-2, call task_done() and take order-3 and order-4, and the producer puts order-6 and order-7. At 0.02 s they take order-5 and order-6, and the producer puts order-8.
No more orders. With all eight orders put, the producer calls queue.shutdown(). order-7 and order-8 are still in the queue, and shutdown() leaves them there.
The last two are packed. At 0.03 s the workers take order-7 and order-8 and pack them. Shutting the queue down did not throw them away.
get() raises QueueShutDown. At 0.04 s each worker's next get() finds the queue empty and shut down and raises QueueShutDown, so both workers return. worker-1 packed four of the eight orders, and the task group sees all three tasks finish.

Version note

Queue.shutdown() and QueueShutDown are new in Python 3.13. Before that, the same stop needed a sentinel value, such as one None per worker, or cancelling the workers.

Checkpoint

Check: A bounded queue between stages

Question 1 of 2 asyncio.Queue(maxsize=3) already holds 3 orders. What does await queue.put(order) do?
Choose one answer.

It drops the oldest order to make room It raises QueueFull It adds the order anyway, past maxsize It waits until a worker has taken an order out

Show the answer: Question 1 of 2

Answer: It waits until a worker has taken an order out

Whenever the queue was full, as at the fourth order, await queue.put() waited until a worker had taken one, so the queue never held more than three orders.
Question 2 of 2 The producer calls queue.shutdown() while order-7 and order-8 are still in the queue. What happens to them?
Choose one answer.

They are thrown away, and get() raises at once get() raises QueueShutDown at once, and they stay in the queue for ever Their put() calls fail with QueueShutDown The workers still take and pack them; get() raises QueueShutDown only once the queue is empty

Show the answer: Question 2 of 2

Answer: The workers still take and pack them; get() raises QueueShutDown only once the queue is empty

queue.shutdown() tells the workers that no more work is coming. They packed all 8 orders, and only when the queue was empty did get() raise asyncio.QueueShutDown and end each worker.

Semaphore, queue or token bucket?

Each of the three bounds something else:

Which limit for which job?
Criterion	Semaphore	Queue with workers	Token bucket
What it limits	how many run at the same moment	how many items wait, and how many workers run	how many start per second
Typical use	calls to one service, open connections	pipelines: crawl, parse, store	an API that allows N requests a minute
Fast service, many tasks	many requests a second, a few at a time	as fast as the workers go	never more than the rate
Built in	yes	yes	no: a few lines of your own
When to choose	Protect a service from too many parallel calls	Feed work through stages at a steady size	Keep to a requests-per-second rule

asyncio has no rate limiter of its own, so rate limits in asyncio are a few lines of your own, such as a token bucket, or part of the client library you use. A rate limit and a concurrency limit answer different rules. A service that answers in 30 milliseconds lets even one request at a time reach about 33 requests a second, so a semaphore alone cannot keep a job under “100 requests a minute”. When a server says you went too fast, it answers with the HTTP status 429 Too Many Requests, often with a Retry-After header that says how long to wait. Rate limits in asyncio code should respect that header rather than retry at once.

HTTP Status Codes Reference Read what 429 Too Many Requests and 503 Service Unavailable mean, the two answers a rate-limited client must handle.

Checkpoint

Check: Choosing the limit for the rule

Question 1 of 2 One request at a time is sent to a service that answers in 20 milliseconds. About how many requests a second is that?
Type a number.
Your answer requests a second

A hint

How many answers of 20 ms fit in one second?

Show the answer: Question 1 of 2

Answer: 50 requests a second (anything from 49 to 51 counts)

1 ÷ 0.02 = 50 requests a second, with only one in flight. That is why a concurrency limit, even of 1, cannot keep a client under a rule such as 100 requests a minute.
Question 2 of 2 What does a token bucket bound?
Choose one answer.

How many requests start per second How many items wait in memory How long each request may take How many requests run at the same moment

Show the answer: Question 2 of 2

Answer: How many requests start per second

Semaphores bound how much runs at once, queues bound how much work waits, and rate limits such as a token bucket bound how fast work starts.

A lock, an event and four more tools cover the rest

asyncio has a few more tools for coordinating tasks, each made for one job:

Tool	What it does
`asyncio.Lock`	Lets one task at a time into a block
`asyncio.Event`	Wakes every task that waits for a signal
`asyncio.Condition`	Combines a lock with waiting for a state to change
`asyncio.BoundedSemaphore`	Raises an error when it is released more often than acquired
`PriorityQueue` and `LifoQueue`	Take items out by priority, or newest first
`queue.join()`	Waits until `task_done()` has been called once for every item put in

Reach for a lock when two tasks must not interleave inside one block, and for an event when many tasks wait for one moment, such as a configuration that has finished loading.

Checkpoint

Check: Locks, events and other primitives

Question 1 of 2 Twenty tasks must wait until a configuration has finished loading, then all go on together. Which primitive fits?
Choose one answer.

asyncio.BoundedSemaphore asyncio.PriorityQueue asyncio.Lock asyncio.Event

Show the answer: Question 1 of 2

Answer: asyncio.Event

An asyncio.Event wakes every task that waits for a signal; a lock is for two tasks that must not interleave inside one block.
Question 2 of 2 Five items were put in a queue, and task_done() has been called four times. What does await queue.join() do?
Choose one answer.

It returns, because the queue is empty It raises QueueShutDown It returns after a default time limit It keeps waiting until task_done() has been called a fifth time

Show the answer: Question 2 of 2

Answer: It keeps waiting until task_done() has been called a fifth time

queue.join() waits until task_done() has been called once for every item put in: five items need five calls, whatever happened to them.

Check yourself

One question at a time on semaphores, bounded queues, choosing a limit and the other primitives, the weakest idea first.

Practice that adapts to you

Practise this lesson

9 questions about 10 ideas of this lesson. Each answer is explained at once, and what you get right is kept in this browser.

Question 1 of 9 What does asyncio.Semaphore(3) guarantee for the block async with gate:?
Choose one answer.

The tasks run in three threads At most three requests a second The block runs three times At most three tasks are inside the block at the same moment

Show the answer: Question 1 of 9

Answer: At most three tasks are inside the block at the same moment

A semaphore lets at most n tasks into a block at the same time; the others wait at async with gate without blocking anything.
Question 2 of 9 100 calls each take 0.2 seconds, and an asyncio.Semaphore(10) lets 10 of them run at a time. About how many seconds does the whole batch take?
Type a number.
Your answer seconds

Show the answer: Question 2 of 9

Answer: 2 seconds (anything from 1.9 to 2.1 counts)

The calls run in waves of 10: 100 ÷ 10 = 10 waves of 0.2 seconds each, so about 2 seconds. Without the semaphore they would all run at once and take about 0.2 seconds, if the service accepted 100 calls at a time.
Question 3 of 9 What does maxsize change about an asyncio.Queue?
Choose one answer.

put() waits while the queue is full, so a fast producer cannot run far ahead The queue drops items beyond maxsize get() waits until the queue is full It sets the number of worker tasks

Show the answer: Question 3 of 9

Answer: put() waits while the queue is full, so a fast producer cannot run far ahead

With a maxsize, put() waits whenever the queue is full, which is called back-pressure: the producer can never get more than maxsize items ahead of the workers.

Question 4 of 9 What does queue_workers.py print?

What does this program print? Choose one answer.

"""A producer and two workers joined by a bounded asyncio.Queue; shutdown() (Python 3.13+) tells the workers that
no more work is coming."""
import asyncio

log = []


async def producer(queue):
    for n in range(1, 9):
        await queue.put(f"order-{n}")  # waits while the queue already holds 3 orders
        log.append(f"put order-{n} (queue holds {queue.qsize()})")
    queue.shutdown()  # no more orders: workers finish what is queued, then stop
    log.append("producer: shut the queue down")


async def worker(name, queue, done):
    while True:
        try:
            order = await queue.get()
        except asyncio.QueueShutDown:
            log.append(f"{name}: queue shut down and empty, stopping")
            return
        await asyncio.sleep(0.01)  # stands in for packing the order
        done.append((name, order))
        queue.task_done()


async def main():
    queue = asyncio.Queue(maxsize=3)
    done = []
    async with asyncio.TaskGroup() as tg:
        tg.create_task(producer(queue))
        tg.create_task(worker("worker-1", queue, done))
        tg.create_task(worker("worker-2", queue, done))
    print("\n".join(log))
    print("orders packed:", len(done), "| by worker-1:", sum(1 for w, _ in done if w == "worker-1"))


asyncio.run(main())

put order-1 (queue holds 1) put order-2 (queue holds 2) put order-3 (queue holds 3) put order-4 (queue holds 4) put order-5 (queue holds 5) put order-6 (queue holds 6) put order-7 (queue holds 7) put order-8 (queue holds 8) producer: shut the queue down worker-1: queue shut down and empty, stopping worker-2: queue shut down and empty, stopping orders packed: 8 | by worker-1: 4 put order-1 (queue holds 1) put order-2 (queue holds 2) put order-3 (queue holds 3) producer: shut the queue down worker-1: queue shut down and empty, stopping worker-2: queue shut down and empty, stopping orders packed: 3 | by worker-1: 2 put order-1 (queue holds 1) put order-2 (queue holds 2) put order-3 (queue holds 3) put order-4 (queue holds 2) put order-5 (queue holds 3) put order-6 (queue holds 2) put order-7 (queue holds 3) put order-8 (queue holds 2) producer: shut the queue down worker-1: queue shut down and empty, stopping worker-2: queue shut down and empty, stopping orders packed: 8 | by worker-1: 4

Show the answer

Answer: it prints

put order-1 (queue holds 1)
put order-2 (queue holds 2)
put order-3 (queue holds 3)
put order-4 (queue holds 2)
put order-5 (queue holds 3)
put order-6 (queue holds 2)
put order-7 (queue holds 3)
put order-8 (queue holds 2)
producer: shut the queue down
worker-1: queue shut down and empty, stopping
worker-2: queue shut down and empty, stopping
orders packed: 8 | by worker-1: 4

The queue holds at most 3 orders, so whenever it is full, as at the fourth order, put() waits until a worker has taken one, and the queue never holds more than 3. shutdown() does not throw the queued orders away: the workers pack all 8, and only when the queue is empty does get() raise QueueShutDown, which ends each worker.

Question 5 of 9 A fast parser feeds a slow database writer through an unbounded asyncio.Queue, and under load the process's memory grows without limit. What bounds it?
Choose one answer.

Use put_nowait() instead of put() Call queue.join() after every put() Add more parser tasks Give the queue a maxsize, so the parser waits whenever the writer falls behind

Show the answer: Question 5 of 9

Answer: Give the queue a maxsize, so the parser waits whenever the writer falls behind

The slow consumer sets the total time either way; a bounded queue only decides how much waits in memory meanwhile.
Question 6 of 9 An API allows each client 50 requests a minute, and its answers come back in 30 milliseconds. Which limit keeps a job of 2,000 requests within the API's rule?
Choose one answer.

asyncio.Semaphore(50) asyncio.Semaphore(1) A token bucket that refills 50 tokens a minute asyncio.Queue(maxsize=50) with one worker

Show the answer: Question 6 of 9

Answer: A token bucket that refills 50 tokens a minute

The rule counts requests per minute, so only a rate limit keeps to it. A semaphore limits how many requests are in progress at once: with 30 ms answers, even Semaphore(1) would send about 2,000 requests a minute, and a queue with one worker behaves the same way.
Question 7 of 9 Does asyncio come with a rate limiter that keeps to a rule such as "100 requests a minute"?
Choose one answer.

Yes, asyncio.Queue(maxsize=100) No; it is a few lines of your own, such as a token bucket, or part of the client library you use Yes, asyncio.Semaphore with a time argument Yes, asyncio.timeout()

Show the answer: Question 7 of 9

Answer: No; it is a few lines of your own, such as a token bucket, or part of the client library you use

The comparison table's last row: semaphores and queues are built in, a token bucket is a few lines of your own.
Question 8 of 9 Which primitive raises an error when it is released more often than it was acquired?
Choose one answer.

asyncio.Lock asyncio.Event asyncio.Semaphore asyncio.BoundedSemaphore

Show the answer: Question 8 of 9

Answer: asyncio.BoundedSemaphore

A BoundedSemaphore raises an error when it is released more often than acquired, which turns a miscounted release into a visible bug.
Question 9 of 9 Jobs must leave a queue by urgency, the most urgent first, whatever order they arrived in. Which queue fits?
Choose one answer.

asyncio.PriorityQueue asyncio.LifoQueue asyncio.Event asyncio.Queue

Show the answer: Question 9 of 9

Answer: asyncio.PriorityQueue

PriorityQueue and LifoQueue take items out by priority or newest first; the plain Queue is first in, first out.

Producer-consumer pipelines in depth: back-pressure, failures and a crawler Pro: opens with a Pro, Premium or Ultimate pass

A queue between stages decides how much memory a pipeline uses and how it fails. Below, back-pressure is measured, a worker that dies is reproduced with its three outcomes, and a crawler is built with a seen-set and a clean stop.

In this part:

Back-pressure measured: 200 items with and without maxsize
A worker that dies, reproduced: join() waits for ever
A crawler with three workers, a seen-set and a clean stop

3 runnable examples (Python) · about 7 minutes

Opening this part with your pass…

This part opens with any MySmartCoPilot pass.

Pro pass · 1 week₹79$2.99
30 days₹179$5.99
1 year₹1,699$45

One payment; a pass never renews. The price shown is the total you pay: no fees or charges are added at checkout.

Choose a pass I have a licence key

It also opens every other in-depth part in Learn and every Pro tool. Sales are final, except where the law gives you a right to withdraw, for example in the EU and the UK (refund policy).

Rate limits with numbers: token buckets, bursts and 429 responses Pro: opens with a Pro, Premium or Ultimate pass

A rate limit is arithmetic over time, so it is easy to get wrong by a factor of ten. Below, a token bucket is worked through on a fake clock, compared with a semaphore on the same requests, and taught to obey a server’s 429 answers.

In this part:

A token bucket worked through on a fake clock
Concurrency limit or rate limit? The same 30 requests both ways
Obeying 429 Too Many Requests and Retry-After
Exercise: crawl a site with a few workers, each page once
Crawl a site with a few workers, each page once
- Results of the sample tests

1 diagram · 3 runnable examples (Python) · 1 exercise with sample tests · about 9 minutes

Opening this part with your pass…

This part opens with any MySmartCoPilot pass.

Pro pass · 1 week₹79$2.99
30 days₹179$5.99
1 year₹1,699$45

One payment; a pass never renews. The price shown is the total you pay: no fees or charges are added at checkout.

Choose a pass I have a licence key

It also opens every other in-depth part in Learn and every Pro tool. Sales are final, except where the law gives you a right to withdraw, for example in the EU and the UK (refund policy).

Interview questions

Mid to senior

A pipeline's await queue.join() never returns in production, although every worker looks idle. List the likely causes, and change the code so that this failure is quick and loud instead of silent.

Model answer, Pro. Model answers open with any pass; prices are in the notice on this page. First step and follow-ups
Senior to staff

Design an asyncio crawler for 1 million URLs on 5,000 domains, with at most 1 request a second per domain and at most 200 connections in total. Estimate the run time and describe the queues, the limits and the failure handling.

Model answer, Pro. First step and follow-ups
Mid to senior

A partner API allows 100 requests a minute with bursts of 20, and your job has 6,000 calls to make. Estimate the run time, design the limiter, and say what the client does when the API answers 429 anyway.

Model answer, Pro. First step and follow-ups

Key takeaways

asyncio.Semaphore(n) lets at most n tasks into a block at once; waiting at it is cheap.
asyncio.Queue connects producers and workers; maxsize makes fast producers wait, which is back-pressure.
Since Python 3.13, Queue.shutdown() lets workers finish what is queued and then raises QueueShutDown in get().
A semaphore limits concurrency, a token bucket limits requests per second; they answer different rules.
asyncio’s queues and synchronization primitives work within one event loop and are not thread-safe.

Terms in this lesson

back-pressure: Slowing a producer down when its consumers fall behind, instead of letting work pile up in memory. A bounded asyncio.Queue does it: put() waits while the queue already holds maxsize items.
token bucket: A rate limit in which each request spends a token and tokens come back at a fixed rate, so requests start no faster than that rate however many tasks are asking.

References

asyncio: Queues (Python Software Foundation)
asyncio: Queue.shutdown() (Python Software Foundation)
asyncio: Synchronization Primitives (Python Software Foundation)
asyncio: Semaphore (Python Software Foundation)
What's New In Python 3.13: asyncio (Queue.shutdown) (Python Software Foundation)
RFC 6585: Additional HTTP Status Codes (429 Too Many Requests) (IETF)
RFC 9110: HTTP Semantics, Retry-After (IETF)

Report a problem with this lesson

In this module: Async programming with asyncio

Async Python: the event loop mental model Free lesson, Pro answers: 25 minutes
Tasks, gather and running coroutines Free core, Pro depth: 9 of 25 minutes free
TaskGroup and structured concurrency Free core, Pro depth: 9 of 25 minutes free
Timeouts, cancellation and clean-up in asyncio Free core, Pro depth: 8 of 20 minutes free
Async iterators, generators and context managers Free core, Pro depth: 7 of 20 minutes free
Queues, semaphores and rate limits in asyncio Free core, Pro depth: 7 of 20 minutes free
Networking with asyncio streams Free core, Pro depth: 7 of 20 minutes free
Mixing blocking code with asyncio Free core, Pro depth: 7 of 20 minutes free
Context variables and thread-local data Free core, Pro depth: 10 of 25 minutes free
Debugging asyncio with 3.14 introspection tools Free core, Pro depth: 7 of 20 minutes free
Choosing concurrency: a measured comparison Free core, Pro depth: 9 of 25 minutes free

Your country

Queues, semaphores and rate limits in asyncio

What you will learn

Before you start

What you will learn

Before you start

A semaphore lets n tasks in at a time, and the rest wait cheaply

A bounded queue makes a fast producer wait for its workers

Semaphore, queue or token bucket?

A lock, an event and four more tools cover the rest

Check yourself

Producer-consumer pipelines in depth: back-pressure, failures and a crawler Pro: opens with a Pro, Premium or Ultimate pass

Rate limits with numbers: token buckets, bursts and 429 responses Pro: opens with a Pro, Premium or Ultimate pass

Interview questions

A pipeline's `await queue.join()` never returns in production, although every worker looks idle. List the likely causes, and change the code so that this failure is quick and loud instead of silent.

Design an asyncio crawler for 1 million URLs on 5,000 domains, with at most 1 request a second per domain and at most 200 connections in total. Estimate the run time and describe the queues, the limits and the failure handling.

A partner API allows 100 requests a minute with bursts of 20, and your job has 6,000 calls to make. Estimate the run time, design the limiter, and say what the client does when the API answers 429 anyway.

Key takeaways

References

Related tools

What you will learn

Before you start

What you will learn

Before you start

A semaphore lets n tasks in at a time, and the rest wait cheaply

A bounded queue makes a fast producer wait for its workers

Semaphore, queue or token bucket?

A lock, an event and four more tools cover the rest

Check yourself

Producer-consumer pipelines in depth: back-pressure, failures and a crawler Pro: opens with a Pro, Premium or Ultimate pass

Rate limits with numbers: token buckets, bursts and 429 responses Pro: opens with a Pro, Premium or Ultimate pass

Interview questions

A pipeline's await queue.join() never returns in production, although every worker looks idle. List the likely causes, and change the code so that this failure is quick and loud instead of silent.

Design an asyncio crawler for 1 million URLs on 5,000 domains, with at most 1 request a second per domain and at most 200 connections in total. Estimate the run time and describe the queues, the limits and the failure handling.

A partner API allows 100 requests a minute with bursts of 20, and your job has 6,000 calls to make. Estimate the run time, design the limiter, and say what the client does when the API answers 429 anyway.

Key takeaways

References

Related tools

A pipeline's `await queue.join()` never returns in production, although every worker looks idle. List the likely causes, and change the code so that this failure is quick and loud instead of silent.