C++ Module 16 – Concurrency: threads, atomics and the memory model

C++ condition variables and producer-consumer queues

Wait for a condition in C++ without busy-waiting: condition_variable with a predicate, a bounded producer-consumer queue, spurious wake-ups and shutdown.

Advanced
30 minutes
Examples run with Clang 21.0.0, GCC 16.2.0 and Python 3.14.8
By MySmartCoPilot

Free: Waiting without burning a core · The rule for waiting: a predicate, under the mutex · A bounded queue needs one mutex and two condition variables · 2 more sections. With a pass: 3 in-depth parts and 3 interview answers, about 19 minutes.

What you will learn

Use a condition variable and a predicate to wait without busy-waiting
Build a bounded producer-consumer queue with one mutex and two condition variables
Handle spurious wake-ups and shut a queue down without losing items
Reproduce a lost wake-up and fix it by changing the state under the mutex
Cancel a waiting thread with a std::stop_token and std::condition_variable_any

Before you start

Data races, mutexes and lock guards in C++

What you’ll learn · Before you start

What you will learn

Use a condition variable and a predicate to wait without busy-waiting
Build a bounded producer-consumer queue with one mutex and two condition variables
Handle spurious wake-ups and shut a queue down without losing items
Reproduce a lost wake-up and fix it by changing the state under the mutex
Cancel a waiting thread with a std::stop_token and std::condition_variable_any

Before you start

Data races, mutexes and lock guards in C++

On this page

A thread that waits 200 ms for a flag by testing it in a loop keeps a core busy for most of that time; a thread that waits on a condition variable: A synchronisation object, std::condition_variable in C++, on which threads sleep until another thread notifies them. It works together with a mutex and a condition that the waiting thread checks before and after it sleeps. uses about a hundredth of a millisecond of it. A condition variable lets a thread sleep until another thread changes some shared state and tells it so. It always works together with a mutex: A mutual-exclusion object, std::mutex in C++: a thread locks it before touching shared data and unlocks it afterwards, and while one thread holds it, every other thread that tries to lock it waits. More in “Data races, mutexes and lock guards in C++” and a condition: the waiting thread locks the mutex, checks the condition and, if it is false, calls wait(), which releases the mutex and goes to sleep in one step. The thread that changes the state does so under the same mutex and then calls notify_one() or notify_all().

C++ condition variables come in two types: std::condition_variable, which works with std::unique_lock<std::mutex>, and std::condition_variable_any, which works with any lock and can also be woken by a stop request. The classic use is a producer-consumer queue: A queue that connects threads that make work, the producers, with threads that do it, the consumers. A bounded one holds at most a fixed number of items, so a fast producer waits instead of filling memory., which this lesson builds with a capacity limit, runs with two producers and two consumers, and shuts down cleanly.

Waiting without burning a core

A thread could wait by testing a flag in a loop. The program below waits 200 ms twice, once like that and once on a condition variable, and each waiter measures the processor time its own thread used:

wait_vs_spin.cpp C++ · wait_vs_spin.cpp

// A thread waits 200 ms for a signal from main, twice: once by spinning on an atomic flag, once with a condition
// variable. Each waiter measures the processor time its own thread used while it waited.
#include <atomic>
#include <chrono>
#include <condition_variable>
#include <ctime>
#include <mutex>
#include <print>
#include <thread>

using namespace std::chrono_literals;

// Processor time used so far by the calling thread, in milliseconds (POSIX clock_gettime).
double thread_cpu_ms() {
    timespec t{};
    clock_gettime(CLOCK_THREAD_CPUTIME_ID, &t);
    return t.tv_sec * 1000.0 + t.tv_nsec / 1e6;
}

int main() {
    // 1. Busy-waiting: the waiter tests the flag again and again until main sets it.
    std::atomic<bool> flag{false};
    double spin_cpu = 0;
    {
        std::jthread waiter([&] {
            const double start = thread_cpu_ms();
            while (!flag.load()) {
            }
            spin_cpu = thread_cpu_ms() - start;
        });
        std::this_thread::sleep_for(200ms);
        flag = true;
    }

    // 2. A condition variable: the waiter sleeps until main notifies it, then checks the condition under the mutex.
    std::mutex m;
    std::condition_variable cv;
    bool ready = false;  // guarded by m
    double cv_cpu = 0;
    {
        std::jthread waiter([&] {
            const double start = thread_cpu_ms();
            std::unique_lock lock(m);
            cv.wait(lock, [&] { return ready; });
            cv_cpu = thread_cpu_ms() - start;
        });
        std::this_thread::sleep_for(200ms);
        {
            std::lock_guard lock(m);
            ready = true;
        }
        cv.notify_one();
    }

    std::println("busy-waiting for 200 ms used {:.0f} ms of processor time", spin_cpu);
    std::println("waiting on a condition variable used {:.2f} ms", cv_cpu);
}

Processor time used while waiting 200 ms Python · spin_session.py

from session import run

# wait_vs_spin.cpp built with optimisation by Apple clang and run once.
run("""
clang++ -std=c++23 -O2 -Wall -Wextra wait_vs_spin.cpp -o wait_vs_spin && ./wait_vs_spin
""")

Output

$ clang++ -std=c++23 -O2 -Wall -Wextra wait_vs_spin.cpp -o wait_vs_spin && ./wait_vs_spin
busy-waiting for 200 ms used 118 ms of processor time
waiting on a condition variable used 0.01 ms

This output changes from run to run: processor times change from run to run, and more while other programs keep the computer busy

Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 spin_session.py

The spinning waiter kept a core busy for most of the 200 ms, doing nothing useful and taking the core from threads that had work. The condition variable cost about a hundredth of a millisecond: the thread slept until the notification and used the processor only to wake up and check the flag. On a phone or a laptop, that difference is battery; on a server, it is a core that other requests can use.

Checkpoint

Check: Waiting by sleeping, not spinning

Question 1 of 2 The condition is false, and a thread calls cv.wait(lock). What does the thread do until it is woken?
Choose one answer.

It keeps checking the condition as fast as it can It releases the mutex and sleeps, using no processor time It sleeps while still holding the mutex It returns at once and lets the caller retry

Show the answer: Question 1 of 2

Answer: It releases the mutex and sleeps, using no processor time

wait() gives up the mutex and puts the thread to sleep in one step; it uses the processor again only to wake up and check the condition. A spinning loop instead keeps a core busy for the whole wait.
Question 2 of 2 Eight threads on a busy 8-core server busy-wait for work that arrives about once a second. What does that cost the server?
Choose one answer.

Nothing, because the threads do no real work Each spinning thread keeps a core busy doing nothing, cores that threads with real work could use The threads sleep automatically after a while Only memory, not processor time

Show the answer: Question 2 of 2

Answer: Each spinning thread keeps a core busy doing nothing, cores that threads with real work could use

Busy-waiting turns waiting into processor time. On a server that time belongs to other requests, and on a phone or a laptop it is battery. A condition variable lets the waiters sleep until the work arrives.

The rule for waiting: a predicate, under the mutex

cv.wait(lock, pred), where pred is a A function or lambda that returns true or false. For a condition-variable wait it takes no arguments and says whether the condition the thread is waiting for holds now., is shorthand for a loop, while (!pred()) cv.wait(lock);, and the loop matters, because a return from wait() does not mean the condition is true:

the standard allows spurious wake-up: A return from a condition variable's wait() that no notification caused. The C++ standard and POSIX both allow it, which is why a wait always rechecks its condition in a loop., returns from wait() that no notification caused (POSIX says the same of pthread_cond_wait, on which both libraries build);
a notified thread must lock the mutex again before wait() returns, and in that time another thread may already have taken the item it was woken for.

The loop handles both, and it also returns at once when the condition was already true before the thread waited. The side that changes the state follows one rule too: change it while holding the mutex, release the mutex, then notify. The Core Guidelines put the waiting half as CP.42, don’t wait without a condition.

The waiter of wait_vs_spin.cpp went through that loop once. Step through its pass, from the first check to the return, with the mutex m and the flag ready it reads:

Interactive · step by step What cv.wait(lock, pred) does, step by step Step through the waiter of wait_vs_spin.cpp. The loop on the left runs while pred() is false; the right column is where the thread goes on.

The drawing shows the whole scene; its 7 steps are listed below.

Every step as text

Lock, then look. The waiter locks m and checks its predicate. ready is false: main will set it only after 200 ms.
Release and sleep, at once. wait() releases m and puts the thread to sleep, in one step. Asleep, the thread uses no processor time until something wakes it.
main changes the state. After 200 ms main locks m, sets ready to true, releases m and calls notify_one(). The change is made while holding the mutex the waiter checks it under.
Woken up. The notification wakes the waiter. A wake-up can also come with no notification at all, a spurious wake-up, which is why the loop never trusts it.
The mutex first. Before wait() returns, the thread locks m again. If another thread holds it, the waiter waits for it here.
Check again. Back at the predicate, with m held. Had another thread already taken what the waiter was woken for, it would find the condition false and sleep again.
Done. ready is true, so the wait returns with m still held, and the waiter may read the state that m guards. The whole wait used about a hundredth of a millisecond of processor time.

Checkpoint

Check: The predicate loop under the mutex

Question 1 of 2 What is wrong with this consumer?
Read the code, then choose one answer.
```
std::unique_lock lock(m);
cv.wait(lock);              // wait for an item
int item = items.front();   // take it
items.pop_front();
```
front() should be called before wait() wait() can return when the queue is empty, after a spurious wake-up or after another consumer took the item, and front() then reads an empty deque Nothing; wait() returns only after a notification for this thread The lock should be released before wait()
Show the answer: Question 1 of 2

Answer: wait() can return when the queue is empty, after a spurious wake-up or after another consumer took the item, and front() then reads an empty deque

Write cv.wait(lock, [&] { return !items.empty(); });, which loops until the condition really holds and returns at once if an item was already there before the wait.
Question 2 of 2 When the predicate of cv.wait(lock, pred) runs, is the mutex held?
Choose one answer.

No, it runs while the thread sleeps Yes, so the predicate may read the shared state that the mutex guards Only on the first check It depends on whether the wake-up was spurious

Show the answer: Question 2 of 2

Answer: Yes, so the predicate may read the shared state that the mutex guards

cv.wait(lock, pred) is while (!pred()) cv.wait(lock);, and the lock is held whenever that loop tests pred(). That is what makes it safe for the predicate to read the queue or the flag.

A bounded queue needs one mutex and two condition variables

Producer-consumer queues connect threads that make work with threads that do it. The standard library has no producer-consumer queues, so programs build them from a mutex and two condition variables. Bounded queues hold at most a fixed number of items, so a fast producer waits instead of filling memory:

bounded_queue.h C header · bounded_queue.h

// A bounded, blocking producer-consumer queue: push() waits while the queue is full, pop() waits while it is empty,
// and close() ends both waits for good.
#pragma once
#include <condition_variable>
#include <cstddef>
#include <deque>
#include <mutex>
#include <optional>
#include <utility>

template <class T>
class BoundedQueue {
public:
    explicit BoundedQueue(std::size_t capacity) : capacity_(capacity) {}

    // Waits while the queue is full. Returns false, and adds nothing, once the queue is closed.
    bool push(T value) {
        std::unique_lock lock(m_);
        not_full_.wait(lock, [&] { return items_.size() < capacity_ || closed_; });
        if (closed_) return false;
        items_.push_back(std::move(value));
        lock.unlock();           // release first, so the woken consumer does not wait for the mutex
        not_empty_.notify_one();
        return true;
    }

    // Waits while the queue is empty. Returns std::nullopt once the queue is closed and every item has been taken.
    std::optional<T> pop() {
        std::unique_lock lock(m_);
        not_empty_.wait(lock, [&] { return !items_.empty() || closed_; });
        if (items_.empty()) return std::nullopt;  // closed, and nothing left
        T value = std::move(items_.front());
        items_.pop_front();
        lock.unlock();
        not_full_.notify_one();
        return value;
    }

    // Accepts no more items. Waiting producers return false; consumers take what is left, then get std::nullopt.
    void close() {
        {
            std::lock_guard lock(m_);
            closed_ = true;
        }
        not_empty_.notify_all();
        not_full_.notify_all();
    }

private:
    std::mutex m_;                      // guards items_ and closed_
    std::condition_variable not_empty_; // a consumer waits here
    std::condition_variable not_full_;  // a producer waits here
    std::deque<T> items_;
    const std::size_t capacity_;
    bool closed_ = false;
};

A bounded queue: one mutex, two condition variables

Text description of the diagram

The diagram reads from top to bottom.

At the top, producers 1 and 2 each call push().
An arrow labelled "add an item, then notify not_empty" leads to the queue, BoundedQueue with capacity 8. Inside it, one mutex guards the items and the closed flag, the items are at most 8, and there are two condition variables: not_full, where producers wait while the queue is full, and not_empty, where consumers wait while it is empty.
An arrow labelled "take an item, then notify not_full" leads to consumers 1 and 2 at the bottom, which each call pop().

Two condition variables share the one mutex: producers wait on not_full_ while the queue is full, consumers on not_empty_ while it is empty. After a push, notify_one() wakes one consumer; after a pop, one producer. Each side releases the lock before notifying, so the thread it wakes does not wake up only to wait for the mutex. Two producers push 50,000 numbers each through a queue of capacity 8 while two consumers take them:

queue_demo.cpp C++ · queue_demo.cpp

// Two producers push 50,000 numbers each into a queue of capacity 8; two consumers pop until the queue is closed and
// empty. Each consumer adds up what it took in its own slot, and main checks the totals.
#include <cstdint>
#include <optional>
#include <print>
#include <thread>
#include <vector>

#include "bounded_queue.h"

int main() {
    BoundedQueue<std::int64_t> queue(8);
    std::int64_t count[2] = {}, sum[2] = {};  // one slot per consumer
    {
        std::vector<std::jthread> consumers;
        for (int c = 0; c < 2; ++c) {
            consumers.emplace_back([&queue, &count, &sum, c] {
                while (std::optional<std::int64_t> item = queue.pop()) {
                    ++count[c];
                    sum[c] += *item;
                }
            });
        }
        {
            std::vector<std::jthread> producers;
            for (std::int64_t p = 0; p < 2; ++p) {
                producers.emplace_back([&queue, p] {
                    for (std::int64_t i = 1; i <= 50'000; ++i) queue.push(p * 1'000'000 + i);
                });
            }
        }               // both producers have finished here
        queue.close();  // the consumers drain what is left, then pop() returns std::nullopt
    }                   // and the consumers are joined here
    std::println("consumed {} items, sum {}", count[0] + count[1], sum[0] + sum[1]);
    std::println("expected {} items, sum {}", 100'000, 52'500'050'000LL);
    std::println("push() after close() returns {}", queue.push(1));
}

Two producers, two consumers, both compilers Python · queue_session.py

from session import run

# queue_demo.cpp under ThreadSanitizer with Apple clang, then with GCC 16 under AddressSanitizer and UBSan.
run("""
clang++ -std=c++23 -g -O1 -fsanitize=thread -Wall -Wextra queue_demo.cpp -o queue_demo && ./queue_demo
g++-16 -std=c++23 -O2 -Wall -Wextra -fsanitize=address,undefined queue_demo.cpp -o queue_demo-gcc && ./queue_demo-gcc
""")

Output

$ clang++ -std=c++23 -g -O1 -fsanitize=thread -Wall -Wextra queue_demo.cpp -o queue_demo && ./queue_demo
consumed 100000 items, sum 52500050000
expected 100000 items, sum 52500050000
push() after close() returns false
$ g++-16 -std=c++23 -O2 -Wall -Wextra -fsanitize=address,undefined queue_demo.cpp -o queue_demo-gcc && ./queue_demo-gcc
consumed 100000 items, sum 52500050000
expected 100000 items, sum 52500050000
push() after close() returns false

Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 queue_session.py

All 100,000 items arrived, the sum matches, and ThreadSanitizer had nothing to report. Bounded queues also give backpressure: What a full bounded queue does to the threads that feed it: they wait at push() until there is room, so the producers slow down to the pace of the consumers instead of piling up items.: when the consumers fall behind, the producers wait at push() instead of piling items up. Follow one push() into a full queue of 8, and the pop() that lets it go on:

Interactive · message by message A full queue makes the producer wait, and a pop lets it go on Step through one push() on a full BoundedQueue of capacity 8, and the pop() that makes room. The middle column is the queue and its mutex.

The drawing shows the whole scene; its 6 steps are listed below.

Every step as text

push() finds it full. Producer 1 locks the mutex and checks not_full_'s predicate: items_.size() < capacity_ is false, and the queue is not closed.
The producer waits. So it waits on not_full_, which releases the mutex as it sleeps. This is backpressure: a producer that runs ahead waits instead of filling memory.
A consumer pops. Consumer 1 locks the mutex, finds an item, takes the oldest from the front and leaves 7. It releases the mutex before it notifies.
notify not_full_. not_full_.notify_one() wakes one thread waiting on not_full_, and only producers wait there, so the wake-up reaches a thread that can use it.
The producer goes on. Producer 1 locks the mutex again, rechecks its predicate, now true with 7 items, and pushes x. The queue is full again.
notify not_empty_. After the push it releases the mutex and calls not_empty_.notify_one(), which wakes a consumer if one is waiting for an item. Two producers and two consumers did this 100,000 times, and every item arrived.

Checkpoint

Check: A bounded producer-consumer queue

Question 1 of 2 In BoundedQueue, which thread does not_empty_.notify_one() after a push wake?
Choose one answer.

One producer that is waiting for space One consumer that is waiting in pop() for an item Every waiting thread The thread that pushed

Show the answer: Question 1 of 2

Answer: One consumer that is waiting in pop() for an item

Producers wait on not_full_ and consumers on not_empty_. After a push, not_empty_.notify_one() wakes one consumer; after a pop, not_full_.notify_one() wakes one producer.
Question 2 of 2 push() calls lock.unlock() before not_empty_.notify_one(). Why?
Choose one answer.

Because notify_one() while holding the mutex is undefined behaviour Because notify_one() releases the mutex itself So that the woken consumer does not wake up only to wait for the mutex the pusher still holds So that the item is added after the notification

Show the answer: Question 2 of 2

Answer: So that the woken consumer does not wake up only to wait for the mutex the pusher still holds

Both orders are correct, as long as the state changes under the mutex. Releasing first means the consumer that notify_one() wakes finds the mutex available instead of blocking on it straight away.

close() wakes every waiter, and the consumers drain the rest

A consumer blocked in pop() needs a way to learn that no more items will come. close() sets closed_ under the mutex and calls notify_all() on both condition variables, so every waiter wakes and rechecks its predicate, which includes closed_. Producers then get false from push(); consumers first drain what is left, then get std::nullopt, which ends their loop. The order in main matters: wait for the producers to finish, close, then let the consumers drain and join. Here is that order in queue_demo.cpp, one event at a time:

Interactive · message by message Closing the queue without losing an item Step through the end of queue_demo.cpp. main closes the queue only after both producers have finished; the consumers drain it.

The drawing shows the whole scene; its 6 steps are listed below.

Every step as text

The producers are done. The producers' jthreads are joined at the end of their block, so every one of the 100,000 numbers has been pushed. A few may still sit in the queue.
close(). main calls close(), which sets closed_ while holding the mutex, releases it and calls notify_all() on both condition variables.
Every waiter wakes. notify_all() wakes every thread waiting on the queue. Each one locks the mutex and rechecks its predicate, and closed_ is part of every predicate.
The consumers drain it. pop() returns std::nullopt only when the queue is closed and empty, so the consumers first take the items that are left.
Then std::nullopt. With the queue closed and empty, pop() returns std::nullopt, which ends each consumer's while loop, and their jthreads are joined.
Nothing was lost. The consumers took 100,000 items with a sum of 52,500,050,000, exactly what was pushed. A push() after close() returns false and adds nothing.

std::condition_variable_any offers a second way to end a wait: it can wait on a std::stop_token: A handle through which a thread can see whether a stop has been requested: its stop_requested() turns true once request_stop() is called on the matching std::jthread or std::stop_source. It never interrupts the thread; the thread checks it. More in “Threads with std::thread and std::jthread” too, so a request to stop a std::jthread wakes its waits. A predicate wait on C++ condition variables ends only when its condition becomes true, so every wait in a program needs a way out: a condition that shutdown makes true, a stop token, or a timeout.

Checkpoint

Check: Closing a queue and draining it

Question 1 of 2 close() is called on the BoundedQueue of this lesson while 3 items are still inside. What do the consumers get from pop()?
Choose one answer.

The 3 items, then std::nullopt They wait for ever, because no producer will push again std::nullopt at once; the 3 items are discarded An exception, because the queue is closed

Show the answer: Question 1 of 2

Answer: The 3 items, then std::nullopt

pop() returns std::nullopt only when the queue is closed and empty, so the consumers drain what is left first. close() changes closed_ under the mutex and notifies every waiter on both condition variables, so nobody keeps sleeping on a queue that will never change.
Question 2 of 2 A program calls close() while its producers are still pushing. What happens to the numbers they have not pushed yet?
Choose one answer.

Each later push() returns false and adds nothing, so those numbers never reach a consumer The program terminates The producers wait until the queue is opened again They are pushed anyway, and the consumers take them

Show the answer: Question 2 of 2

Answer: Each later push() returns false and adds nothing, so those numbers never reach a consumer

The order in main matters: wait for the producers to finish, close, then let the consumers drain and join. Closing first turns every later push into a refused item.

Practise: sleeping waits, predicates and a closing queue

Eleven questions follow on spinning against sleeping, the predicate loop, the two condition variables of BoundedQueue and close(). They begin with the idea you have shown least so far.

Lost wake-ups and wasted ones, reproduced Pro: opens with a Pro, Premium or Ultimate pass

A condition variable remembers nothing: a notification that arrives when no thread is waiting is gone. This part reproduces the two ways that happens in real code, runs the fix beside them, and shows why one condition variable for two conditions forces notify_all().

In this part:

Two lost wake-ups and the fix, side by side
One condition variable for two conditions

1 diagram · 2 runnable examples (C++, Python) · about 6 minutes

Opening this part with your pass…

This part opens with any MySmartCoPilot pass.

Pro pass · 1 week₹79$2.99
30 days₹179$5.99
1 year₹1,699$45

One payment; a pass never renews. The price shown is the total you pay: no fees or charges are added at checkout.

Choose a pass I have a licence key

It also opens every other in-depth part in Learn and every Pro tool. Sales are final, except where the law gives you a right to withdraw, for example in the EU and the UK (refund policy).

Waiting with timeouts and stop tokens, with an exercise Pro: opens with a Pro, Premium or Ultimate pass

A wait that cannot end makes a shutdown that cannot finish. This part waits with deadlines on the right clock, stops a waiter through a std::stop_token, and ends with an exercise that adds both to the queue.

In this part:

Timeouts with one deadline on the steady clock
Stopping a waiter with a stop token
Your turn: try_pop_for and pop(stop_token)
Which way out for which wait

4 runnable examples (C++, Python) · about 5 minutes

Opening this part with your pass…Pro part. It opens with the same pass as the part above.

What a bounded queue costs: capacity and wake-ups, measured Pro: opens with a Pro, Premium or Ultimate pass

Every handover between threads can cost a sleep and a wake-up, and the queue’s capacity decides how often that happens. This part measures four versions of the queue and turns the numbers into sizing rules.

In this part:

Capacity 1, 16 and 1,024, measured
Sizing a queue

2 runnable examples (C++, Python) · about 4 minutes

Opening this part with your pass…Pro part. It opens with the same pass as the part above.

Interview questions

Fresher to mid · Warm-up

Why should a condition-variable wait always use a predicate?

Because a return from wait() does not mean the condition is true. The standard allows spurious wake-ups, returns that no notification caused, and even a real notification can be stale by the time the woken thread has locked the mutex again: another thread may already have taken the item. The predicate form, cv.wait(lock, pred), is a loop, while (!pred()) cv.wait(lock);, so the thread goes back to sleep until the condition really holds. It also checks the condition before sleeping at all, so a thread that arrives after the notification does not wait for one that already happened. The predicate runs with the mutex held, so it may read the shared state that the mutex guards.
Mid to senior

In a production queue, consumers wait with a 500 ms timeout. Logs show consumers sometimes sleeping for the full 500 ms while an item is already in the queue. List the likely causes and explain how you would prove which one it is.

Model answer, Pro. Model answers open with any pass; prices are in the notice on this page. First step and follow-ups
Mid to senior

Design the queue between request threads that produce log records and one thread that writes them to a file. What happens when the writer falls behind, and how does shutdown work without losing records?

Model answer, Pro. First step and follow-ups
Mid to senior

A bounded queue uses one condition variable for both "not full" and "not empty", and calls notify_one() after every push and pop. Under load, with several producers and consumers, it sometimes stalls with every thread waiting. Why, and what are the fixes?

Model answer, Pro. First step and follow-ups

Key takeaways

A condition variable lets a thread sleep until another thread changes shared state; it always goes with a mutex and a condition.
Always wait with a predicate: wait() can return spuriously or after another thread took what it was woken for.
Change the shared state while holding the mutex, then notify.
A bounded queue needs one mutex and two condition variables, not full and not empty, and a close() that wakes every waiter.

Terms in this lesson

backpressure: What a full bounded queue does to the threads that feed it: they wait at push() until there is room, so the producers slow down to the pace of the consumers instead of piling up items.
condition variable: A synchronisation object, std::condition_variable in C++, on which threads sleep until another thread notifies them. It works together with a mutex and a condition that the waiting thread checks before and after it sleeps.
producer-consumer queue: A queue that connects threads that make work, the producers, with threads that do it, the consumers. A bounded one holds at most a fixed number of items, so a fast producer waits instead of filling memory.
spurious wake-up: A return from a condition variable's wait() that no notification caused. The C++ standard and POSIX both allow it, which is why a wait always rechecks its condition in a loop.

References

Working Draft, Programming Languages: C++ (N4950), 33.7 Condition variables (ISO/IEC JTC1/SC22/WG21)
pthread_cond_wait, pthread_cond_timedwait (POSIX.1-2024) (The Open Group)
Stop Token and Joining Thread, Rev 10 (P0660R10) (ISO/IEC JTC1/SC22/WG21)
C++ Core Guidelines: CP, Concurrency and parallelism (Standard C++ Foundation)
ThreadSanitizer (Clang documentation) (LLVM Project)

Report a problem with this lesson

In this module: Concurrency: threads, atomics and the memory model

Threads with std::thread and std::jthread Free lesson, Pro answers: 25 minutes
Data races, mutexes and lock guards in C++ Free core, Pro depth: 11 of 30 minutes free
C++ condition variables and producer-consumer queues Free core, Pro depth: 11 of 30 minutes free
std::async, futures and promises Free core, Pro depth: 11 of 30 minutes free
C++ atomics, compare-exchange and atomic waiting Free core, Pro depth: 11 of 30 minutes free
The C++ memory model and memory orders Free core, Pro depth: 10 of 30 minutes free
Latches, barriers and semaphores (C++20) Free core, Pro depth: 10 of 25 minutes free
Building a thread pool in C++ Free core, Pro depth: 11 of 30 minutes free
Concurrency-bug hunt: races and deadlocks Free core, Pro depth: 15 of 40 minutes free

What you will learn

Before you start

What you will learn

Before you start

Waiting without burning a core

The rule for waiting: a predicate, under the mutex

A bounded queue needs one mutex and two condition variables

close() wakes every waiter, and the consumers drain the rest

Practise: sleeping waits, predicates and a closing queue

Lost wake-ups and wasted ones, reproduced Pro: opens with a Pro, Premium or Ultimate pass

Waiting with timeouts and stop tokens, with an exercise Pro: opens with a Pro, Premium or Ultimate pass

What a bounded queue costs: capacity and wake-ups, measured Pro: opens with a Pro, Premium or Ultimate pass

Interview questions

Why should a condition-variable wait always use a predicate?

In a production queue, consumers wait with a 500 ms timeout. Logs show consumers sometimes sleeping for the full 500 ms while an item is already in the queue. List the likely causes and explain how you would prove which one it is.

Design the queue between request threads that produce log records and one thread that writes them to a file. What happens when the writer falls behind, and how does shutdown work without losing records?

A bounded queue uses one condition variable for both "not full" and "not empty", and calls notify_one() after every push and pop. Under load, with several producers and consumers, it sometimes stalls with every thread waiting. Why, and what are the fixes?

Key takeaways

References