Python Module 16 – Practical Python: text, CLIs, web, automation and security

Logging for scripts, applications and libraries

Set up Python logging for a script, an application and a library: levels, loggers per module, handlers, formatters and propagation, with examples run.

Intermediate
20 minutes
Examples run with Python 3.14.8 and Pyodide 314.0.7
By MySmartCoPilot

Free: A script needs one basicConfig() call: a format and a level · An application configures the root logger once, and every module logs by name · A library adds a NullHandler and leaves the rest to the application · Check yourself. With a pass: 2 in-depth parts and 3 interview answers, about 12 minutes.

What you will learn

Configure logging with levels, handlers and formatters
Use module-level loggers and propagation correctly
Add a NullHandler in a library and leave configuration to the application
Configure an application's logging from a file with dictConfig
Log safely without leaking secrets or personal data

Before you start

Designing error handling for real programs

What you’ll learn · Before you start

What you will learn

Configure logging with levels, handlers and formatters
Use module-level loggers and propagation correctly
Add a NullHandler in a library and leave configuration to the application
Configure an application's logging from a file with dictConfig
Log safely without leaking secrets or personal data

Before you start

Designing error handling for real programs

On this page

A backup script copies 118 of 120 files, and the two it skipped leave no trace, because its progress went to print() and its problems nowhere. The logging module records what a program does with a level of importance on every line, the name of the code that made it and the time, and what to show and where to send it is decided once, at the program’s entry point, not at every call.

How you set it up depends on what you are writing. Logging for scripts can be a single basicConfig() call. Applications configure handlers once, where they start, and every module logs through logging.getLogger(__name__). Libraries only log: they add a NullHandler and leave every decision to the applications that import them. Get this split right and a program can turn on detailed output for one module during an incident, send everything else to a file, and never print a library’s chatter into a user’s terminal.

A script needs one basicConfig() call: a format and a level

A backup script with timestamps, levels and a logged exception Python · script_basic.py

import logging
import sys

# One call configures logging for a small script. Logs go to standard error by default;
# stream=sys.stdout keeps this page's output in one block.
logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s %(levelname)-8s %(name)s: %(message)s",
    datefmt="%H:%M:%S",
    stream=sys.stdout,
)
log = logging.getLogger("backup")

log.debug("listing %d files", 120)                 # below INFO: never formatted, never shown
log.info("copied %d of %d files", 118, 120)
log.warning("skipped %s: permission denied", "notes.db")
try:
    ratio = 118 / 0
except ZeroDivisionError:
    log.exception("could not compute the copy ratio")   # ERROR, with the traceback

Output

21:21:08 INFO     backup: copied 118 of 120 files
21:21:08 WARNING  backup: skipped notes.db: permission denied
21:21:08 ERROR    backup: could not compute the copy ratio
Traceback (most recent call last):
  File "script_basic.py", line 18, in <module>
    ratio = 118 / 0
            ~~~~^~~
ZeroDivisionError: division by zero

This output changes from run to run: each line starts with the time of day at which it was logged.

Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 script_basic.py

basicConfig() gave the root logger one handler with a format and a threshold, and that is the whole setup. Logging for scripts rarely needs more: a format that shows the time and the level, and a level that hides the debugging details. The DEBUG line was skipped, and because the call passed its values as arguments, "listing %d files", 120, the message was never even built. log.exception(), called inside an except block, logged at ERROR and added the traceback: The report Python prints when an exception ends a program: the calls that led to the failing line, oldest first, and on the last line the exception's type and message. More in “Handling exceptions with try and except”, which print() would have lost. To keep the log in a file instead, pass filename="backup.log" and encoding="utf-8"; basicConfig() does nothing once the root logger has a handler, unless you pass force=True.

Level	Number	Use it for
DEBUG	10	Details for whoever is fixing a problem
INFO	20	Confirmation that things work as expected
WARNING	30	Something unexpected that the program handled; the default threshold
ERROR	40	A failure of one operation
CRITICAL	50	A failure that stops the program

An application configures the root logger once, and every module logs by name

Two modules, one handler on the root logger, one module turned up Python · app_modules.py

import logging
import sys

# In a real application each module creates its own logger: logging.getLogger(__name__).
# Here two names stand for two modules, shop/cart.py and shop/payment.py.
cart_log = logging.getLogger("shop.cart")
payment_log = logging.getLogger("shop.payment")


def add_to_cart(item, quantity):
    cart_log.debug("adding %s x%d", item, quantity)
    cart_log.info("cart now has %s", item)


def pay(order_id, amount):
    payment_log.debug("calling the gateway for %s", order_id)
    payment_log.warning("gateway slow for %s: %.1f s", order_id, 2.4)


def main():
    # The entry point configures logging once: one handler, on the root logger.
    handler = logging.StreamHandler(sys.stdout)
    handler.setFormatter(logging.Formatter("%(levelname)-7s %(name)-12s %(message)s"))
    root = logging.getLogger()
    root.addHandler(handler)
    root.setLevel(logging.INFO)
    logging.getLogger("shop.payment").setLevel(logging.DEBUG)    # more detail from one module

    add_to_cart("tea", 2)
    pay("A17", 349.0)

    for name in ("shop.cart", "shop.payment"):
        logger = logging.getLogger(name)
        level = logging.getLevelName(logger.getEffectiveLevel())
        print(f"{name}: effective level {level}, own handlers {len(logger.handlers)}, propagate {logger.propagate}")


main()

Output

INFO    shop.cart    cart now has tea
DEBUG   shop.payment calling the gateway for A17
WARNING shop.payment gateway slow for A17: 2.4 s
shop.cart: effective level INFO, own handlers 0, propagate True
shop.payment: effective level DEBUG, own handlers 0, propagate True

Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 app_modules.py

What happens to one log call

Text description of the diagram

The diagram is a column of six boxes, from the call down to the written line.

The code calls payment_log.debug with a message and the order id as an argument.
The logger shop.payment, whose level is DEBUG, checks whether the call's level is high enough. If it is not, nothing else happens and the message is never built.
Because it is, a LogRecord is created. The message and its argument are kept apart; the text is not built yet.
The record propagates up the tree of loggers, to shop and then to the root logger, and is offered to their handlers.
The root logger's handler checks its own level and runs its filters, which may change or drop the record.
The handler's formatter builds the line, DEBUG shop.payment calling the gateway for A17, and the handler writes it.

Logger names form a tree by their dots: shop.payment is a child of shop, which is a child of the root logger. Applications create one logger per module with logging.getLogger(__name__), so the names follow the package, and attach handlers only to the root logger, in main(). A record travels up the tree to every ancestor’s handlers, which is called The way a log record, once a logger has accepted it, is passed to the handlers of each ancestor logger up the tree of dotted names, until it reaches the root logger or a logger whose propagate is False., so the two module loggers needed no handlers of their own. Step through one call of the program, and one that stops at the first check:

Interactive · stage by stage One log call, from the call to the written line Step through the DEBUG call of payment_log in app_modules.py, then the DEBUG call of cart_log, which stops at the first check.

The drawing shows the whole scene; its 7 steps are listed below.

Every step as text

The call. payment_log.debug() is called with the message "calling the gateway for %s" and the order A17 as its argument. The two are passed separately; nothing has been formatted.
The first check. shop.payment was given its own level, DEBUG, in main(), so a DEBUG call is high enough and the call goes on.
A record. A LogRecord is created with the level, the logger's name, the time, the message and its argument. The text of the message is still not built.
Propagation. shop.payment has no handler of its own, so the record goes up the tree of dotted names. shop has none either, and the root logger has one.
The handler's checks. The handler has no level of its own and no filters in this program, so it takes the record.
The line. The handler's formatter builds the line from the record, filling in %s, and the handler writes it: DEBUG shop.payment calling the gateway for A17.
A call that stops early. cart_log.debug() stops at the first check. shop.cart has no level of its own, so it uses the root's INFO, its effective level; no record is made and the message is never built.

A logger without a level of its own uses its nearest ancestor’s, its The level a logger really uses: its own level if it has one, otherwise that of its nearest ancestor with a level, up to the root logger, whose default is WARNING.: shop.cart stayed at the root’s INFO, while shop.payment was turned up to DEBUG without touching any other module. Handlers have levels too, checked after the logger’s, so one application can write everything from DEBUG up to a file while its console shows only warnings.

Larger applications describe the same setup as data. logging.config.dictConfig() builds loggers, handlers and formatters from a dictionary, which can come from a TOML or JSON file. A filter can change or drop records before a handler writes them. A formatter decides how each record becomes text, for example one JSON object per line.

Checkpoint

Check: Logging for a script · Loggers, propagation and levels in an application

Question 1 of 2 Why did the backup script not show its line "listing 120 files"?
Choose one answer.

DEBUG lines appear only with force=True The format had no %(levelname)s field It was logged at DEBUG, and basicConfig() set the threshold to INFO log.debug() always writes to standard error, which the page does not show

Show the answer: Question 1 of 2

Answer: It was logged at DEBUG, and basicConfig() set the threshold to INFO

Every call is compared with the logger's level first. With level=logging.INFO, DEBUG calls are skipped, and because the values were passed as arguments, the message was never even formatted.
Question 2 of 2 A program calls basicConfig(level=logging.WARNING) and sets the logger shop to DEBUG. What happens to logging.getLogger('shop.cart.items').debug('saved')?
Choose one answer.

It raises an error, because shop.cart was never created It is dropped, because a logger without a level uses WARNING It is dropped, because the root logger is at WARNING It is written: the nearest ancestor with a level is shop, at DEBUG

A hint

Which ancestor of shop.cart.items is the nearest one with a level of its own?

Show the answer: Question 2 of 2

Answer: It is written: the nearest ancestor with a level is shop, at DEBUG

The effective level of a logger is its own level or that of its nearest ancestor with one, so shop.cart.items works at DEBUG, 10, and the root's handler writes the line.

A library adds a NullHandler and leaves the rest to the application

A library logger with and without a NullHandler Python · library_quiet.py

import contextlib
import io
import logging
import sys

# A library logs under its own name and adds a NullHandler, and nothing else.
quiet = logging.getLogger("paylib")
quiet.addHandler(logging.NullHandler())
# The same code without the NullHandler, for comparison.
noisy = logging.getLogger("oldlib")


def charge(logger, amount):
    logger.warning("retrying a charge of %.2f", amount)
    logger.info("charge accepted")


# 1. The application has configured nothing yet. Capture standard error to see what reaches it.
captured = io.StringIO()
with contextlib.redirect_stderr(captured):
    charge(quiet, 100)
    charge(noisy, 100)
print("1. standard error got:", repr(captured.getvalue()))

# 2. Now the application configures logging, and the library's messages follow its rules.
logging.basicConfig(level=logging.INFO, format="%(levelname)s %(name)s: %(message)s", stream=sys.stdout)
print("2. after basicConfig():")
charge(quiet, 100)

Output

1. standard error got: 'retrying a charge of 100.00\n'
2. after basicConfig():
WARNING paylib: retrying a charge of 100.00
INFO paylib: charge accepted

Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 library_quiet.py

When nothing is configured, Python sends WARNING and above to a handler of last resort, which writes bare messages to standard error. The library without a NullHandler, oldlib, leaked “retrying a charge of 100.00” into the user’s terminal before the application had decided anything; paylib, with one, stayed silent. Once the application called basicConfig(), both of paylib’s messages appeared in the application’s format and at its level. The Logging HOWTO strongly advises libraries to log through a logger of their own name, never the root logger, and to add no handler other than a NullHandler, because configuring handlers is the application’s job; basicConfig() adds a handler to the root logger, so a library never calls it either. The HOWTO leaves the NullHandler itself to the library, and adding it is what keeps the library silent until the application decides where the output goes.

Logging an error and raising it too

A library that writes log.error("charge failed") and then raises makes the same failure appear twice, once in its log and once wherever the caller handles it. Raise only: the caller decides whether the failure deserves a log line, a retry or neither.

Checkpoint

Check: Logging in a library

One question A library without a NullHandler logs a warning before the application has configured logging. What does the user see?
Choose one answer.

The message in the application's format Nothing, because nothing was configured An error saying that no handler could be found The bare message on standard error, written by the handler of last resort

Show the answer: One question

Answer: The bare message on standard error, written by the handler of last resort

oldlib's warning reached the terminal as "retrying a charge of 100.00" before the application decided anything. A NullHandler on the library's top-level logger keeps it silent until the application configures logging.

Check yourself

Scripts, applications and libraries each get their own questions here, asked one at a time, beginning with the one your answers so far show you know least.

Practice that adapts to you

Practise this lesson

8 questions about 10 ideas of this lesson. Each answer is explained at once, and what you get right is kept in this browser.

Question 1 of 8 A script has configured nothing. What does it write to standard error?
Read the code, then choose one answer.
```
import logging
logging.warning("disk %d%% full", 91)
logging.info("cleanup done")
```
WARNING:root:disk 91% full WARNING:root:disk 91% full INFO:root:cleanup done Nothing, because logging was never configured disk 91% full
Show the answer: Question 1 of 8

Answer: WARNING:root:disk 91% full

The module-level functions such as logging.warning() call basicConfig() when the root logger has no handler, which installs the default format LEVEL:name:message and the default threshold, WARNING. The INFO line is below that threshold, so only the warning is written.
Question 2 of 8 DEBUG is switched off. Which call does the least work?
Choose one answer.

log.debug(f"cart {cart}") log.debug("cart {}".format(cart)) log.debug("cart " + str(cart)) log.debug("cart %s", cart)

Show the answer: Question 2 of 8

Answer: log.debug("cart %s", cart)

With %s and a separate argument, the logger checks the level first and never builds the message, so str(cart) never runs. The other three build the text before debug() is even called.
Question 3 of 8 A script calls basicConfig(level=logging.WARNING), and later basicConfig(level=logging.DEBUG). What is the root logger's level afterwards?
Choose one answer.

NOTSET, because the two calls cancel out An error, because logging may be configured only once DEBUG, because the last call wins WARNING, because the second call does nothing once the root logger has a handler

Show the answer: Question 3 of 8

Answer: WARNING, because the second call does nothing once the root logger has a handler

The level stays 30, WARNING, after the second call, and becomes 10, DEBUG, only when that call passes force=True.
Question 4 of 8 Which of the two DEBUG lines is written?
Read the code, then choose one answer.
```
import logging, sys
logging.basicConfig(level=logging.INFO, stream=sys.stdout, format="%(levelname)s %(name)s %(message)s")
logging.getLogger("shop.payment").setLevel(logging.DEBUG)
logging.getLogger("shop.payment.upi").debug("polling status")
logging.getLogger("shop.cart").debug("cart saved")
```
Both lines Neither line Only DEBUG shop.cart cart saved Only DEBUG shop.payment.upi polling status
Show the answer: Question 4 of 8

Answer: Only DEBUG shop.payment.upi polling status

A logger without a level of its own uses the level of its nearest ancestor that has one. shop.payment.upi inherits DEBUG from shop.payment; shop.cart inherits INFO from the root logger, so its DEBUG line is dropped.
Question 5 of 8 How does each module of an application get its own logger?
Choose one answer.

log = logging.basicConfig() in every module log = logging.getLogger(__name__) at the top of the module log = logging.getLogger() in every module log = logging.Logger("app") in every module

Show the answer: Question 5 of 8

Answer: log = logging.getLogger(__name__) at the top of the module

getLogger(__name__) gives shop.cart for shop/cart.py and shop.payment for shop/payment.py, so the application can set levels by package and module.
Question 6 of 8 An application must write every line from DEBUG up to a file, while its console shows only warnings and errors. Which setup does that?
Choose one answer.

The root logger at DEBUG, with a file handler at DEBUG and a console handler at WARNING The console handler at DEBUG, and a filter that drops everything below WARNING on the file handler The root logger at WARNING, with a file handler at DEBUG Two root loggers, one at DEBUG and one at WARNING

Show the answer: Question 6 of 8

Answer: The root logger at DEBUG, with a file handler at DEBUG and a console handler at WARNING

A record must pass the logger's level and then each handler's level. Set the logger as low as the most detailed output needs, and give every handler the level of its own audience.
Question 7 of 8 You are writing a library that other programs import. What should its logging code do?
Choose one answer.

Use print() so it does not depend on the application's settings Log through logging.getLogger(__name__) and add a logging.NullHandler to its top-level logger Add a StreamHandler to the root logger so its messages are never lost Call logging.basicConfig(level=logging.DEBUG) when it is imported

Show the answer: Question 7 of 8

Answer: Log through logging.getLogger(__name__) and add a logging.NullHandler to its top-level logger

Configuration belongs to the application. A library only creates loggers and adds a NullHandler, which keeps its records from reaching Python's last-resort handler while the application has configured nothing.
Question 8 of 8 Users of your library report that its warnings appear in their terminals, although their programs never configured logging. What do you change?
Choose one answer.

Add logging.NullHandler() to the library's top-level logger Raise exceptions instead of logging warnings Remove the warnings from the library Call basicConfig(level=logging.ERROR) inside the library

Show the answer: Question 8 of 8

Answer: Add logging.NullHandler() to the library's top-level logger

The Logging HOWTO advises libraries to add no handler other than a NullHandler, which keeps the last-resort handler out of the way; basicConfig() adds a handler to the root logger, so it belongs to the application too.

Configuring real applications: dictConfig, JSON lines and three silent traps Pro: opens with a Pro, Premium or Ultimate pass

A service needs its logging set up from configuration, written in a form that machines can search, and without the mistakes that silently lose or double lines. Here a TOML file configures the logging, records become JSON with a request id, and three traps are reproduced with their fixes.

In this part:

Logging configured from a TOML file with dictConfig
One JSON object per line, with a request id that follows each request
Slow handlers off the request path: a queue, measured
Three traps reproduced: lost loggers, doubled lines and filters that miss records

1 diagram · 4 runnable examples (Python) · about 10 minutes

Opening this part with your pass…

This part opens with any MySmartCoPilot pass.

Pro pass · 1 week₹79$2.99
30 days₹179$5.99
1 year₹1,699$45

One payment; a pass never renews. The price shown is the total you pay: no fees or charges are added at checkout.

Choose a pass I have a licence key

It also opens every other in-depth part in Learn and every Pro tool. Sales are final, except where the law gives you a right to withdraw, for example in the EU and the UK (refund policy).

Logging safely and cheaply: secrets redacted, tracebacks covered, costs measured Pro: opens with a Pro, Premium or Ultimate pass

Logs outlive the requests that wrote them and are read by more people than the code, so they must not hold passwords, tokens or full card numbers, and they must not slow the program down. Here a redaction filter is built and tested, a secret is caught inside a traceback, and the cost of a log call is measured three ways.

In this part:

A redaction filter on the handler, and the password it misses in a traceback
What a log call costs when its level is off, measured
What never belongs in a log line, and what to log instead
Exercise: a redaction filter and a setup that never doubles lines
- A redaction filter and a logging setup that never doubles lines

1 diagram · 2 runnable examples (Python) · 1 exercise with sample tests · about 8 minutes

Opening this part with your pass…

This part opens with any MySmartCoPilot pass.

Pro pass · 1 week₹79$2.99
30 days₹179$5.99
1 year₹1,699$45

One payment; a pass never renews. The price shown is the total you pay: no fees or charges are added at checkout.

Choose a pass I have a licence key

It also opens every other in-depth part in Learn and every Pro tool. Sales are final, except where the law gives you a right to withdraw, for example in the EU and the UK (refund policy).

Interview questions

Fresher to mid · Warm-up

Why use the logging module instead of print() in an application?

print() writes text to one place and nothing else. A log call also records a level, from DEBUG to CRITICAL, the name of the logger that made it and the time, and the application decides once, at its entry point, which levels to show and where each line goes: the console, a file or a log server, in one format or as JSON. One module can be turned up to DEBUG during an incident without touching any other, and libraries can log without printing into their users' programs, because they add a NullHandler and leave the decision to the application. log.exception() keeps the traceback, which print() would lose, and passing values as arguments, log.info("copied %d files", n), means a switched-off level costs almost nothing. For a quick throwaway script, print() is fine; for code that others run, logging is the tool.
Mid to senior

Users of your open-source library complain that importing it writes WARNING lines to their terminal, and that they cannot turn them off without patching your code. What is the library doing wrong, and what should its logging code look like?

Model answer, Pro. Model answers open with any pass; prices are in the notice on this page. First step and follow-ups
Mid to senior

After a refactor moved logging setup into a dictConfig() loaded from a TOML file, warnings from two modules stopped appearing in production while other modules still log, and nothing raised an error. How do you find the cause, and what do you change?

Model answer, Pro. First step and follow-ups
Senior to staff

Design logging for a payments API that serves 2,000 requests a second across 30 instances: what each line contains, how a request id reaches every line, where secrets are removed, and how logging is kept from slowing requests down.

Model answer, Pro. First step and follow-ups

Key takeaways

Logging for scripts is one basicConfig() call with a format and a level, and log.exception() adds the traceback.
Applications create logging.getLogger(__name__) in each module and attach handlers once, to the root logger, because records propagate up the dotted names.
A logger without its own level uses its nearest ancestor’s, so one module can be turned up alone.
Libraries add a NullHandler and never configure logging.
Pass values as arguments, log.info("copied %d files", n), so that a skipped level costs little.

References

Logging HOWTO (Python Software Foundation)
Logging HOWTO: Configuring logging for a library (Python Software Foundation)
logging: Logging facility for Python (Python Software Foundation)
logging: Filter objects (Python Software Foundation)
logging.config: dictConfig() and the configuration dictionary schema (Python Software Foundation)
Logging Cookbook (Python Software Foundation)
contextvars: Context Variables (Python Software Foundation)
Logging Cheat Sheet (OWASP Foundation)

Report a problem with this lesson

In this module: Practical Python: text, CLIs, web, automation and security

Regular expressions with the re module Free lesson, Pro answers: 25 minutes
Text tools: textwrap, difflib and string Free core, Pro depth: 8 of 20 minutes free
Template strings (t-strings) in Python 3.14 Free core, Pro depth: 8 of 20 minutes free
Logging for scripts, applications and libraries Free core, Pro depth: 8 of 20 minutes free
Command-line apps with argparse Free core, Pro depth: 8 of 20 minutes free
Working with the OS: os, sys, shutil, platform Free core, Pro depth: 8 of 20 minutes free
Calling web APIs from Python with urllib Free core, Pro depth: 9 of 25 minutes free
Serving HTTP: http.server and WSGI basics Free core, Pro depth: 7 of 20 minutes free
Web scraping responsibly: robots.txt and parsing Free core, Pro depth: 7 of 20 minutes free
Socket programming basics: TCP and UDP Free core, Pro depth: 8 of 20 minutes free
Automate office tasks: files, Excel, PDF, email Free core, Pro depth: 7 of 20 minutes free
Desktop GUIs with tkinter: a first app Free core, Pro depth: 7 of 20 minutes free
Secure tokens, password hashes and HMAC Free core, Pro depth: 8 of 20 minutes free

Your country

Logging for scripts, applications and libraries

What you will learn

Before you start

What you will learn

Before you start

A script needs one basicConfig() call: a format and a level

An application configures the root logger once, and every module logs by name

A library adds a NullHandler and leaves the rest to the application

Check yourself

Configuring real applications: dictConfig, JSON lines and three silent traps Pro: opens with a Pro, Premium or Ultimate pass

Logging safely and cheaply: secrets redacted, tracebacks covered, costs measured Pro: opens with a Pro, Premium or Ultimate pass

Interview questions

Why use the logging module instead of print() in an application?

Users of your open-source library complain that importing it writes WARNING lines to their terminal, and that they cannot turn them off without patching your code. What is the library doing wrong, and what should its logging code look like?

After a refactor moved logging setup into a dictConfig() loaded from a TOML file, warnings from two modules stopped appearing in production while other modules still log, and nothing raised an error. How do you find the cause, and what do you change?

Design logging for a payments API that serves 2,000 requests a second across 30 instances: what each line contains, how a request id reaches every line, where secrets are removed, and how logging is kept from slowing requests down.

Key takeaways

References

Related tools