Python regex techniques and the regex module

Three transaction messages, one of them without a reference number, go through two calls to re.findall: one for the amounts, one for the references. Zipped together, the two lists pair the 18,000.00 credit with the reference of the next message and drop the last debit, and nothing raises an error. The calls themselves are the subject of the Python track’s lesson on the re module; this one turns them into techniques that keep a text job correct: a scanner that reads the text once, a function that computes each replacement, splits that keep their separators, and calls written the way Python 3.13 and later expect.

The last section is a choice only your project can make: whether the third-party regex module, with its timeouts, Unicode properties and approximate matching, is worth adding, or whether the job belongs outside regular expressions altogether. Every example ran on Python 3.14.8, and each one runs again in your browser when you press Run.

One scan keeps the fields of each record together

The statement below holds three messages in a format made up for this lesson. The first part of the run is the bug from the opening: re.findall scans the whole text once for the amounts and a second time for the references, and zip pairs the two lists by position.

"""Three transaction messages in one text, in a format this lesson made up.

Two findall calls, one per field, give two lists that drift out of step as soon as one message
lacks a field. One record pattern scanned with finditer keeps every message's fields together,
and a token scanner reads fields that come in any order.
"""

import re

TEXT = """\
Rs 1,250.00 debited from A/c XX4821 on 05-03-31. Ref 7731.
Rs 18,000.00 credited to A/c XX4821 on 07-03-31.
Rs 99.50 debited from A/c XX4821 on 09-03-31. Ref 7790.
"""

# One findall per field: each call scans the whole text again, and zip pairs by position.
amounts = re.findall(r'Rs ([\d,]+\.\d\d)', TEXT)
refs = re.findall(r'Ref (\d+)', TEXT)
print('zip of two findall lists:')
for amount, ref in zip(amounts, refs):
    print(f'  {amount:>9}  ref {ref}')

# One pattern per message and one scan: a part that is missing comes back as None.
RECORD = re.compile(
    r'Rs (?P<amount>[\d,]+\.\d\d) (?P<kind>debited|credited)'
    r'.*? on (?P<date>\d\d-\d\d-\d\d)\.'
    r'(?: Ref (?P<ref>\d+)\.)?'
)
print('finditer, one dict per message:')
for m in RECORD.finditer(TEXT):
    print(' ', m.groupdict())

# A token scanner: one alternation of named groups, and lastgroup says which one matched.
TOKEN = re.compile(
    r'(?P<amount>Rs\.? ?[\d,]+(?:\.\d\d)?)'
    r'|(?P<date>\d\d-\d\d-\d\d)'
    r'|(?P<kind>debited|credited)'
)
line = 'On 11-03-31 A/c XX4821 was credited with Rs.2,000'
print('token scanner, fields in any order:')
for m in TOKEN.finditer(line):
    print(f'  {m.lastgroup:<6} {m[0]!r} at {m.start()}')

zip of two findall lists:
   1,250.00  ref 7731
  18,000.00  ref 7790
finditer, one dict per message:
  {'amount': '1,250.00', 'kind': 'debited', 'date': '05-03-31', 'ref': '7731'}
  {'amount': '18,000.00', 'kind': 'credited', 'date': '07-03-31', 'ref': None}
  {'amount': '99.50', 'kind': 'debited', 'date': '09-03-31', 'ref': '7790'}
token scanner, fields in any order:
  date   '11-03-31' at 3
  kind   'credited' at 27
  amount 'Rs.2,000' at 41

The second message has no reference, so the list of references is one item short and every pair after the first is wrong: the 18,000.00 credit got 7790, which belongs to the 99.50 debit, and zip dropped that debit because it stops at the shorter list. One call per field costs twice: the text is read again for every field, and the fields of one message stay together only while every message has all of them.

The cure is one pattern for a whole message, with a named groupnamed group: A capturing group that carries a name as well as a number, so the match reads back by name instead of by slot: (?<year>\d{4}) in JavaScript, PCRE2, Go, PHP, Java and .NET; (?P<year>\d{4}) in Python, Go, PHP and PCRE2. More in “Named groups across regex flavours” for each field, and one scan with re.finditer. Each match it yields is one message, so its fields cannot drift apart, and groupdict() turns the match into a dict. The reference is an optional group, (?: Ref (?P<ref>\d+)\.)?, so the credit comes back with 'ref': None: an unmatched groupunmatched group: A capturing group that took no part in the match, because its branch of an alternation was not used or its repetition ran zero times. Engines report it as undefined, None, null or the empty string. More in “Capturing and non-capturing groups” stays inside its own record instead of moving every record after it. A loop like this is a scannerscanner: Code that reads a text from left to right with one pattern and handles each match in turn, as a loop over re.finditer does. A tokenizer is a scanner whose alternatives name the kind of each token.. It reads the text from left to right with one pattern and handles each match as it comes, and since re.finditer returns an iterator, the matches of a long text are never collected in one list. pandas makes the same move for a whole column of texts: its Series.str.extract turns each named group into a column of a DataFrame (pandas API reference).

When the fields of a message can come in any order, scan tokens instead of records. The last part of the run joins three named alternatives into one pattern. For each match, m.lastgroup names the alternative that matched, so the loop knows whether it read a date, a kind or an amount, and m.start() says where. The tokenizer in the re documentation is built the same way (Writing a Tokenizer).

A function can compute each replacement

re.sub(r'\d+', lambda m: str(int(m[0]) * 2), 'a3 b4') returns a6 b8: each number came back doubled, which no fixed replacement stringreplacement string: The new text given to a replace function. It has its own escapes, so a dollar sign or a backslash in it may name part of the match instead of standing for itself. More in “Writing patterns in code: strings and escaping” could do. When the replacement is a function, re.sub calls it once for every match, from left to right, hands it the match object and puts the string it returns where the match was (re.sub). Text between the matches is copied unchanged, and a backslash in the returned string stays a backslash, since only a replacement given as a string is read as a template.

A job that needs this: amounts written with grouping commas, turned into a whole number of the currency’s smallest unit, so that no floating-point rounding can creep in. The program below writes out as a loop what re.sub does with a function, then calls re.sub itself. Step through it and watch out grow by one piece of copied text and one return value of to_paise for each match:

import re

DIGITS = r'(\d+(?:,\d+)*)'
PAISE = r'(?:\.(\d\d))?'
AMOUNT = re.compile(DIGITS + PAISE)

def to_paise(m):
    whole = m[1].replace(',', '')
    frac = m[2] or '00'
    return str(int(whole + frac))

text = 'Rs 1,23,456.50 and Rs 99'
out = []
end = 0
for m in AMOUNT.finditer(text):
    found = m[0]
    out.append(text[end:m.start()])
    out.append(to_paise(m))
    end = m.end()
out.append(text[end:])
print(''.join(out))
print(AMOUNT.sub(to_paise, text))

Rs 12345650 and Rs 9900
Rs 12345650 and Rs 9900

Both lines print Rs 12345650 and Rs 9900. For each match, to_paise read the two groups, the whole amount with its commas and the optional two digits after the point, and joined them as text before a single int(). Press Edit, change text to 'Rs 1,23,456.50 and Rs 99 and Rs 7.05' and run it: the third amount comes out as 705.

The function has one rule to keep: it must return a str. The next run breaks that rule on purpose, and its table shows why the digits travel as text and not as a float:

"""re.sub with a function: it is called with each match and returns the text that replaces it."""

import re

# The function's return value goes where the match was.
print(re.sub(r'\d+', lambda m: str(int(m[0]) * 2), 'a3 b4'))

# An amount through float() can lose a paisa; the same digits joined as text cannot.
for amount in ['19.99', '2.30', '1,23,456.29']:
    whole, _, frac = amount.replace(',', '').partition('.')
    via_float = int(float(f'{whole}.{frac}') * 100)
    via_digits = int(whole + frac.ljust(2, '0'))
    print(f'{amount:>12}  float {via_float:>9}  digits {via_digits:>9}')

# The function must return a str: an int stops the whole call.
try:
    re.sub(r'\d+', lambda m: int(m[0]) * 2, 'a3 b4')
except TypeError as e:
    print('TypeError:', e)

a6 b8
       19.99  float      1998  digits      1999
        2.30  float       229  digits       230
 1,23,456.29  float  12345629  digits  12345629
TypeError: sequence item 1: expected str instance, int found

An int stops the whole call with TypeError: sequence item 1: expected str instance, int found. The message comes from the join at the end of re.sub: item 0 is the a copied from before the first match, and item 1 is the first replacement. The table is the money trap: int(float('19.99') * 100) gives 1998, one short, because most decimal fractions have no exact binary form (Python tutorial, floating-point arithmetic), while joining the digits gives 1999. The float happens to be right for 1,23,456.29, which is how such a bug survives a test with a few round amounts.

Everything the function needs comes with the match: m[0] is the whole match, m[1] or m['name'] a group, and m.start() its position, so a replacement can depend on any of them.

Split keeps what its groups capture, and empty matches count

re.split(r'(\d)', 'a1b2c') returns ['a', '1', 'b', '2', 'c']: the digits it split on are in the list too, each between the two pieces it separated. Python adds the text of every capturing groupgroup: A part of a pattern wrapped in parentheses whose matched text the engine remembers separately, for reuse or extraction. More in “What a regular expression is and where it runs” of the separator to the result (re.split). With one group around the whole separator, ''.join() of the list gives the original text back, as the first line shows, which makes split a lossless way to cut a text apart, change some of its pieces and put it back together.

"""re.split keeps what a capturing group matched, and re.sub replaces empty matches too."""

import re

# One group around the separator: the separators stay in the list, in their places.
parts = re.split(r'(\d)', 'a1b2c')
print(parts, '->', ''.join(parts))
print(re.split(r'\s*([+*/-])\s*', '12 + 3*4 - 5'))

# Two groups: every split adds one item per group, None for a group that took no part.
print(re.split(r'(-)|(\+)', 'a-b+c'))

# maxsplit, given by keyword: the rest of the text stays in the last item.
print(re.split(r'(\d)', 'a1b2c', maxsplit=1))

# a* also matches the empty string: where it matches, and what sub and split do there.
print([m.span() for m in re.finditer(r'a*', 'baac')])
print(re.sub(r'a*', '-', 'baac'))
print(re.split(r'a*', 'baac'))

['a', '1', 'b', '2', 'c'] -> a1b2c
['12', '+', '3', '*', '4', '-', '5']
['a', '-', None, 'b', None, '+', 'c']
['a', '1', 'b2c']
[(0, 0), (1, 3), (3, 3), (4, 4)]
-b--c-
['', 'b', '', 'c', '']

The second line keeps the operators of an arithmetic expression and drops the spaces around them, because the spaces sit outside the group. The third shows the price of two groups: every split adds one item per group, and the group that took no part adds None, so (-)|(\+) leaves two None items in the list and ''.join() of it would fail. The fourth line stops after one split, maxsplit=1, and leaves the rest of the text in the last item.

The last three lines are about the empty matchempty match: A match of length zero: the pattern succeeds at a position without taking any character, as x* does where no x stands. Global searches, splits and replacements report such matches, each engine by its own rules. More in “Repetition with *, +, ? and {m,n}”. a* matches the empty string wherever no a stands, and the spans show four matches in baac: at 0, from 1 to 3, at 3 and at 4. An empty match may come right after a non-empty one (re.sub), which is where the span (3, 3) comes from:

So re.sub(r'a*', '-', 'baac') is -b--c-, and re.split with the same pattern gives five items, three of them empty. JavaScript’s replace gives the same -b--c-, while Go’s regexp ignores an empty match that touches the match before it (package regexp) and gives -b-c-. The porting lesson covers moving such a pattern between engines.

Compile once, and give count, maxsplit and flags by name

re.sub('x', '-', 'XxXx', re.I) returns X-X-, not ----. The fourth parameter of re.sub is count, and re.I is the number 2, so the call replaced the first two lowercase x and did not ignore case at all. Python 3.13 deprecated passing count, maxsplit or flags to re.sub, re.subn and re.split by position (What is new in Python 3.13), so the call now warns as well. The run below collects its warnings with warnings.catch_warnings(record=True) and prints them, so it ends normally:

"""Python 3.13 and later warn when count, maxsplit or flags reach re.sub, re.subn or re.split
by position. The warnings are caught and printed here, so the run still ends normally."""

import re
import warnings


def call(label, fn):
    with warnings.catch_warnings(record=True) as caught:
        warnings.simplefilter('always')
        result = fn()
    print(f'{label} -> {result!r}')
    for w in caught:
        print(f'    {w.category.__name__}: {w.message}')


call("re.sub('x', '-', 'XxXx', re.I)", lambda: re.sub('x', '-', 'XxXx', re.I))
call("re.sub('x', '-', 'XxXx', flags=re.I)", lambda: re.sub('x', '-', 'XxXx', flags=re.I))
call("re.split(',', 'a,b,c', 1)", lambda: re.split(',', 'a,b,c', 1))
call("re.split(',', 'a,b,c', maxsplit=1)", lambda: re.split(',', 'a,b,c', maxsplit=1))
call("re.compile('x', re.I).sub('-', 'XxXx', 1)", lambda: re.compile('x', re.I).sub('-', 'XxXx', 1))
print('re.I is the number', int(re.I))

# Compiled once, used for every value: match anchors only the start, fullmatch the whole value.
CODE = re.compile(r'\d{6}')
CODE_DOLLAR = re.compile(r'\d{6}$')
for value in ['400001', '4000012', '400001\n']:
    print(f'{value!r:<11} match {bool(CODE.match(value))!s:<5}  '
          f'match with $ {bool(CODE_DOLLAR.match(value))!s:<5}  '
          f'fullmatch {bool(CODE.fullmatch(value))}')

re.sub('x', '-', 'XxXx', re.I) -> 'X-X-'
    DeprecationWarning: 'count' is passed as positional argument
re.sub('x', '-', 'XxXx', flags=re.I) -> '----'
re.split(',', 'a,b,c', 1) -> ['a', 'b,c']
    DeprecationWarning: 'maxsplit' is passed as positional argument
re.split(',', 'a,b,c', maxsplit=1) -> ['a', 'b,c']
re.compile('x', re.I).sub('-', 'XxXx', 1) -> '-xXx'
re.I is the number 2
'400001'    match True   match with $ True   fullmatch True
'4000012'   match True   match with $ False  fullmatch False
'400001\n'  match True   match with $ True   fullmatch False

Each warning names the parameter that came by position, as in 'count' is passed as positional argument. Written with flags=re.I and maxsplit=1, the same calls run silently and say what they mean. A compiled patterncompiled pattern: The object that re.compile returns, a re.Pattern: the pattern parsed once with its flags fixed, so its search, finditer, sub and fullmatch methods take the text but no flags argument. has no such trap: its flags were fixed by re.compile, its sub method has no flags parameter, and the run shows it taking a count by position without a warning.

Compile a pattern once, at the top of the module, as soon as more than one place uses it. The module-level functions also keep their recently compiled patterns in a cache (re.compile), so compiling the same pattern again costs little; what compiling buys is a name that says what the pattern is for, flags written in one place, and methods that take a start and an end position as well.

For validation, the call matters as much as the pattern. The last three lines run a six-digit check three ways. match anchors only at the start, so it accepts 4000012. A $ at the end refuses that value but still accepts 400001 with a line feed after it, the trailing-newline traptrailing-newline trap: A validation bug: a pattern such as ^abc$ accepts abc followed by a line feed in engines whose $ also holds before a final newline, so the checked value still carries the extra character. More in “Anchors: start, end and multiline mode”, because $ also matches before a final newline. fullmatch asks for a full matchfull match: A match that must cover the whole text, from its first character to its last. Python's re.fullmatch asks for it; match and search do not. More in “How an engine finds a match, step by step” and refuses both.

What re cannot do, and when the regex module earns its place

Ask re itself what it lacks. The first three lines below print every parameter of re.search, re.sub and re.finditer: a pattern, a string, the replacement and count of sub, flags, and nothing that limits time. The next three compile escapes that other engines know, and re refuses each one with PatternError: bad escape:

"""What Python 3.14's re module accepts and refuses, asked of re itself."""

import inspect
import re

# No timeout: these are every parameter the matching functions take.
for name in ('search', 'sub', 'finditer'):
    print(f're.{name}{inspect.signature(getattr(re, name))}')

# Escapes of other engines that re refuses.
for pattern in (r'\p{L}', r'\X', r'\Ga'):
    try:
        re.compile(pattern)
    except re.PatternError as e:
        print(f'{pattern:<6} {type(e).__name__}: {e}')

# Syntax that re accepts: atomic groups and possessive quantifiers (3.11),
# a missing lower bound, and \z (3.14).
print('(?>a+)b on aab:', bool(re.fullmatch(r'(?>a+)b', 'aab')))
print('a++a on aaa:', bool(re.fullmatch(r'a++a', 'aaa')))
print('a{,2} on aaa:', re.findall(r'a{,2}', 'aaa'))
print(r'\d$ on "42\n":', bool(re.search(r'\d$', '42\n')))
print(r'\d\z on "42\n":', bool(re.search(r'\d\z', '42\n')))

re.search(pattern, string, flags=0)
re.sub(pattern, repl, string, count=0, flags=0)
re.finditer(pattern, string, flags=0)
\p{L}  PatternError: bad escape \p at position 0
\X     PatternError: bad escape \X at position 0
\Ga    PatternError: bad escape \G at position 0
(?>a+)b on aab: True
a++a on aaa: False
a{,2} on aaa: ['aa', 'a', '']
\d$ on "42\n": True
\d\z on "42\n": False

With no match timeoutmatch timeout: A limit on how long one matching call may run, after which the engine gives up with an error instead of an answer. Python's re has none; the third-party regex module takes timeout= in seconds., a slow match runs until it ends, and nothing outside the pattern stops it. For a pattern you write yourself, the fix is the pattern, which the track’s module on performance, ReDoS and security takes up. \p{L} and the other Unicode property escapesUnicode property escape: A pattern escape such as \p{L} (any letter) or \p{Nd} (any decimal digit) that matches characters by a Unicode property instead of a list; \P matches the rest. JavaScript needs the u or v flag for it, and Python's re module lacks it. More in “Shorthand classes for digits, words and spaces” are missing too; the shorthand classes lesson builds such a class from the unicodedata module instead. There is no \X for one user-perceived character, and \G is refused as well.

The last five lines show the other side: syntax that re accepts, three pieces of it added in recent versions. An atomic groupatomic group: A group written (?>...) that keeps the first match its own pattern finds: a failure later in the pattern cannot backtrack into it to make it try another way. x++ means the same as (?>x+). More in “Possessive quantifiers and atomic groups”, (?>a+), and a possessive quantifierpossessive quantifier: A quantifier with an extra plus sign, such as x*+, x++, x?+ or x{m,n}+, that matches as much as a greedy one and then keeps it: it saves no shorter run, so a failure later in the pattern never makes it give characters back. More in “Possessive quantifiers and atomic groups”, a++, are accepted, so a++a cannot match aaa: a++ keeps all three letters and gives none back. A repetition boundrepetition bound: The lowest or the highest count a quantifier allows, such as 3 and 5 in {3,5}. A missing highest bound, as in {3,}, means no limit, though each engine caps how large a written count may be. More in “Repetition with *, +, ? and {m,n}” without its lower number, a{,2}, means a{0,2}, and \z matches only at the very end, where $ also takes a final line feed (re, regular expression syntax).

The third-party regex module is the usual answer when these limits bite. Its README calls it backwards-compatible with re, and it adds each thing re lacks: a timeout= argument, in seconds, on its matching functions, which raises TimeoutError when the time runs out; \p{…} for general categories, scripts and blocks; \X for one graphemeWhat a reader sees as one character, which may be several code points: a Hindi letter with its vowel sign is one grapheme made of two code points., following Unicode’s rules for text boundaries; recursive patterns such as (?R); and fuzzy matchingfuzzy match: A match that allows a set number of wrong characters, each one inserted, deleted or substituted. The third-party regex module for Python writes the limit after a group, as {e<=1}; re only matches exactly., where {e<=1} after a group lets the text differ by one inserted, deleted or substituted character (mrab-regex README). It is not part of Python. A project that wants it adds it from PyPI as a dependency, pins a version and keeps it updated, and the PyPI page lists Python 3.10 or later; no example on this page imports it.

Whether that dependency pays for itself depends on your project more than on the module. Answer the four questions for a job you have in mind and see which ways stay open:

In short: stay on re for patterns you write, with classes built from unicodedata where categories are needed; add regex when users write the patterns, when graphemes, scripts or misspellings matter, and the project can take a dependency; and leave regular expressions out when users mean plain text or the input is a format with a parser of its own.

Practise: parse a statement of transactions

The exercise puts the lesson into one function. parse_transactions(text) reads messages in five formats made up for it, finds each message’s date, amount and direction, and returns one dict per transaction, with the date in ISO form and the amount as a whole number of paise. The starter knows one format and converts through a float, and the sample tests show where both choices break.

{"date": "2031-03-05", "amount": 125000, "type": "debit"}

Rs 1,250.00 debited from A/c XX4821 on 05-03-31. Ref 7731.
INR 18,000 credited to A/c XX4821 on 07/03/2031.
A/c XX4821 debited by Rs.99.50 on 09-Mar-31.
Received Rs 2,45,000.75 in A/c XX4821 on 2031-03-12.
Spent Rs 19.99 at a store on 13 Mar 2031 from A/c XX4821.

"""parse_transactions(text): one dict per transaction message of the text, in order.

The prompt describes the exercise's message format: five ways to write the date, three currency markers, four words
for the type. This starter reads only the first format, and it converts the amount through float().
"""

import re

# Starter: the first format only, 'Rs 1,250.00 debited from A/c XX4821 on 05-03-31. Ref 7731.'
FIRST_FORMAT = re.compile(
    r'Rs (?P<amount>[\d,]+\.\d\d) (?P<kind>debited|credited)'
    r'.*? on (?P<day>\d\d)-(?P<month>\d\d)-(?P<year>\d\d)'
)


def parse_transactions(text):
    """One dict per transaction line: {'date': 'YYYY-MM-DD', 'amount': paise as an int, 'type': 'debit' or 'credit'}."""
    found = []
    for m in FIRST_FORMAT.finditer(text):
        found.append({
            'date': f"20{m['year']}-{m['month']}-{m['day']}",
            'amount': int(float(m['amount'].replace(',', '')) * 100),
            'type': 'debit' if m['kind'] == 'debited' else 'credit',
        })
    return found

from transactions import parse_transactions


def test_first_format():
    """a debit with grouping commas and a two-digit year"""
    text = 'Rs 1,250.00 debited from A/c XX4821 on 05-03-31. Ref 7731.'
    assert parse_transactions(text) == [{'date': '2031-03-05', 'amount': 125000, 'type': 'debit'}]


def test_credit_without_decimals():
    """INR, no decimal point, a slash date with a four-digit year"""
    text = 'INR 18,000 credited to A/c XX4821 on 07/03/2031.'
    assert parse_transactions(text) == [{'date': '2031-03-07', 'amount': 1800000, 'type': 'credit'}]


def test_month_names():
    """Rs. without a space, and three-letter months in two date forms"""
    text = 'A/c XX4821 debited by Rs.99.50 on 09-Mar-31.\nSpent Rs 450 at a store on 13 MAR 2031 from A/c XX4821.'
    assert parse_transactions(text) == [
        {'date': '2031-03-09', 'amount': 9950, 'type': 'debit'},
        {'date': '2031-03-13', 'amount': 45000, 'type': 'debit'},
    ]


def test_received_and_iso_date():
    """received counts as a credit, Indian grouping, an ISO date"""
    text = 'Received Rs 2,45,000.75 in A/c XX4821 on 2031-03-12.'
    assert parse_transactions(text) == [{'date': '2031-03-12', 'amount': 24500075, 'type': 'credit'}]


def test_amounts_stay_exact():
    """no float on the way: 19.99 is 1999 paise, and one decimal digit means tens of paise"""
    text = 'Rs 19.99 debited from A/c XX4821 on 13-03-31.\nRs 2.5 credited to A/c XX4821 on 14-03-31.'
    assert [t['amount'] for t in parse_transactions(text)] == [1999, 250]


def test_other_numbers_are_not_amounts():
    """the account number and the reference are never the amount"""
    text = 'A/c XX4821 debited by INR 300 on 15/03/2031. Ref 4500.'
    assert parse_transactions(text) == [{'date': '2031-03-15', 'amount': 30000, 'type': 'debit'}]


def test_lines_that_are_not_transactions():
    """a balance line and an empty line are skipped, and the order of the rest is kept"""
    text = (
        'Rs 99.50 debited from A/c XX4821 on 09-03-31. Ref 7790.\n'
        'Your balance in A/c XX4821 is Rs 5,000.00.\n'
        '\n'
        'INR 1,000 credited to A/c XX4821 on 10/03/2031.\n'
    )
    assert parse_transactions(text) == [
        {'date': '2031-03-09', 'amount': 9950, 'type': 'debit'},
        {'date': '2031-03-10', 'amount': 100000, 'type': 'credit'},
    ]


def test_empty_text():
    """an empty text has no transactions"""
    assert parse_transactions('') == []

Five ideas, one round of practice. It opens on the idea your answers show you know least, and each choice tells you why it is right or wrong as soon as you choose it.

Interview questions

Warm-up: what does re.sub do when its replacement is a function and not a string?

It calls the function once for every match, from left to right, and passes it the match object; the string the function returns takes the place of that match, and the text between matches is copied unchanged. So the replacement can be computed from what was matched: m[0] is the whole match, m['name'] a named group and m.start() its position. Doubling every number, re.sub(r'\d+', lambda m: str(int(m[0]) * 2), 'a3 b4'), gives a6 b8. Two rules come with it. The function must return a str, since an int stops the call with a TypeError. And the returned text is used as it is: a template escape such as \g<1> means something only in a replacement string.

Two harder questions build on the scanner and on the choice of the last section. Sketch an answer to each before you open its first step:

Key takeaways

• One pattern per record, scanned once with re.finditer, keeps every record’s fields together, while one findall per field pairs the fields only by position and shifts them all when a single record lacks one.
• A function passed to re.sub runs once per match, left to right, and the str it returns takes the match’s place, which is how an amount with grouping commas becomes an exact whole number of its smallest unit without a float.
• A capturing group in a re.split pattern keeps each separator in the result, a group that took no part adds None, and an empty match counts as a match, so a* turns baac into -b--c-.
• Since Python 3.13, count, maxsplit and flags go by keyword; a pattern used in several places is compiled once, and fullmatch is the call that checks a whole value.
• re has no timeout, no \p{…} and no \X; the third-party regex module adds them, with fuzzy matching, and adding it is a dependency decision your project makes.