Regex literals and RegexBuilder

The alert “Debited 1,250.00 from account XX4321. Status: SUCCESS” holds three facts, and one regex literal pulls all three out as named values. Written carelessly, the same tool tried 2,097,151 times to match twenty digits and an x before it gave up, where a better pattern needed 21 tries.

Here you will match and capture text with regex literals, write the same patterns as Swift code with RegexBuilder and turn captures into numbers, search, validate, replace and split strings with a Regex, find and fix a pattern that backtracks without end, and build Regex code for the macOS versions that have it.

A regex literal is a pattern the compiler checks

A regular expressionregular expression: A pattern that describes the text it matches, such as [A-Z]{3}\d{6} for three capital letters and six digits. Swift's Regex type matches one against a string and returns the parts it captured. describes the text it matches: \d is a digit, [A-Z]+ one or more capital letters, X+\d{4} one or more X followed by four digits. In Swift it is written as a literal between #/ and /#, and each capturecapture: A part of a regex in brackets, (…) or (?<name>…), whose matched text is returned in the match's output. Swift types it as a Substring, or as the value a TryCapture transform returns. (?<name>…) becomes a named part of the result. This program reads an alert:

// alert.swift: one regex literal pulls three facts out of an alert.
let alert = "Debited 1,250.00 from account XX4321. Status: SUCCESS"
let pattern = #/(?<amount>\d[\d,]*\.\d\d) from account (?<account>X+\d{4})\. Status: (?<status>[A-Z]+)/#

if let match = alert.firstMatch(of: pattern) {
    print("amount:", match.output.amount)
    print("account:", match.output.account)
    print("status:", match.output.status)
    print("whole match:", match.output.0)
}
print(type(of: pattern))
let other = "Credited 99.00 to account XX9876"
print("another alert:", other.firstMatch(of: pattern) == nil ? "firstMatch returned nil" : "matched")

amount: 1,250.00
account: XX4321
status: SUCCESS
whole match: 1,250.00 from account XX4321. Status: SUCCESS
Regex<(Substring, amount: Substring, account: Substring, status: Substring)>
another alert: firstMatch returned nil

firstMatch(of:) returns an optional: the first match in the text, or nil, as for the other alert. The match’s output is a tuple whose types the compiler worked out from the pattern itself: the whole match first, then amount, account and status, each a Substring of the alert. Step through how the pattern lines up with the text:

Because the compiler reads the pattern, a broken one is a compile error, not a surprise at run time. The other form of the literal, between plain slashes, /\d+/, needs the Swift 6 language mode (SE-0354); in Swift 5 mode, the default of a plain swiftc, the compiler reads the slashes as operators, and the extended form #/…/# is the one that works. This session builds the two files below:

// bad_pattern.swift: a regex literal with a bracket that is never closed.
let reference = #/Ref (\d+/#
print("Ref 4417".contains(reference))

// bare.swift: a regex literal between plain slashes.
let code = "Ref 4417"
let digits = /\d+/
print(code.contains(digits))

from session import run, version

# bad_pattern.swift has an unclosed bracket; bare.swift uses /…/, which needs the Swift 6 language mode.
version()
run("""
!swiftc bad_pattern.swift -o bad_pattern
swiftc -swift-version 6 bare.swift -o bare
./bare
!swiftc bare.swift -o bare
""", max_lines=8)

# Apple Swift version 6.4 (swiftlang-6.4.0.34.1)
$ swiftc bad_pattern.swift -o bad_pattern
bad_pattern.swift:2:27: error: cannot parse regular expression: expected ')'
1 | // bad_pattern.swift: a regex literal with a bracket that is never closed.
2 | let reference = #/Ref (\d+/#
  |                           `- error: cannot parse regular expression: expected ')'
3 | print("Ref 4417".contains(reference))
4 | 
[exit status 1]
$ swiftc -swift-version 6 bare.swift -o bare
$ ./bare
true
$ swiftc bare.swift -o bare
bare.swift:3:14: error: '/' is not a prefix unary operator
1 | // bare.swift: a regex literal between plain slashes.
2 | let code = "Ref 4417"
3 | let digits = /\d+/
  |              `- error: '/' is not a prefix unary operator
4 | print(code.contains(digits))
5 | 

[23 more lines]
[exit status 1]

The first build stopped at the pattern with “expected ‘)’”; bare.swift compiled and printed true in Swift 6 mode, and failed in Swift 5 mode. A package whose manifest says swift-tools-version: 6.0 builds its targets in Swift 6 mode unless the package or a target asks for another, as the Swift 6 migration guide states, so both forms work there.

RegexBuilder spells a pattern as Swift code

A pattern of many symbols is hard to read and harder to change. RegexBuilder (SE-0351) writes the same pattern as Swift values: Repeat(letter, count: 4) for [A-Z]{4}, Capture { … } for a bracket, ChoiceOf { … } for alternatives, OneOrMore, Optionally and CharacterClass for the rest. This program checks codes shaped like an IFSC, which names a bank branch in India’s NEFT payments: four letters for the bank, a zero, then six letters or digits for the branch, as the Reserve Bank of India describes it. The codes in the program are made up. Then it reads debits:

// builder.swift: patterns written as Swift code with RegexBuilder, with a typed capture.
import RegexBuilder

let letter = CharacterClass("A"..."Z")
let letterOrDigit = CharacterClass("A"..."Z", "0"..."9")
let branchCode = Regex {
    Capture { Repeat(letter, count: 4) }
    "0"
    Capture { Repeat(letterOrDigit, count: 6) }
}
for code in ["ABCB0001234", "XYZB0A12B3C", "ABCB1234567", "abcb0001234", "ABCB00012345"] {
    if let match = code.wholeMatch(of: branchCode) {
        print(code, "->", match.1, match.2)
    } else {
        print(code, "-> no match")
    }
}

let debit = Regex {
    ChoiceOf {
        "Debited "
        "Debit of "
    }
    TryCapture {
        OneOrMore(CharacterClass(.digit, .anyOf(",")))
        "."
        Repeat(.digit, count: 2)
    } transform: { Int($0.filter(\.isNumber)) }
}
for text in ["Debited 1,250.00 from XX4321", "Debit of 99.00 from XX4321 failed", "Debited one rupee"] {
    if let match = text.firstMatch(of: debit) {
        print("paise:", match.1, "a value of type", type(of: match.1))
    } else {
        print("no debit in:", text)
    }
}

ABCB0001234 -> ABCB 001234
XYZB0A12B3C -> XYZB A12B3C
ABCB1234567 -> no match
abcb0001234 -> no match
ABCB00012345 -> no match
paise: 125000 a value of type Int
paise: 9900 a value of type Int
no debit in: Debited one rupee

Step through how each builder component matches a part of the literal syntax:

TryCapture captures and converts in one step: its transform closure turned “1,250.00” into the Int 125000, the amount in paise. When a transform returns nil, the engine backtracks and tries another way, and the match fails when no way is left, so a capture that is not a number never gets through. The match’s .1 is an Int, not a Substring. The lower-case code failed the letter class, the code without its zero failed the "0", and the code with one character too many failed because wholeMatch must cover the whole string, as the next section shows.

String methods take a Regex, and wholeMatch validates

The string methods of SE-0357 take a Regex: contains, ranges, replacing, split and trimmingPrefix take one where they also take plain text, and firstMatch, wholeMatch, prefixMatch and matches return the matches themselves. This program uses them on booking references and a message about cards:

// methods.swift: String methods that take a Regex.
let reference = #/[A-Z]{3}\d{6}/#
for text in ["PNR442177", "pnr442177", "PNR442177 and more", "Ref PNR442177"] {
    let whole = text.wholeMatch(of: reference) != nil
    let first = text.firstMatch(of: reference)?.output ?? "none"
    print(text, "| wholeMatch:", whole, "| firstMatch:", first)
}

let message = "Card XX4321 paid 450.75; card XX9876 refunded 99.00"
print(message.replacing(#/XX(\d{4})/#) { match in "account ending \(match.1)" })
print(message.matches(of: #/\d+\.\d\d/#).map(\.output))
print(message.contains(#/refunded/#), message.ranges(of: #/XX\d{4}/#).count)
print("Asha ,Ravi,  Meera".split(separator: #/\s*,\s*/#))
print("Ref: 4417".trimmingPrefix(#/Ref:\s*/#))

PNR442177 | wholeMatch: true | firstMatch: PNR442177
pnr442177 | wholeMatch: false | firstMatch: none
PNR442177 and more | wholeMatch: false | firstMatch: PNR442177
Ref PNR442177 | wholeMatch: false | firstMatch: PNR442177
Card account ending 4321 paid 450.75; card account ending 9876 refunded 99.00
["450.75", "99.00"]
true 2
["Asha", "Ravi", "Meera"]
4417

firstMatch found PNR442177 inside “Ref PNR442177” and inside “PNR442177 and more”, so it is the wrong test for “is this a booking reference?”. wholeMatch succeeded only when the pattern covered the whole string. OWASP’s Input Validation Cheat Sheet asks for exactly that when a regex validates a field: require a match of the entire value. Step through the four references:

replacing took a closure that builds each replacement from the match, so both card numbers became “account ending” and their last four digits. matches(of:) returned every amount, split cut at commas with any spaces around them, and trimmingPrefix removed a label from the front.

Nested quantifiers can try every split of the input

Swift’s Regex is a backtrackingbacktracking: What a regex engine does when part of a pattern fails: it returns to an earlier choice, such as how many characters a + took, and tries another. Nested quantifiers can make the choices grow exponentially. engine: when part of a pattern fails, it goes back to an earlier choice, such as how many digits a + took, and tries another. In ^(\d+)+$, the inner \d+ and the outer + can share out a row of digits in every possible way, and when the text ends in something that is not a digit, every way fails and every way is tried. This program counts the single-digit tries of three patterns, with a component that adds one to a tally each time the engine asks it for a digit:

// backtracking.swift: how many single-digit tries three patterns make on digits followed by an x.
import RegexBuilder

final class Tally {
    var tries = 0
}

/// Matches one ASCII digit, and counts every time the regex engine asks it to.
struct CountedDigit: CustomConsumingRegexComponent {
    typealias RegexOutput = Substring
    let tally: Tally

    func consuming(
        _ input: String, startingAt index: String.Index, in bounds: Range<String.Index>
    ) throws -> (upperBound: String.Index, output: Substring)? {
        tally.tries += 1
        guard index < bounds.upperBound, input[index].isASCII, input[index].isNumber else {
            return nil
        }
        let next = input.index(after: index)
        return (next, input[index..<next])
    }
}

for length in [3, 5, 10, 15, 20] {
    let text = String(repeating: "1", count: length) + "x"
    let nestedTally = Tally()
    let flatTally = Tally()
    let atomicTally = Tally()
    let nested = Regex { OneOrMore { OneOrMore { CountedDigit(tally: nestedTally) } } }  // like ^(\d+)+$
    let flat = Regex { OneOrMore { CountedDigit(tally: flatTally) } }  // like ^\d+$
    let atomic = Regex { OneOrMore { Local { OneOrMore { CountedDigit(tally: atomicTally) } } } }  // like ^(?>\d+)+$
    let matched = [text.wholeMatch(of: nested), text.wholeMatch(of: flat), text.wholeMatch(of: atomic)].contains { $0 != nil }
    print(
        "\(length) digits: nested \(nestedTally.tries) tries, flat \(flatTally.tries),",
        "atomic \(atomicTally.tries), any match: \(matched)")
}

3 digits: nested 15 tries, flat 4, atomic 5, any match: false
5 digits: nested 63 tries, flat 6, atomic 7, any match: false
10 digits: nested 2047 tries, flat 11, atomic 12, any match: false
15 digits: nested 65535 tries, flat 16, atomic 17, any match: false
20 digits: nested 2097151 tries, flat 21, atomic 22, any match: false

Each extra digit doubled the nested pattern’s work: 63 tries for 5 digits, 2,047 for 10 and 2,097,151 for 20, which is 2 to the power of (n + 1), minus 1. The flat pattern ^\d+$ needed n + 1 tries. Step through the four ways the nested pattern shares out three digits:

Two fixes keep the meaning and remove the explosion. Remove the nesting, as the flat pattern does, when the brackets add nothing. Or make the inner part an atomic groupatomic group: A part of a regex, (?>…) or RegexBuilder's Local, that keeps the first match it finds and drops its other choices, so the engine cannot retry it in other ways after a later failure.: (?>\d+) in a literal, Local in RegexBuilder, or a possessive quantifier such as \d++. An atomic group keeps the match it found and gives up its other choices, which made the third pattern linear: n + 2 tries.

Regex needs macOS 13, so set the platform you build for

Regex, its literals and RegexBuilder came with Swift 5.7, and on Apple platforms they need the operating system that ships that runtime: macOS 13, iOS 16 or later, as Apple’s documentation lists. A program built for an older deployment targetdeployment target: The oldest operating-system version a program is built to run on, set with swiftc -target or a package's platforms. Code that uses an API newer than the target, such as Regex below macOS 13, does not compile. does not compile. This session builds alert.swift for macOS 12 and for macOS 13, then tries a lookbehind in both language modes:

// lookbehind.swift: a lookbehind assertion, which Swift's Regex does not support yet.
let alert = "Balance: 4,500.00"
print(alert.firstMatch(of: #/(?<=Balance: )[\d,.]+/#)?.output ?? "none")

from session import run, version

# alert.swift built for macOS 12 and for macOS 13, then lookbehind.swift in the Swift 5 and Swift 6 language modes.
version()
run("""
!swiftc -target arm64-apple-macosx12.0 alert.swift -o alert
swiftc -target arm64-apple-macosx13.0 alert.swift -o alert
./alert
!swiftc lookbehind.swift -o lookbehind
!swiftc -swift-version 6 lookbehind.swift -o lookbehind
""", max_lines=8)

# Apple Swift version 6.4 (swiftlang-6.4.0.34.1)
$ swiftc -target arm64-apple-macosx12.0 alert.swift -o alert
alert.swift:3:15: error: 'Regex' is only available in macOS 13.0 or newer
 1 | // alert.swift: one regex literal pulls three facts out of an alert.
 2 | let alert = "Debited 1,250.00 from account XX4321. Status: SUCCESS"
 3 | let pattern = #/(?<amount>\d[\d,]*\.\d\d) from account (?<account>X+\d{4})\. Status: (?<status>[A-Z]+)/#
   |               |- error: 'Regex' is only available in macOS 13.0 or newer
   |               `- note: add 'if #available' version check
 4 | 
 5 | if let match = alert.firstMatch(of: pattern) {
[53 more lines]
[exit status 1]
$ swiftc -target arm64-apple-macosx13.0 alert.swift -o alert
$ ./alert
amount: 1,250.00
account: XX4321
status: SUCCESS
whole match: 1,250.00 from account XX4321. Status: SUCCESS
Regex<(Substring, amount: Substring, account: Substring, status: Substring)>
another alert: firstMatch returned nil
$ swiftc lookbehind.swift -o lookbehind
lookbehind.swift:3:30: error: cannot parse regular expression: lookbehind is not currently supported
1 | // lookbehind.swift: a lookbehind assertion, which Swift's Regex does not support yet.
2 | let alert = "Balance: 4,500.00"
3 | print(alert.firstMatch(of: #/(?<=Balance: )[\d,.]+/#)?.output ?? "none")
  |                              `- error: cannot parse regular expression: lookbehind is not currently supported
4 | 
[exit status 1]
$ swiftc -swift-version 6 lookbehind.swift -o lookbehind
lookbehind.swift:3:30: error: cannot parse regular expression: lookbehind is not currently supported
1 | // lookbehind.swift: a lookbehind assertion, which Swift's Regex does not support yet.
2 | let alert = "Balance: 4,500.00"
3 | print(alert.firstMatch(of: #/(?<=Balance: )[\d,.]+/#)?.output ?? "none")
  |                              `- error: cannot parse regular expression: lookbehind is not currently supported
4 | 
[exit status 1]

For macOS 12 the compiler stopped at the first Regex with “‘Regex’ is only available in macOS 13.0 or newer”, followed by more lines of the same kind; for macOS 13 the program built and ran. A Swift package that names no platforms gets SwiftPM’s predefined minimum version for each one, which for macOS is far older than 13, so a package that uses Regex declares platforms: [.macOS(.v13)] in Package.swift, or wraps the Regex code in if #available(macOS 13, *) with a fallback for older systems.

The lookbehind (?<=Balance: ) failed too, in both modes: lookbehind assertions are accepted for Swift (SE-0448) but not implemented in Swift 6.4, and the compiler says so. Match the context as plain text and capture only the part you want instead, as #/Balance: (?<amount>[\d,.]+)/# does. Walk through the versions:

Practise: alerts in four formats

Write parse(_:), which turns an alert in any of four formats into a transaction: debit or credit, the amount in paise as an Int, the masked account and whether it succeeded. The formats are “Debited … from account XX…. Status: SUCCESS”, “Credited … to account XX…. Status: SUCCESS”, “Debit of … from XX… failed” and “Refund of … credited to XX…”. Anything else returns nil, and so does text longer than 200 Characters, which the function refuses before matching. Then add up the successful debits.

Write and test yours before reading this version:

// practice.swift: parse alerts in four formats into transactions, each pattern matched against the whole text.
struct Transaction {
    let credit: Bool
    let paise: Int
    let account: String
    let ok: Bool
}

typealias AlertPattern = Regex<(Substring, amount: Substring, account: Substring)>

func parse(_ text: String) -> Transaction? {
    guard text.count <= 200 else {
        return nil
    }
    let formats: [(pattern: AlertPattern, credit: Bool, ok: Bool)] = [
        (#/Debited (?<amount>[\d,]+\.\d\d) from account (?<account>XX\d{4})\. Status: SUCCESS/#, false, true),
        (#/Credited (?<amount>[\d,]+\.\d\d) to account (?<account>XX\d{4})\. Status: SUCCESS/#, true, true),
        (#/Debit of (?<amount>[\d,]+\.\d\d) from (?<account>XX\d{4}) failed/#, false, false),
        (#/Refund of (?<amount>[\d,]+\.\d\d) credited to (?<account>XX\d{4})/#, true, true),
    ]
    for format in formats {
        if let match = text.wholeMatch(of: format.pattern), let paise = Int(match.output.amount.filter(\.isNumber)) {
            return Transaction(credit: format.credit, paise: paise, account: String(match.output.account), ok: format.ok)
        }
    }
    return nil
}

let alerts = [
    "Debited 1,250.00 from account XX4321. Status: SUCCESS",
    "Credited 15,000.00 to account XX9876. Status: SUCCESS",
    "Debit of 99.00 from XX4321 failed",
    "Refund of 450.75 credited to XX4321",
    "Debited 1,250.00 from account XX4321. Status: SUCCESS and more",
]
var spent = 0
for alert in alerts {
    guard let t = parse(alert) else {
        print("not recognised:", alert)
        continue
    }
    print(t.credit ? "credit" : "debit", t.paise, "paise", t.account, t.ok ? "ok" : "failed")
    if !t.credit && t.ok {
        spent += t.paise
    }
}
print("spent:", spent, "paise")

debit 125000 paise XX4321 ok
credit 1500000 paise XX9876 ok
debit 9900 paise XX4321 failed
credit 45075 paise XX4321 ok
not recognised: Debited 1,250.00 from account XX4321. Status: SUCCESS and more
spent: 125000 paise

Every pattern has the same named captures, so the four fit in one array of the same Regex type, and the function tries them in turn with wholeMatch. That refused the last alert, which has extra text after a valid one. The amount keeps only its digits, so “15,000.00” becomes 1500000 paise, and the account is copied into a String before it is stored. Close with the practice round. Questions on your shakiest topic come first, and a right answer earns a harder one:

Interview questions

Warm-up: what is the difference between firstMatch(of:) and wholeMatch(of:) in Swift?

firstMatch(of:) searches: it tries the pattern at each position of the string from the start and returns the first match it finds, or nil, so "Ref PNR442177".firstMatch(of: #/[A-Z]{3}\d{6}/#) finds PNR442177 in the middle. wholeMatch(of:) succeeds only when the pattern matches the entire string, from its first Character to its last, so the same call returns nil, because “Ref “ is in the way. Use firstMatch or matches(of:) to extract data from longer text, and wholeMatch to validate a value such as a booking reference or a code, since a validator that accepts a valid piece inside invalid text accepts invalid input. prefixMatch(of:) sits between them: the match must start at the beginning but may end anywhere. All three return a match whose output holds the typed captures.

Key takeaways

• A regex literal, #/…/# in any language mode or /…/ in Swift 6 mode, is checked by the compiler, and its named captures become typed parts of the match’s output.
• RegexBuilder writes patterns as Swift code with Capture, Repeat, ChoiceOf and CharacterClass, and TryCapture converts a capture, such as an amount into an Int, failing the match when it cannot.
• String methods take a Regex: firstMatch and matches find, replacing and split change and cut, and wholeMatch is the test for validating a whole value.
• Nested quantifiers such as (\d+)+ can make the work double with every character; flatten them or use an atomic group, and bound the length of untrusted input.
• Regex needs macOS 13 or iOS 16, so a package declares its platforms, and lookbehind is not available in Swift 6.4.