Regex, sigils and Unicode identifiers

A support tool must pull TEA-1 and 2 out of "order TEA-1 qty 2", accept the code "110001" and refuse "११०००१", six digits in Devanagari script that \d matches once Unicode is switched on. And a short pattern, ^(a+)+$, gave up on a text of 23 characters without saying so.

Below, you will match and extract text with Regex, see what the u modifier changes, keep untrusted input from making a pattern work without end, use the built-in sigilssigil: A tilde, a letter or an uppercase name, text between delimiters and optional modifiers, such as ~w(tea mug)a or ~r/\d+/. It is a call of a function named sigil_ and the letter, which receives the text and the modifiers. and write your own, and see how the compiler guards names against letters that only look alike. Erlang/OTP 29.1.1, the release these runs used with Elixir 1.20.4, bundles PCRE2 10.47, as :re.version/0 prints at the end of the first session.

A regex finds text by its shape, and captures the parts you name

An order number is letters, a hyphen and digits; a quantity is digits after the word qty. A regular expressionregular expression: A pattern that describes the shape of text, such as ~r/order (\w+-\d+)/ for an order number. Elixir's Regex module compiles it with PCRE2, through Erlang's :re, and finds, captures, replaces or splits on its matches. says that shape once, and Regex finds every place in a text that has it. The Regex documentation says the module “is based on PCRE (Perl Compatible Regular Expressions)” and “built on top of Erlang’s :re module”. regex.exs reads one line of an order log in six ways:

# One line of an order log, read six ways with regular expressions.
line = "order TEA-1 qty 2 at 09:30; order MUG-12 qty 1 at 10:05"

IO.inspect(Regex.match?(~r/qty \d+/, line), label: "match?")
IO.inspect(Regex.run(~r/order (\w+-\d+)/, line), label: "run")
IO.inspect(Regex.scan(~r/order (\w+-\d+) qty (\d+)/, line), label: "scan")
IO.inspect(Regex.named_captures(~r/order (?<sku>\w+-\d+) qty (?<qty>\d+)/, line), label: "named_captures")
IO.inspect(Regex.replace(~r/(\d\d):(\d\d)/, line, "\\1h\\2"), label: "replace")
IO.inspect(Regex.replace(~r/qty (\d+)/, line, fn _whole, n -> "qty #{String.to_integer(n) * 10}" end), label: "replace with a function")
IO.inspect(Regex.split(~r/;\s*/, line), label: "split")
IO.inspect(Regex.run(~r/\d+/, "price 450", return: :index), label: "run with return: :index")
IO.inspect(:re.version(), label: "the PCRE2 of Erlang/OTP")

from session import run

run("""
elixir regex.exs
""")

$ elixir regex.exs
match?: true
run: ["order TEA-1", "TEA-1"]
scan: [["order TEA-1 qty 2", "TEA-1", "2"], ["order MUG-12 qty 1", "MUG-12", "1"]]
named_captures: %{"qty" => "2", "sku" => "TEA-1"}
replace: "order TEA-1 qty 2 at 09h30; order MUG-12 qty 1 at 10h05"
replace with a function: "order TEA-1 qty 20 at 09:30; order MUG-12 qty 10 at 10:05"
split: ["order TEA-1 qty 2 at 09:30", "order MUG-12 qty 1 at 10:05"]
run with return: :index: [{6, 3}]
the PCRE2 of Erlang/OTP: "10.47 2025-10-21"

Step through the scan and watch where each piece of its result comes from:

replace/3 with "\\1h\\2" turned 09:30 into 09h30: in an Elixir string the backslash is written twice. With a function, each match was turned into a new quantity, ten times the old one. return: :index gave [{6, 3}], the byte position and length of 450 in "price 450", which binary_part/3 can cut out without copying the match.

The u modifier makes classes Unicode-aware, digits included

Without modifiers, a regex works on bytes and knows ASCII. The u modifier, in the documentation’s words, “causes character classes like \w, \W, \s, and the like to also match on Unicode”, and it makes \p{…} classes, such as \p{Lu} for capital letters, available. The session tries both on names and on digits:

# What the u modifier changes: letters, capitals, scripts and digits; then i and x.
IO.inspect(Regex.match?(~r/^\w+$/, "José"), label: "\\w+ without u")
IO.inspect(Regex.match?(~r/^\w+$/u, "José"), label: "\\w+ with u")
IO.inspect(Regex.scan(~r/\p{Lu}/u, "Asha, Élodie and Ömer"), label: "\\p{Lu}")
IO.inspect(Regex.match?(~r/^\p{Devanagari}+$/u, "नमस्ते"), label: "\\p{Devanagari}")
IO.inspect(Regex.match?(~r/^\d{6}$/, "११०००१"), label: "\\d{6} without u, Devanagari digits")
IO.inspect(Regex.match?(~r/^\d{6}$/u, "११०००१"), label: "\\d{6} with u, Devanagari digits")
IO.inspect(Regex.match?(~r/^[0-9]{6}$/u, "११०००१"), label: "[0-9]{6} with u, Devanagari digits")
IO.inspect(Regex.match?(~r/^[0-9]{6}$/, "110001\n"), label: "^…$ and a newline at the end")
IO.inspect(Regex.match?(~r/\A[0-9]{6}\z/, "110001\n"), label: "\\A…\\z and a newline at the end")
IO.inspect(Regex.match?(~r/^tea$/i, "TEA"), label: "i, caseless")

spaced = ~r/
  (?<hour>\d\d) : (?<minute>\d\d)   # x ignores these spaces and this comment
/x
IO.inspect(Regex.named_captures(spaced, "at 09:30"), label: "x, extended")

from session import run

run("""
elixir unicode.exs
""")

$ elixir unicode.exs
\w+ without u: false
\w+ with u: true
\p{Lu}: [["A"], ["É"], ["Ö"]]
\p{Devanagari}: true
\d{6} without u, Devanagari digits: false
\d{6} with u, Devanagari digits: true
[0-9]{6} with u, Devanagari digits: false
^…$ and a newline at the end: true
\A…\z and a newline at the end: false
i, caseless: true
x, extended: %{"hour" => "09", "minute" => "30"}

\w+ refused "José" without u and accepted it with u; \p{Lu} found the capitals A, É and Ö, and \p{Devanagari} matched नमस्ते. The digits are the trap. With u, \d{6} accepted "११०००१", six digits of the Devanagari script, because Unicode counts every script’s decimal digits as digits. A check that means “six digits from 0 to 9” must say [0-9]{6}, which refused them.

The next two lines show the second trap: ^[0-9]{6}$ accepted "110001\n", because $ also matches just before a newline at the end of the text. \A and \z mean the very start and the very end, so \A[0-9]{6}\z refused it. The last two lines show two other modifiers: i ignores case, and x ignores spaces and allows comments, which makes a long pattern readable.

Nested repeats can make a pattern give up without a word

^(a+)+$ looks like a longer way of writing ^a+$, and on a text of a’s alone the two agree. On a text of a’s that ends in a b, they do not: the nested repeat makes the engine try every way to divide the a’s into runs before it can say “no match”. The Erlang re documentation describes the case: “A common example is a pattern with nested unlimited repeats applied to a long string that does not match.” limits.exs runs both patterns on growing texts:

# A nested repeat, (a+)+, on text that almost matches, and a flat one, a+, on the same text.
{:ok, nested} = :re.compile("^(a+)+$")
{:ok, flat} = :re.compile("^a+$")
run = fn pattern, text, extra -> :re.run(text, pattern, [:report_errors, {:capture, :none} | extra]) end

for n <- [16, 18, 20, 21, 22, 24] do
  text = String.duplicate("a", n) <> "b"
  IO.puts("#{n} a's and a b: #{inspect(run.(nested, text, []))}; Regex.run/2: #{inspect(Regex.run(~r/^(a+)+$/, text))}")
end

IO.inspect(run.(flat, String.duplicate("a", 22) <> "b", []), label: "the flat pattern, 22 a's and a b")

# The nested pattern under smaller limits: the longest input each limit still decides.
for limit <- [1_000, 10_000, 100_000, 1_000_000] do
  decided =
    Enum.filter(4..30, fn n -> run.(nested, String.duplicate("a", n) <> "b", [{:match_limit, limit}]) == :nomatch end)

  IO.puts("match_limit #{limit}: up to #{Enum.max(decided)} a's")
end

from session import run

# How far the match limit of Erlang/OTP lets a nested repeat go, and the limits that decide less.
run("""
elixir limits.exs
""")

$ elixir limits.exs
16 a's and a b: :nomatch; Regex.run/2: nil
18 a's and a b: :nomatch; Regex.run/2: nil
20 a's and a b: :nomatch; Regex.run/2: nil
21 a's and a b: :nomatch; Regex.run/2: nil
22 a's and a b: {:error, :match_limit}; Regex.run/2: nil
24 a's and a b: {:error, :match_limit}; Regex.run/2: nil
the flat pattern, 22 a's and a b: :nomatch
match_limit 1000: up to 8 a's
match_limit 10000: up to 11 a's
match_limit 100000: up to 15 a's
match_limit 1000000: up to 18 a's

Up to 21 a’s, :re.run/3 answered :nomatch. From 22 a’s it answered {:error, :match_limit}: the engine had used up its budget, which the documentation puts at “10,000,000 in the Erlang VM”, before it could decide. Regex.run/2 said nil in both cases, so a caller cannot tell “no match” from “gave up”. The flat pattern decided 22 a’s at once. Step through the tries on a text of three a’s:

The smaller limits show the growth from the other side. A limit of 1,000 decided texts of up to 8 a’s, 10,000 up to 11, 100,000 up to 15 and 1,000,000 up to 18, so each tenfold budget buys only three or four more characters: this catastrophic backtrackingcatastrophic backtracking: A regex trying exponentially many ways to match before it can fail, as ^(a+)+$ does on a row of a's that ends in another letter. Bound the input and avoid a repeat inside a repeat. doubles its work with each extra a. For patterns or inputs that come from outside, three habits keep it bounded: refuse text longer than the field can be before matching, write patterns without a repeat inside a repeat, and when a pattern itself is untrusted, call :re.run/3 with a smaller {:match_limit, n} and :report_errors, so a pattern that runs away fails as an error you can see.

Sigils write words, text, regexes and dates, and you can add your own

~r/…/ is one of several sigils: a tilde, a letter, text between delimiters and optional modifiers after them. The sigils guide notes that “sigils support 8 different delimiters”, so a regex full of slashes can use braces, and that “uppercase-letter textual sigils do not perform interpolation nor escaping”. A custom sigil is a function, and the guide gives the rule for its name: “Custom sigils may be either a single lowercase character, or an uppercase character followed by more uppercase characters and digits.” sigils.exs uses the built-in ones and two of its own:

defmodule Sigils do
  # ~i(13) is 13 and ~i(42)n is -42; ~PCT(18) is 0.18.
  def sigil_i(text, []), do: String.to_integer(text)
  def sigil_i(text, [?n]), do: -String.to_integer(text)
  def sigil_PCT(text, []), do: String.to_integer(text) / 100
end

defmodule Pin do
  # A regex kept in a module attribute, compiled with the module.
  @pin ~r/\A[1-9][0-9]{5}\z/
  def valid?(text), do: Regex.match?(@pin, text)
end

defmodule Main do
  import Sigils

  def run do
    IO.inspect(~w(tea mug cup), label: "~w")
    IO.inspect(~w(tea mug cup)a, label: "~w with a")
    IO.inspect(~s(she said "hi" #{1 + 1}), label: "~s")
    IO.inspect(~S(she said "hi" #{1 + 1}\n), label: "~S")
    IO.inspect(~c(abc), label: "~c")
    IO.inspect(~r{/api/v\d+}.source, label: "~r with braces")
    IO.inspect(~i(13), label: "~i")
    IO.inspect(~i(42)n, label: "~i with n")
    IO.inspect(~PCT(18), label: "~PCT")
    IO.inspect({Pin.valid?("110001"), Pin.valid?("011001")}, label: "a regex in an attribute")
    exported = ~r/a+/E
    IO.inspect({exported, Regex.import(exported)}, label: "E, then Regex.import/1")
  end
end

Main.run()

from session import run

run("""
elixir sigils.exs
""")

$ elixir sigils.exs
~w: ["tea", "mug", "cup"]
~w with a: [:tea, :mug, :cup]
~s: "she said \"hi\" 2"
~S: "she said \"hi\" \#{1 + 1}\\n"
~c: ~c"abc"
~r with braces: "/api/v\\d+"
~i: 13
~i with n: -42
~PCT: 0.18
a regex in an attribute: {true, false}
E, then Regex.import/1: {~r/a+/E, ~r/a+/}

~i(42)n called sigil_i("42", [?n]): the modifiers arrive as a charlist, so one function can have a clause for each. A custom sigil must be imported to be used, as Main imported Sigils. The Pin module kept a regex in a module attribute, which compiled and matched on Elixir 1.20.4 with Erlang/OTP 29. The E modifier makes an exported regex that, as the documentation puts it, “can be shared across nodes or passed through config”, and Regex.import/1, new in Elixir 1.20, turns it back into an ordinary one, so it “won’t need to be imported on the fly every time it is executed.”

Identifiers may be Unicode, but never mixed scripts

Elixir allows Unicode in names: the Unicode syntax reference says it “supports Unicode in variables, atoms, and calls since Elixir v1.5”. It also stops one trick that letters from two scripts make possible, a name that looks like another but is a different name. The session compiles four small files:

# Names in several scripts, each name in one script, or scripts joined by _.
naïve = true
ação = :ação
दाम = 450
prix_аренды = 12
IO.inspect({naïve, ação, दाम, prix_аренды})

from session import run

# Names in Unicode: accepted, refused, warned about, and made equal by normalisation.
run("""
cd names
elixir unicode_names.exs
!elixir mixed.exs
elixir confusable.exs
elixir normalised.exs
""")

$ cd names
$ elixir unicode_names.exs
{true, :ação, 450, 12}
$ elixir mixed.exs
** (SyntaxError) invalid syntax found on mixed.exs:2:5:
    error: invalid mixed-script identifier found: аdmin

    Mixed-script identifiers are not supported for security reasons. 'аdmin' is made of the following scripts:

      \u0430 а {Cyrillic}
      \u0064 d {Latin}
      \u006D m {Latin}
      \u0069 i {Latin}
      \u006E n {Latin}

    Characters in identifiers from different scripts must be separated by underscore (_).
    Codepoint failed identifier tokenization, but a simpler form was found.

    Got:

        "аdmin" (code points 0x00430 0x00064 0x0006D 0x00069 0x0006E)

    Hint: You could write the above in a similar way that is accepted by Elixir:

        "admin" (code points 0x00061 0x00064 0x0006D 0x00069 0x0006E)

    See https://hexdocs.pm/elixir/unicode-syntax.html for more information.
    │
  2 │ аdmin = true
    │     ^
    │
    └─ mixed.exs:2:5
    (elixir 1.20.4) lib/code.ex:1639: Code.require_file/2
[exit status 1]
$ elixir confusable.exs
    warning: confusable identifier: 'сор' looks like 'cop' on line 2, but they are written using different characters


    │
  3 │ сор = 2
    │ ~
    │
    └─ confusable.exs:3:1

    warning: confusable identifier: 'сор' looks like 'cop' on line 2, but they are written using different characters


    │
  4 │ IO.inspect({cop, сор})
    │                  ~
    │
    └─ confusable.exs:4:18

{1, 2}
$ elixir normalised.exs
    warning: variable "café" is unused (if the variable is not meant to be used, prefix it with an underscore)
    │
  2 │ café = 1
    │ ~~~~
    │
    └─ normalised.exs:2:1

2
a quoted atom, not checked: :"аdmin"

naïve, ação and दाम were fine, each a name in one script, and so was prix_аренды, whose Latin and Cyrillic halves are separated by an underscore. mixed.exs was refused before it ran: its name begins with a Cyrillic а and continues in Latin, and the compiler said “Mixed-script identifiers are not supported for security reasons”, listed the script of every letter and suggested the all-Latin spelling. The reference states the rule: Elixir “disallows mixed-scripts which are not explicitly separated by _”, the check of Unicode’s security standard, UTS #39.

confusable.exs compiled, with a warning: its Cyrillic сор “looks like ‘cop’”, the Latin name on the line before, because Elixir “will also warn of confusable identifiers in the same file.” normalised.exs wrote café twice, once with é as one code point and once as e and a combining accent; Elixir “normalizes all atoms and variables to NFC form”, so both were the same variable, which the warning about the first one being unused shows. The last line shows the limit of all this: a quoted atom, :"аdmin", is not checked at all.

Practise: three checks for a form

A form sends a postal code, a phone number and a date as text. The rules are the exercise’s own: a PIN code is six ASCII digits whose first digit is 1 to 9, a phone number is +91 followed by ten ASCII digits and nothing else, and a ~d sigil turns ISO 8601 text into a Date.

Copy the two files into an empty folder:

# Checks for text a form sends, by rules this exercise defines:
#   a PIN code is six ASCII digits whose first digit is 1 to 9;
#   a phone number is +91 followed by ten ASCII digits, and nothing else.
defmodule Validate do
  # {:ok, text} or {:error, :bad_pin}.
  def pin_code(text) do
    if Regex.match?(~r/^\d{6}$/u, text), do: {:ok, text}, else: {:error, :bad_pin}
  end

  # {:ok, text} or {:error, :bad_phone}.
  def phone(text) do
    if Regex.match?(~r/\+91\d{10}/, text), do: {:ok, text}, else: {:error, :bad_phone}
  end

  # ~d[2026-10-04] is the Date ~D[2026-10-04]; text that is not an ISO 8601
  # date raises ArgumentError. Import it with: import Validate, only: [sigil_d: 2]
  def sigil_d(text, []), do: text
end

Code.require_file("validate.ex", __DIR__)
ExUnit.start(seed: 0)

defmodule ValidateTest do
  use ExUnit.Case, async: true
  import Validate, only: [sigil_d: 2]

  test "a PIN code: six ASCII digits, the first from 1 to 9" do
    assert Validate.pin_code("110001") == {:ok, "110001"}

    for bad <- ["011001", "11000", "1100011", "11 0001", "११०००१", "110001\n"] do
      assert Validate.pin_code(bad) == {:error, :bad_pin}, "accepted #{inspect(bad)}"
    end
  end

  test "a phone number: +91 and ten ASCII digits, nothing more" do
    assert Validate.phone("+919876543210") == {:ok, "+919876543210"}

    for bad <- ["9876543210", "+91 98765 43210", "+9198765432109", "call +919876543210", "+91९८७६५४३२१०"] do
      assert Validate.phone(bad) == {:error, :bad_phone}, "accepted #{inspect(bad)}"
    end
  end

  test "a long string of digits is refused" do
    phone = "+91" <> String.duplicate("9", 100_000)
    pin = String.duplicate("1", 100_000)
    assert Validate.phone(phone) == {:error, :bad_phone}, "accepted a phone number of #{byte_size(phone)} characters"
    assert Validate.pin_code(pin) == {:error, :bad_pin}, "accepted a PIN code of #{byte_size(pin)} characters"
  end

  test "~d turns ISO 8601 text into a Date, and refuses a day that does not exist" do
    assert ~d[2026-10-04] == ~D[2026-10-04]
    assert_raise ArgumentError, fn -> ~d[2026-02-30] end
  end
end

Run elixir validate_test.exs. The starter’s patterns are too loose and its sigil returns the text, so all four tests fail, each on the first input it should have refused:

from session import run

# The tests against the starter, whose patterns are too loose.
run("""
cd practice
!elixir validate_test.exs
""")

$ cd practice
$ elixir validate_test.exs
Running ExUnit with seed: 0, max_cases: 36



  1) test a PIN code: six ASCII digits, the first from 1 to 9 (ValidateTest)
     validate_test.exs:8
     accepted "011001"
     code: for bad <- ["011001", "11000", "1100011", "11 0001", "११०००१", "110001\n"] do
     stacktrace:
       (elixir 1.20.4) lib/enum.ex:1725: Enum."-map/2-lists^map/1-1-"/2
       validate_test.exs:11: (test)



  2) test a phone number: +91 and ten ASCII digits, nothing more (ValidateTest)
     validate_test.exs:16
     accepted "+9198765432109"
     code: for bad <- ["9876543210", "+91 98765 43210", "+9198765432109", "call +919876543210", "+91९८७६५४३२१०"] do
     stacktrace:
       (elixir 1.20.4) lib/enum.ex:1725: Enum."-map/2-lists^map/1-1-"/2
       (elixir 1.20.4) lib/enum.ex:1725: Enum."-map/2-lists^map/1-1-"/2
       validate_test.exs:19: (test)



  3) test a long string of digits is refused (ValidateTest)
     validate_test.exs:24
     accepted a phone number of 100003 characters
     code: assert Validate.phone(phone) == {:error, :bad_phone}, "accepted a phone number of #{byte_size(phone)} characters"
     stacktrace:
       validate_test.exs:27: (test)



  4) test ~d turns ISO 8601 text into a Date, and refuses a day that does not exist (ValidateTest)
     validate_test.exs:31
     Assertion with == failed
     code:  assert ~d[2026-10-04] == ~D[2026-10-04]
     left:  "2026-10-04"
     right: ~D[2026-10-04]
     stacktrace:
       validate_test.exs:32: (test)


Finished in 0.03 seconds (0.03s on load, 0.00s async, 0.00s sync)

Result: 0/4 passed
Failed: 4 tests
[exit status 2]

Each fix is a lesson of this page: [0-9] instead of \d with u, \A and \z instead of ^ and $, an anchor at both ends so that no extra text slips through, and Date.from_iso8601!/1 behind the sigil. The long input needs no special code once the pattern is anchored and flat. A finished version passes all four:

from session import run

# The same tests against a finished Validate.
run("""
cd finished
cp ../practice/validate_test.exs .
elixir validate_test.exs
""")

$ cd finished
$ cp ../practice/validate_test.exs .
$ elixir validate_test.exs
Running ExUnit with seed: 0, max_cases: 36

....
Finished in 0.03 seconds (0.03s on load, 0.00s async, 0.00s sync)

Result: 4 passed

Then check what you can do now. The self-check asks one question at a time, from the idea you know least, and every choice explains itself when you pick it:

Interview questions

Warm-up: what does the u modifier change in an Elixir regex?

It makes the pattern work on Unicode text instead of bytes. Character classes such as \w, \s and \d then match letters, spaces and digits of every script, \p{…} classes such as \p{Lu} and \p{Devanagari} become available, and the input must be valid UTF-8. That is what you want for names: \w+ accepts José only with u. It is not what you want for formats with fixed ASCII digits, because \d{6} with u also accepts six Devanagari digits; write [0-9]{6} there. Whichever you choose, anchor whole-field checks with \A and \z, since $ also matches before a newline at the end.

Two questions about patterns that meet real input:

Key takeaways

• Regex.run/2 returns the first match and its groups, Regex.scan/2 every match, named_captures/2 a map of named groups, and replace/3 a new string, with \\1 for a group or a function for each match.
• With the u modifier, classes are Unicode-aware, so \d matches digits of every script; exact checks use [0-9] and the whole-text anchors \A and \z.
• A repeat inside a repeat, such as ^(a+)+$, doubles its work with each extra character on near-matches; Erlang/OTP stops it at its match limit, and Regex.run/2 then says only nil.
• Sigils are a tilde, a letter and delimiters; a custom sigil is a sigil_x/2 function, imported, with a lowercase letter or an uppercase name.
• Elixir accepts Unicode names in one script, refuses mixed scripts not separated by _, warns about confusable names in one file and normalises names to NFC; data from users needs its own checks.