Your country

Tools that support it use your country for local currency, number formats, units and paper size. Your choice is saved only in this browser.

Type a name or a two-letter code. Use the up and down arrow keys to move through the countries, Enter to choose one and Escape to close.

Elixir & Erlang/OTP  Module 6 – Errors, files, time and text

IO, files, paths and JSON

Write files from iodata, read and inspect them with File, keep names from users inside a folder with Path.safe_relative/2, and encode and check JSON.

  • Intermediate
  • 15 minutes
  • Examples run with elixir 1.20.4, erlang 29.1.1 and Python 3.14.8
  • By MySmartCoPilot
Free lesson, Pro answers: 15 minutes
Free: the whole lesson. With a pass: the model answers to its 2 interview questions.

What you will learn

  • Build output as iodata and write it to a file without joining strings first
  • Read, inspect and remove files with File, and handle each reason it returns
  • Validate a file name from a user so that its path stays inside a base folder
  • Encode and decode JSON with Elixir's JSON module, choosing which fields of a struct it writes
  • Check decoded JSON against an allowlist of fields, types and sizes before using it

Before you start

What you’ll learn · Before you start

What you will learn

  • Build output as iodata and write it to a file without joining strings first
  • Read, inspect and remove files with File, and handle each reason it returns
  • Validate a file name from a user so that its path stays inside a base folder
  • Encode and decode JSON with Elixir's JSON module, choosing which fields of a struct it writes
  • Check decoded JSON against an allowlist of fields, types and sizes before using it

Before you start

On this page

A report of three learners’ marks is 34 bytes, a file name a user sends can be ../private/keys.txt, and an order a client posts can carry a "price" it was never meant to set. Writing files, taking paths from people and reading JSON each has a trap worth knowing before it costs you data.

Below, you will build output as iodata: A binary, or a list of binaries, bytes (integers from 0 to 255) and further such lists, nested at any depth. File.write/2, IO.write/2 and sockets write its pieces in order, as if they were one string, so output need not be joined first. and write it in one call, act on the reason File gives for each failure, keep names from users inside their folder with Path.safe_relative/2, encode and decode JSON with the module Elixir has shipped since version 1.18, and check what a client sent before using it. The outputs come from Elixir 1.20.4 running on Erlang/OTP 29.1.1.

iodata writes pieces without joining them first

A CSV report is a header and a line for each learner. Gluing those together with <> makes a new string at every step; iodata skips the gluing. The IO documentation defines it: “A term of type IO data is a binary or a list containing bytes (integers within the 0..255 range) or nested IO data”, and adds that “Building IO data is cheaper than concatenating binaries.” report.exs builds its report as a list of lists and passes it to File.write/2 as it is:

report.exs Elixir · report.exs
# A CSV report built as iodata: nested lists of strings and bytes, never joined.
rows = [{"Asha", 91}, {"Dev", 78}, {"Meera", 85}]

report = [
  "name,mark\n",
  Enum.map(rows, fn {name, mark} -> [name, ?,, Integer.to_string(mark), ?\n] end)
]

IO.inspect(report, label: "the iodata")
IO.inspect(IO.iodata_length(report), label: "bytes, counted without joining")
:ok = File.write("marks.csv", report)
IO.inspect(File.read!("marks.csv"), label: "marks.csv")
IO.inspect(IO.iodata_empty?([[], "", [[]]]), label: "nested empty pieces, empty?")

# In iodata an integer is one byte; in chardata it is a code point.
IO.inspect(IO.iodata_to_binary([?é]), label: "?é as iodata")
IO.inspect(IO.chardata_to_string([?é]), label: "?é as chardata")
A report built as iodata, measured and written Python · report_session.py
from session import run

run("""
elixir report.exs
""")

Output

$ elixir report.exs
the iodata: [
  "name,mark\n",
  [["Asha", 44, "91", 10], ["Dev", 44, "78", 10], ["Meera", 44, "85", 10]]
]
bytes, counted without joining: 34
marks.csv: "name,mark\nAsha,91\nDev,78\nMeera,85\n"
nested empty pieces, empty?: true
?é as iodata: <<233>>
?é as chardata: "é"

Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 report_session.py

The report printed as nested lists, with the commas and newlines as the integers 44 and 10, one byte each. IO.iodata_length/1 counted 34 bytes without making a string, and marks.csv held the four lines. Step through the write and watch the bytes add up:

Interactive · stage by stage File.write("marks.csv", report): the pieces, in order Step through the iodata of the report. Its pieces reach the file in order, depth first, left to right, and the bytes add up to 34.
File.write("marks.csv", report): the pieces, in order, step 6 of 6. Showing "name,mark\n" (10 bytes: 10 so far), the row for Asha (8 bytes: 18 so far), the row for Dev (7 bytes: 25 so far), the row for Meera (9 bytes: 34 so far), marks.csv. In focus: marks.csv."name,mark\n"10 bytes: 10 so farthe row for Asha8 bytes: 18 so farthe row for Dev7 bytes: 25 so farthe row for Meera9 bytes: 34 so farmarks.csv34 bytes from one call34 bytes

The drawing shows the whole scene; its 6 steps are listed below.

Every step as text
  1. One call, one list. File.write/2 receives the whole report: a list whose first element is the header and whose second is a list of three rows, each a list of its own. Nothing was joined beforehand.
  2. The header. The first piece is the string "name,mark\n", 10 bytes. They are the first 10 bytes of the file.
  3. Asha's row. Then the rows, in order. Asha's row is "Asha", the integer ?, (44, one byte), "91" and ?\n (10, one byte): 4 + 1 + 2 + 1 = 8 bytes, 18 so far.
  4. Dev's row. "Dev", 44, "78" and 10 are 3 + 1 + 2 + 1 = 7 bytes, 25 so far. An integer in iodata is always one byte, from 0 to 255.
  5. Meera's row. "Meera" adds 5 bytes, the comma, "85" and the newline 4 more: 9 bytes, 34 in all, the number IO.iodata_length/1 gave without building the string.
  6. The file. marks.csv holds the 34 bytes in that order, the same text IO.iodata_to_binary/1 would have built. The program never joined a string to get there: File.write/2 took the pieces as they were and put them together in one pass.

Three more lines of the session finish the picture. The first is IO.iodata_empty?/1, which says whether iodata holds no bytes, however deeply its empty lists and strings are nested: it was true for [[], "", [[]]].

New in Elixir 1.20

IO.iodata_empty?/1 arrived in Elixir 1.20.0, and so did File.read/2, the form that takes options such as [:raw]. On 1.19 and older, compare IO.iodata_length(data) == 0 instead, and call File.read/1.

The last two lines show the rule that trips people up: in iodata an integer is one byte, so [?é] became the single byte 233, which is not valid UTF-8. Text beyond ASCII goes in as strings, where "é" is its 2 bytes of UTF-8, or as chardata: Like iodata, but its integers are Unicode code points rather than bytes: a string, or a list of strings, code points and further such lists. IO.chardata_to_string/1 turns it into a UTF-8 string., whose integers are code points, through IO.chardata_to_string/1.

Key idea

iodata is a nested list of strings and bytes that File.write/2, IO.write/2 and sockets take as it is, so a program builds its output from pieces and never joins them into one big string itself.

Checkpoint

Check: iodata

  1. Question 1 of 2 How many bytes does IO.iodata_length(["ab", [?c, "de"], []]) count?

    Type a number.

    A hint

    Count the bytes of each string, one byte for each integer, nothing for an empty list.

    Show the answer: Question 1 of 2

    Answer: 5 bytes

    "ab" is 2 bytes, ?c is 1, "de" is 2 and the empty list adds nothing: 5. The nesting does not matter, which is why the report's 34 bytes could be counted without joining its pieces.

  2. Question 2 of 2 What does IO.iodata_to_binary([?é]) return?

    Choose one answer.

    Show the answer: Question 2 of 2

    Answer: <<233>>, one byte that is not valid UTF-8

    iodata integers are bytes, chardata integers are code points. Put text beyond ASCII into iodata as strings.

File answers each failure with a reason you can act on

The File documentation states the rule for its functions: they “return :ok or {:ok, result} in case of success, {:error, reason} otherwise”, and each has a bang twin that raises. The reasons are short atoms, the error names of the operating system. files.exs makes a folder, reads and inspects a file, then tries five things that fail:

files.exs Elixir · files.exs
# What File returns when it works, a reason for each way it fails, and two ways to write.
File.mkdir_p!("reports/q1")
File.write!("reports/q1/summary.txt", "3 learners\n")
IO.inspect(File.ls("reports/q1"), label: "ls")
{:ok, info} = File.stat("reports/q1/summary.txt")
IO.inspect({info.type, info.size}, label: "stat: type and size")
IO.inspect(File.read("reports/q1/summary.txt", [:raw]), label: "read with [:raw]")

IO.inspect(File.read("reports/q2/summary.txt"), label: "read a missing file")
IO.inspect(File.read("reports"), label: "read a folder")
IO.inspect(File.mkdir("reports"), label: "mkdir a folder that exists")
IO.inspect(File.write("drafts/notes.txt", "x"), label: "write into a missing folder")
File.write!("locked.txt", "x")
File.chmod!("locked.txt", 0o000)
IO.inspect(File.read("locked.txt"), label: "read without permission")
File.chmod!("locked.txt", 0o644)
IO.inspect(:file.format_error(:eacces), label: "the words for :eacces")

# :append keeps what is there; File.open/2 with :write empties the file first.
File.write!("log.txt", "line one\n")
File.write!("log.txt", "line two\n", [:append])
IO.inspect(File.read!("log.txt"), label: "after :append")
{:ok, device} = File.open("log.txt", [:write])
IO.inspect(is_pid(device), label: "File.open/2 gave a process")
File.close(device)
IO.inspect(File.read!("log.txt"), label: "after File.open with :write")

{:ok, removed} = File.rm_rf("reports")
IO.inspect(length(removed), label: "files and folders rm_rf removed")
File, when it works and five ways it fails Python · files_session.py
from session import run

run("""
elixir files.exs
""")

Output

$ elixir files.exs
ls: {:ok, ["summary.txt"]}
stat: type and size: {:regular, 11}
read with [:raw]: {:ok, "3 learners\n"}
read a missing file: {:error, :enoent}
read a folder: {:error, :eisdir}
mkdir a folder that exists: {:error, :eexist}
write into a missing folder: {:error, :enoent}
read without permission: {:error, :eacces}
the words for :eacces: ~c"permission denied"
after :append: "line one\nline two\n"
File.open/2 gave a process: true
after File.open with :write: ""
files and folders rm_rf removed: 3

Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 files_session.py

Call Reason What it means
File.read("reports/q2/summary.txt") :enoent no such file or folder
File.read("reports") :eisdir a folder where a file was expected
File.mkdir("reports") :eexist something already exists at that path
File.write("drafts/notes.txt", "x") :enoent the folder drafts does not exist
File.read("locked.txt"), no permission :eacces permission denied

Writing into a folder that does not exist fails with the same :enoent as reading a missing file, so make the folder first with File.mkdir_p/1, which creates missing parents and succeeds when the folder is already there. :file.format_error/1 turns a reason into words, ~c"permission denied" for :eacces. File.stat/1 gave the type and the size, File.ls/1 the names in a folder, and File.rm_rf/1 returned what it removed: 3 entries, the folder, its subfolder and the file. The [:raw] option of File.read/2 is, in the documentation’s words, “a single atom to bypass the file server and only check for the file locally”.

The last lines are about writing. File.write/3 with [:append] added a line to log.txt. File.open/2 with :write then emptied the file before anything was written, the data loss the Elixir guide warns about: “If an existing file is available at the given path, its contents will be deleted.” It also returned a process, because, as the File documentation says, “Every time a file is opened, Elixir spawns a new process.”

Opening a log with :write to add a line

File.open("app.log", [:write]) empties app.log first, so each run keeps only its own lines. Open it with [:append], or call File.write("app.log", line, [:append]), to add to what is there.

Checkpoint

Check: File and its reasons

  1. Question 1 of 2 File.write("drafts/notes.txt", "x") returned {:error, :enoent}. What is missing?

    Choose one answer.

    Show the answer: Question 1 of 2

    Answer: The folder drafts

    :enoent means no such file or folder along the path. Create folders first with File.mkdir_p/1.

  2. Question 2 of 2 A script saves reports under reports/q1, which may or may not exist yet, and neither may reports. Which call makes the folder safely?

    Choose one answer.

    Show the answer: Question 2 of 2

    Answer: File.mkdir_p("reports/q1")

    mkdir_p/1 makes the whole path and treats an existing folder as success, so it is safe to call on every run.

A file name from a user can walk out of its folder

An upload form lets people name their files, and a name is only text: report.pdf, or ../etc/passwd. Putting it after the upload folder does not keep it there. paths.exs tries five names against /srv/uploads, a folder that need not exist, because these functions work on the text of a path:

paths.exs Elixir · paths.exs
# Five names a user might send, joined to and expanded against the upload folder.
base = "/srv/uploads"

for name <- ["report.pdf", "q1/../report.pdf", "../etc/passwd", "/etc/passwd", ""] do
  IO.puts(inspect(name))
  IO.puts("  Path.join      #{inspect(Path.join(base, name))}")
  IO.puts("  Path.expand    #{inspect(Path.expand(name, base))}")
  IO.puts("  safe_relative  #{inspect(Path.safe_relative(name, base))}")
end
Five names, three ways to combine them with a folder Python · paths_session.py
from session import run

# Five names against a folder that only exists as a path.
run("""
elixir paths.exs
""")

Output

$ elixir paths.exs
"report.pdf"
  Path.join      "/srv/uploads/report.pdf"
  Path.expand    "/srv/uploads/report.pdf"
  safe_relative  {:ok, "report.pdf"}
"q1/../report.pdf"
  Path.join      "/srv/uploads/q1/../report.pdf"
  Path.expand    "/srv/uploads/report.pdf"
  safe_relative  {:ok, "report.pdf"}
"../etc/passwd"
  Path.join      "/srv/uploads/../etc/passwd"
  Path.expand    "/srv/etc/passwd"
  safe_relative  :error
"/etc/passwd"
  Path.join      "/srv/uploads/etc/passwd"
  Path.expand    "/etc/passwd"
  safe_relative  :error
""
  Path.join      "/srv/uploads"
  Path.expand    "/srv/uploads"
  safe_relative  {:ok, ""}

Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 paths_session.py

Path.join/2 resolves nothing: ../etc/passwd became /srv/uploads/../etc/passwd, which the operating system reads as /srv/etc/passwd, outside the folder. Path.expand/2 resolves the .. and shows where a name really leads; for /etc/passwd it ignored the folder entirely, as documented: “If the path is already an absolute path, relative_to is ignored.” Path.safe_relative/2 is the function made for this path traversal: Reaching a file outside the folder a program meant to use, through a name such as ../private/keys.txt, an absolute path or a link. Path.safe_relative/2 refuses such names.: it returned {:ok, "report.pdf"} for the two safe names, after removing q1/.., and :error for the two that leave the folder. Its documentation lists three unsafe cases, the third being when “A symbolic link in the path points to something above the root of relative_to.”

The empty name came back as {:ok, ""}: safe, but it names the folder itself. UploadPath.resolve/2 adds that one check, and links.exs gives it six names against a real folder, one of them through a link that nobody could spot by reading the name:

upload_path.ex Elixir · upload_path.ex
defmodule UploadPath do
  # The full path of a file a user named, kept inside base: {:ok, path}, or
  # {:error, :unsafe_name} for a name that leaves base or names base itself.
  def resolve(base, name) do
    case Path.safe_relative(name, base) do
      {:ok, ""} -> {:error, :unsafe_name}
      {:ok, relative} -> {:ok, Path.join(base, relative)}
      :error -> {:error, :unsafe_name}
    end
  end
end
links.exs Elixir · links.exs
Code.require_file("upload_path.ex", __DIR__)

# An upload folder, a private folder beside it, and a link from one into the other.
File.mkdir_p!("uploads/q1")
File.mkdir_p!("private")
File.write!("private/keys.txt", "not for uploads\n")
:ok = File.ln_s("../private", "uploads/shared")
base = Path.expand("uploads")

for name <- ["q1/report.pdf", "q1/../report.pdf", "../private/keys.txt", "shared/keys.txt", "/etc/passwd", "."] do
  IO.puts("#{inspect(name)} -> #{inspect(UploadPath.resolve(base, name))}")
end
Six names, one folder with a link out of it Python · links_session.py
from session import run

# UploadPath.resolve/2 against a real folder, with a link out of it.
run("""
elixir links.exs
""")

Output

$ elixir links.exs
"q1/report.pdf" -> {:ok, "uploads/q1/report.pdf"}
"q1/../report.pdf" -> {:ok, "uploads/report.pdf"}
"../private/keys.txt" -> {:error, :unsafe_name}
"shared/keys.txt" -> {:error, :unsafe_name}
"/etc/passwd" -> {:error, :unsafe_name}
"." -> {:error, :unsafe_name}

Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 links_session.py

Step through the six names:

Interactive · stage by stage UploadPath.resolve/2: six names, one folder Step through the names. Each passes Path.safe_relative/2 and the check for an empty name, or is refused.
UploadPath.resolve/2: six names, one folder, step 6 of 6. Showing The name ("."), Path.safe_relative/2, Not the folder itself, Path.join/2, {:error, :unsafe_name}. In focus: {:error, :unsafe_name}.:erroran empty nameThe name"."Path.safe_relative/2drops . and .., checks linksNot the folder itselfrefuses {:ok, ""}Path.join/2uploads + the relative name{:error, :unsafe_name}nothing is opened""

The drawing shows the whole scene; its 6 steps are listed below.

Every step as text
  1. q1/report.pdf. A plain relative name: safe_relative/2 returns it as it is, it is not empty, and Path.join/2 puts it under the upload folder: uploads/q1/report.pdf.
  2. q1/../report.pdf. The .. cancels q1, and the name stays inside the folder, so safe_relative/2 returns the shorter report.pdf, and the file is uploads/report.pdf.
  3. ../private/keys.txt. The .. climbs above the upload folder. safe_relative/2 returns :error, and the name is refused before any file is opened.
  4. shared/keys.txt. No .. at all, but uploads/shared is a link to ../private. safe_relative/2 follows links inside the folder and refuses one that points out of it.
  5. /etc/passwd. An absolute name is never relative to the folder, so safe_relative/2 returns :error for it too.
  6. ".". safe_relative/2 accepts ".", and "" too, as {:ok, ""}: the folder itself. resolve/2 refuses that, because a file name must name a file.

Check every path that comes from outside

The OTP secure coding guide’s rule MSC-002, Consider Path Traversal Attacks, asks for this check on any path from an untrusted source; Erlang code uses filelib:safe_relative_path/2. Pass the base folder explicitly, as resolve/2 does: links are checked against it, while the one-argument form checks against the current folder.

Checkpoint

Check: Names from users and their folder

  1. Question 1 of 2 What does Path.join("/srv/uploads", "../etc/passwd") return, and where does that path lead?

    Choose one answer.

    Show the answer: Question 1 of 2

    Answer: "/srv/uploads/../etc/passwd", which leads to /srv/etc/passwd

    Joining a name to a folder does not keep it inside. Check the name with Path.safe_relative/2 before joining.

  2. Question 2 of 2 Path.safe_relative(".", base) returned {:ok, ""}. Why does UploadPath.resolve/2 refuse it?

    Choose one answer.

    Show the answer: Question 2 of 2

    Answer: The empty relative path names the upload folder itself, not a file in it

    A safe path is not always a usable one: "" and "." are safe and name the folder, so a store refuses them.

The JSON module encodes and decodes in both directions

JSON.encode!/1 turns Elixir data into JSON text and JSON.decode/1 turns the text back, and the documentation says both “fully conform to RFC 8259 and ECMA 404 standards.” json.exs makes a round trip, encodes a struct, decodes two broken texts, and compares Erlang’s own :json; the session ends with a value JSON has no shape for:

json.exs Elixir · json.exs
defmodule Learner do
  # Only these two fields go into JSON; password_hash never does.
  @derive {JSON.Encoder, only: [:name, :city]}
  defstruct [:name, :city, :password_hash]
end

defmodule Main do
  def run do
    order = %{"sku" => "TEA-1", "qty" => 2, "gift" => false, "note" => nil}
    text = JSON.encode!(order)
    IO.puts(text)
    IO.inspect(JSON.decode(text), label: "decoded")
    IO.inspect(JSON.encode!(%{"tags" => [:new, :sale], "due" => ~D[2026-12-25]}), label: "atoms and a date")
    IO.puts(JSON.encode!(%Learner{name: "Asha", city: "Pune", password_hash: "x9f2"}))
    IO.inspect(JSON.decode("{bad"), label: "not JSON")
    IO.inspect(JSON.decode(~s({"qty": 2)), label: "cut short")
    IO.inspect(:json.decode(~s({"note": null})), label: "Erlang's :json")
  end
end

Main.run()
JSON both ways, and a tuple that cannot be encoded Python · json_session.py
from session import run

# Encoding and decoding with JSON, then a value JSON has no shape for.
run("""
elixir json.exs
!elixir -e 'JSON.encode!({:ok, 1})'
""")

Output

$ elixir json.exs
{"gift":false,"note":null,"qty":2,"sku":"TEA-1"}
decoded: {:ok, %{"gift" => false, "note" => nil, "qty" => 2, "sku" => "TEA-1"}}
atoms and a date: "{\"due\":\"2026-12-25\",\"tags\":[\"new\",\"sale\"]}"
{"name":"Asha","city":"Pune"}
not JSON: {:error, {:invalid_byte, 1, 98}}
cut short: {:error, {:unexpected_end, 9}}
Erlang's :json: %{"note" => :null}
$ elixir -e 'JSON.encode!({:ok, 1})'
** (Protocol.UndefinedError) protocol JSON.Encoder not implemented for Tuple, the protocol must be explicitly implemented.

If you have a struct, you can derive the implementation specifying which fields should be encoded to JSON:

    @derive {JSON.Encoder, only: [....]}
    defstruct ...

It is also possible to encode all fields, although this should be used carefully to avoid accidentally leaking private information when new fields are added:

    @derive JSON.Encoder
    defstruct ...

Finally, if you don't own the struct you want to encode to JSON, you may use Protocol.derive/3 placed outside of any module:

    Protocol.derive(JSON.Encoder, NameOfTheStruct, only: [...])
    Protocol.derive(JSON.Encoder, NameOfTheStruct)

Got value:

    {:ok, 1}

    (elixir 1.20.4) lib/json.ex:4: JSON.Encoder.impl_for!/1
    (elixir 1.20.4) lib/json.ex:117: JSON.Encoder.encode/2
    (elixir 1.20.4) lib/json.ex:489: JSON.encode!/2
    (stdlib 8.1) erl_eval.erl:1048: :erl_eval.do_apply/7
[exit status 1]

Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 json_session.py

Elixir JSON Decoded again
"TEA-1", 2, false "TEA-1", 2, false the same
nil null nil
atoms, such as :new strings, "new" strings
maps, keys as atoms or strings objects maps with string keys
~D[2026-12-25] "2026-12-25" the string
tuples, structs without an encoder Protocol.UndefinedError none

Decoding never creates atoms: keys come back as strings, which keeps text from outside out of the atom table. Errors are data, too: {:invalid_byte, 1, 98} is the b of {bad at byte 1, and {:unexpected_end, 9} is a text that stopped after 9 bytes. A struct goes into JSON only when it says how. @derive {JSON.Encoder, only: [:name, :city]} wrote the learner as {"name":"Asha","city":"Pune"} and left the password hash out, and the error for the tuple explains why only: is the careful choice: encoding every field “should be used carefully to avoid accidentally leaking private information when new fields are added”.

One difference matters when Elixir and Erlang code share data: Erlang’s :json, part of Erlang/OTP since 27, decoded null to the atom :null, where Elixir’s module gives nil.

Checkpoint

Check: Encoding and decoding JSON

  1. Question 1 of 2 What does JSON.decode(~s({"qty": 2})) return?

    Choose one answer.

    Show the answer: Question 1 of 2

    Answer: {:ok, %{"qty" => 2}}

    String keys keep text from outside out of the atom table; match on them as strings.

  2. Question 2 of 2 A struct has :name, :email and :password_hash. How do you make JSON.encode!/1 write only the first two?

    Choose one answer.

    Show the answer: Question 2 of 2

    Answer: @derive {JSON.Encoder, only: [:name, :email]} before defstruct

    only: is an allowlist of fields, so a field added later stays out of JSON until someone adds it on purpose.

Decoded JSON is data to check, not data to trust

A valid JSON text can still hold the wrong things: a quantity as the string "2", a list where an object belongs, a field the client must not set. The OTP secure coding guide recommends a data format such as JSON over Erlang’s term format for untrusted input, in rule DSG-011, Only Deserialize Trusted Data, and the decoded data still needs checking. The OWASP Input Validation Cheat Sheet puts the rule in one line: “Define what the application accepts and reject values outside those rules.” OrderCheck.from_json/1 does that for an order:

order_check.ex Elixir · order_check.ex
defmodule OrderCheck do
  # An order a client sent as JSON: {:ok, %{sku: sku, qty: qty}} or {:error, reason}.
  # Only the fields on this list are read; anything else the client sent is ignored.
  @max_bytes 1_000

  def from_json(text) when byte_size(text) > @max_bytes, do: {:error, :too_large}

  def from_json(text) do
    with {:ok, data} <- decode(text),
         {:ok, sku} <- sku(data),
         {:ok, qty} <- qty(data) do
      {:ok, %{sku: sku, qty: qty}}
    end
  end

  defp decode(text) do
    case JSON.decode(text) do
      {:ok, %{} = data} -> {:ok, data}
      {:ok, _other} -> {:error, :not_an_object}
      {:error, reason} -> {:error, {:bad_json, reason}}
    end
  end

  defp sku(%{"sku" => sku}) when is_binary(sku) and byte_size(sku) in 1..20, do: {:ok, sku}
  defp sku(_data), do: {:error, :bad_sku}

  defp qty(%{"qty" => qty}) when is_integer(qty) and qty in 1..99, do: {:ok, qty}
  defp qty(_data), do: {:error, :bad_qty}
end
orders.exs Elixir · orders.exs
Code.require_file("order_check.ex", __DIR__)

inputs = [
  ~s({"sku": "TEA-1", "qty": 2}),
  ~s({"sku": "TEA-1", "qty": "2"}),
  ~s({"sku": "TEA-1", "qty": 2, "price": 0}),
  ~s(["TEA-1", 2]),
  ~s({"sku": "TEA-1", "qty": 2),
  String.duplicate(" ", 1_001)
]

for text <- inputs do
  shown = if byte_size(text) > 40, do: "#{byte_size(text)} spaces", else: inspect(text)
  IO.puts("#{shown} -> #{inspect(OrderCheck.from_json(text))}")
end
Six requests, checked Python · orders_session.py
from session import run

# Six requests from a client, checked by OrderCheck.from_json/1.
run("""
elixir orders.exs
""")

Output

$ elixir orders.exs
"{\"sku\": \"TEA-1\", \"qty\": 2}" -> {:ok, %{sku: "TEA-1", qty: 2}}
"{\"sku\": \"TEA-1\", \"qty\": \"2\"}" -> {:error, :bad_qty}
"{\"sku\": \"TEA-1\", \"qty\": 2, \"price\": 0}" -> {:ok, %{sku: "TEA-1", qty: 2}}
"[\"TEA-1\", 2]" -> {:error, :not_an_object}
"{\"sku\": \"TEA-1\", \"qty\": 2" -> {:error, {:bad_json, {:unexpected_end, 25}}}
1001 spaces -> {:error, :too_large}

Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 orders_session.py

Step through the six requests:

Interactive · stage by stage OrderCheck.from_json/1: six requests from a client Step through the six requests. Each passes the size limit, the decoder and the two fields in turn, or stops at the first check it fails.
OrderCheck.from_json/1: six requests from a client, step 6 of 6. Showing At most 1, JSON.decode/1, "sku": a short string, "qty": an integer, The order, Refused. In focus: Refused.:too_largenot an object:bad_qtyAt most 1else :too_largeJSON.decode/1an object, else a reason"sku": a short string1 to 20 bytes, else :bad_sku"qty": an integer1 to 99, else :bad_qtyThe order{:ok, %{sku: …, qty: …}}Refused{:error, reason}:too_large

The drawing shows the whole scene; its 6 steps are listed below.

Every step as text
  1. A good order. {"sku": "TEA-1", "qty": 2} is small, decodes to a map, and both fields pass, so the result is {:ok, %{sku: "TEA-1", qty: 2}}.
  2. "qty": "2". Valid JSON, but the quantity is the string "2". The guard is_integer(qty) fails, and the request stops with :bad_qty.
  3. An extra "price". The client added "price": 0. The order is built from sku and qty only, so the extra field is ignored and cannot set anything.
  4. A list. ["TEA-1", 2] decodes, but to a list, not an object, so decode/1 answers :not_an_object.
  5. Cut short. The closing brace is missing. JSON.decode/1 reports {:unexpected_end, 25}, and the reason is passed on as {:bad_json, …}.
  6. 1,001 spaces. The first clause compares byte_size/1 with the limit before anything is decoded, so an oversized request costs one comparison: :too_large.

Three choices make the check strict. The size is checked first, on the raw text, so a 1,001-byte request costs one comparison and is never decoded. The fields are read by name from an allowlist: A list of what is allowed, such as the JSON fields a client may set: anything not on it is dropped or refused, instead of trying to list everything that is forbidden. More in “Structs and data modelling”, so the "price": 0 a client added did nothing: the result is built from sku and qty alone. And each field is checked for its type and its range, so "qty": "2" was refused, though it looks like a number. Each failure is a reason of its own, :bad_qty, :not_an_object or {:bad_json, …}, as the first lesson of this module set out.

Turning JSON keys into atoms

Map.new(data, fn {k, v} -> {String.to_atom(k), v} end) lets any client add names to the atom table. Erlang’s documentation warns that atoms are never removed and that the VM stops when the table reaches its limit, 1,048,576 atoms by default. Match the keys you accept as strings, as sku/1 and qty/1 do, and build the atom keys yourself.

Checkpoint

Check: Checking decoded JSON

  1. Question 1 of 2 A client adds "price": 0 to an order. What did OrderCheck.from_json/1 do with it?

    Choose one answer.

    Show the answer: Question 1 of 2

    Answer: Nothing: the order is built from sku and qty alone, so the field is ignored

    Reading fields by name from a list of allowed ones means a client cannot set anything the code did not ask for.

  2. Question 2 of 2 What does OrderCheck.from_json(~s({"sku": "TEA-1", "qty": "2"})) return?

    Choose one answer.

    Show the answer: Question 2 of 2

    Answer: {:error, :bad_qty}

    Valid JSON can hold the wrong types. Check each field's type and range, as OWASP's rule says: define what is accepted, and reject the rest.

JSON Validator Paste a JSON text to see whether it parses and where it breaks, before your code decodes it.

Practise: a document store under one folder

A small store keeps each document as a JSON file under a base folder, by a name the caller chooses, such as orders/42.json. Names come from outside, so save/3 and load/2 must keep every one inside the folder, and a document whose JSON is over 10,000 bytes is refused before anything is written. Copy the two files into an empty folder:

practice/store.ex: the starter Elixir · practice/store.ex
# A small document store: each document is a JSON file under a base folder,
# saved and loaded by a name the caller chooses, such as "orders/42.json".
defmodule Store do
  # :ok, or {:error, reason}:
  #   :unsafe_name  the name leaves base, is absolute, or names base itself
  #   :too_large    the JSON of doc is longer than 10_000 bytes
  #   a reason from File, such as :eacces
  # Folders in the name are created as needed.
  def save(_base, _name, _doc), do: {:error, :not_written}

  # {:ok, doc}, or {:error, reason}: :unsafe_name as above, :enoent for a
  # document that does not exist, {:bad_json, reason} for a file that is not JSON.
  def load(_base, _name), do: {:error, :not_written}
end
practice/store_test.exs: the tests Elixir · practice/store_test.exs
Code.require_file("store.ex", __DIR__)
ExUnit.start(seed: 0)

defmodule StoreTest do
  use ExUnit.Case, async: true
  # Each test gets an empty folder of its own, under tmp/ beside this file.
  @moduletag :tmp_dir

  test "a document comes back as it was saved", %{tmp_dir: dir} do
    assert Store.save(dir, "orders/42.json", %{"sku" => "TEA-1", "qty" => 2}) == :ok
    assert Store.load(dir, "orders/42.json") == {:ok, %{"sku" => "TEA-1", "qty" => 2}}
  end

  test "a name that leaves the folder is refused, and nothing is written", %{tmp_dir: dir} do
    for name <- ["../escape.json", "/etc/escape.json", "a/../../escape.json", ""] do
      assert Store.save(dir, name, %{"x" => 1}) == {:error, :unsafe_name}
      assert Store.load(dir, name) == {:error, :unsafe_name}
    end

    refute File.exists?(Path.join(dir, "../escape.json"))
  end

  test "a document over the limit is refused before anything is written", %{tmp_dir: dir} do
    big = %{"text" => String.duplicate("a", 10_000)}
    assert Store.save(dir, "big.json", big) == {:error, :too_large}
    refute File.exists?(Path.join(dir, "big.json"))
  end

  test "a missing document and a broken file have their own reasons", %{tmp_dir: dir} do
    assert Store.load(dir, "nothing.json") == {:error, :enoent}
    File.write!(Path.join(dir, "broken.json"), "{bad")
    assert Store.load(dir, "broken.json") == {:error, {:bad_json, {:invalid_byte, 1, 98}}}
  end
end

Run elixir store_test.exs. The starter answers {:error, :not_written} everywhere, so all four tests fail:

The tests against the starter Python · practice_session.py
from session import run

# The tests against the starter, whose functions are placeholders.
run("""
cd practice
!elixir store_test.exs
""")

Output

$ cd practice
$ elixir store_test.exs
Running ExUnit with seed: 0, max_cases: 36



  1) test a document comes back as it was saved (StoreTest)
     store_test.exs:9
     Assertion with == failed
     code:  assert Store.save(dir, "orders/42.json", %{"sku" => "TEA-1", "qty" => 2}) == :ok
     left:  {:error, :not_written}
     right: :ok
     stacktrace:
       store_test.exs:10: (test)



  2) test a name that leaves the folder is refused, and nothing is written (StoreTest)
     store_test.exs:14
     Assertion with == failed
     code:  assert Store.save(dir, name, %{"x" => 1}) == {:error, :unsafe_name}
     left:  {:error, :not_written}
     right: {:error, :unsafe_name}
     stacktrace:
       store_test.exs:16: anonymous fn/3 in StoreTest."test a name that leaves the folder is refused, and nothing is written"/1
       (elixir 1.20.4) lib/enum.ex:2622: Enum."-reduce/3-lists^foldl/2-0-"/3
       store_test.exs:15: (test)



  3) test a document over the limit is refused before anything is written (StoreTest)
     store_test.exs:23
     Assertion with == failed
     code:  assert Store.save(dir, "big.json", big) == {:error, :too_large}
     left:  {:error, :not_written}
     right: {:error, :too_large}
     stacktrace:
       store_test.exs:25: (test)



  4) test a missing document and a broken file have their own reasons (StoreTest)
     store_test.exs:29
     Assertion with == failed
     code:  assert Store.load(dir, "nothing.json") == {:error, :enoent}
     left:  {:error, :not_written}
     right: {:error, :enoent}
     stacktrace:
       store_test.exs:30: (test)


Finished in 0.2 seconds (0.05s on load, 0.1s async, 0.00s sync)

Result: 0/4 passed
Failed: 4 tests
[exit status 2]

This output changes from run to run: ExUnit prints how long the tests took, and max_cases follows the computer's processors

Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 practice_session.py

Write one private function that turns a name into a path inside the base folder, as UploadPath.resolve/2 does, and use it in both functions. Encode with JSON.encode_to_iodata!/1 and measure with IO.iodata_length/1, so the size is known before a byte is written, then create the folders with File.mkdir_p/1 and write. Loading is a read and a decode, with the decoder’s reason wrapped as {:bad_json, reason}. A finished version passes all four:

The tests against a finished Store Python · finished_session.py
from session import run

# The same tests against a finished Store.
run("""
cd finished
cp ../practice/store_test.exs .
elixir store_test.exs
""")

Output

$ cd finished
$ cp ../practice/store_test.exs .
$ elixir store_test.exs
Running ExUnit with seed: 0, max_cases: 36

....
Finished in 0.1 seconds (0.05s on load, 0.08s async, 0.00s sync)

Result: 4 passed

This output changes from run to run: ExUnit prints how long the tests took, and max_cases follows the computer's processors

Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 finished_session.py

Then see how much of it stays with you. The questions arrive one at a time, starting from the idea where your answers are shakiest, and each pick is explained the moment you make it:

Practice that adapts to you

Practise this lesson

10 questions about 5 ideas of this lesson. Each answer is explained at once, and what you get right is kept in this browser.

  1. Question 1 of 10 Which of these is valid iodata?

    Choose one answer.

    Show the answer: Question 1 of 10

    Answer: ["name,", [?A, "sha"], ?\n]

    The IO documentation's definition: a binary, or a list of bytes from 0 to 255 and nested IO data.

  2. Question 2 of 10 A program writes File.write("names.txt", String.to_charlist("José")). What goes wrong?

    Choose one answer.

    Show the answer: Question 2 of 10

    Answer: The é, code point 233, is written as the single byte 233, so the file is not valid UTF-8

    Write the string itself, or convert chardata with IO.chardata_to_string/1 first, as report.exs showed with ?é.

  3. Question 3 of 10 What does the reason :eisdir mean?

    Choose one answer.

    Show the answer: Question 3 of 10

    Answer: A folder was found where a file was expected

    The reasons are the operating system's error names; :file.format_error/1 gives the words, such as "illegal operation on a directory".

  4. Question 4 of 10 A program opens app.log with File.open("app.log", [:write]) each time it starts. What happens to the lines from the last run?

    Choose one answer.

    Show the answer: Question 4 of 10

    Answer: They are deleted: :write empties an existing file

    The Elixir guide warns that opening for writing deletes an existing file's contents. Use :append to add.

  5. Question 5 of 10 What does Path.expand("/etc/passwd", "/srv/uploads") return?

    Choose one answer.

    Show the answer: Question 5 of 10

    Answer: "/etc/passwd"

    expand/2 shows where a name leads, which is why it cannot keep a name inside a folder.

  6. Question 6 of 10 A name has no .. and is not absolute, yet Path.safe_relative(name, base) returned :error. What explains it?
    Show the answer: Question 6 of 10

    Answer: The name goes through a symbolic link inside base that points outside it

    The third unsafe case: a symbolic link in the path that points above the base. Give the base folder explicitly so links are checked against it.

  7. Question 7 of 10 What does Elixir's JSON.decode/1 turn a JSON null into?

    Choose one answer.

    Show the answer: Question 7 of 10

    Answer: nil

    Elixir maps null to nil and back; Erlang's :json maps it to the atom null.

  8. Question 8 of 10 JSON.encode!({:ok, 1}) raised. Why?

    Choose one answer.

    Show the answer: Question 8 of 10

    Answer: JSON has no shape for a tuple, so JSON.Encoder is not implemented for it

    Turn tuples into maps or lists first, and give structs an encoder with @derive, ideally with only:.

  9. Question 9 of 10 Why does OrderCheck.from_json/1 compare byte_size(text) with the limit before decoding?

    Choose one answer.

    Show the answer: Question 9 of 10

    Answer: An oversized request is refused with one comparison, before any time or memory goes into decoding it

    Cheap checks first: size, then syntax, then the fields. A limit on input size is part of every allowlist.

  10. Question 10 of 10 A teammate turns decoded keys into atoms with String.to_atom/1 so the code can write order.qty. What is the risk?

    Choose one answer.

    Show the answer: Question 10 of 10

    Answer: Every new key a client sends adds an atom, the table is never emptied, and once it is full the node stops

    Match accepted keys as strings and build atom keys yourself, as OrderCheck does, or use String.to_existing_atom/1 for a fixed set.

Interview questions

Warm-up: what is iodata, and why do Elixir programs write it instead of one big string?

iodata is a binary, or a list of binaries, bytes (integers from 0 to 255) and more such lists, nested as deeply as you like. IO functions, File.write/2 and sockets take it as it is and write its pieces in order, as if they were one string, and JSON.encode_to_iodata!/1 returns it. So output is built by putting pieces into lists, [header, rows], instead of copying everything written so far into a new string at each step, which the IO documentation says is cheaper; IO.iodata_length/1 measures it without joining it. The one trap is that integers are bytes: [?é] is the single byte 233, not UTF-8, so text beyond ASCII goes in as strings.

Two questions about data that comes from outside:

Model answers open with any pass: ₹79 for 1 week ($2.99), one payment that never renews.

  1. Mid to senior

    A service saves uploads with File.write(Path.join("/srv/uploads", params["name"]), body). A security scan of your own staging service reports that the name ../config/runtime.exs overwrote a file outside the folder. Fix the code, and say how you would prove the fix holds, links included.

    Model answer, Pro. Model answers open with any pass; prices are in the notice on this page. First step and follow-ups

  2. Fresher to mid

    An endpoint builds an order with Map.merge(%{status: "new"}, decoded), where decoded is the request's JSON with its keys turned into atoms. Clients have sent orders with their own price and status, and memory use creeps up after every scan. Explain both problems and rewrite the intake.

    Model answer, Pro. First step and follow-ups

Key takeaways

  • iodata, nested lists of strings and bytes, is taken as it is by File.write/2 and the IO functions, so a program builds output from pieces and never joins them itself; its integers are bytes, not code points.
  • Every File function answers a failure with a reason such as :enoent, :eisdir, :eexist or :eacces; File.mkdir_p/1 creates missing folders, and opening a file with :write empties it.
  • Path.join/2 keeps .. in a name and Path.expand/2 follows it out of the folder; Path.safe_relative/2 with the base folder refuses names that leave it, links included, and an empty result still needs refusing.
  • JSON.encode!/1 and JSON.decode/1 map Elixir data to JSON and back with string keys and nil for null, and a struct is encoded only through @derive, best with only:.
  • Decoded JSON is checked before use: a size limit first, then the allowed fields by name, each with its type and range, and a reason for every way it can fail.

Terms in this lesson

chardata
Like iodata, but its integers are Unicode code points rather than bytes: a string, or a list of strings, code points and further such lists. IO.chardata_to_string/1 turns it into a UTF-8 string.
iodata
A binary, or a list of binaries, bytes (integers from 0 to 255) and further such lists, nested at any depth. File.write/2, IO.write/2 and sockets write its pieces in order, as if they were one string, so output need not be joined first.
path traversal
Reaching a file outside the folder a program meant to use, through a name such as ../private/keys.txt, an absolute path or a link. Path.safe_relative/2 refuses such names.

References

Related tools

Report a problem with this lesson

Quick answers and tool search

Type to search tools or to get a quick answer, for example 18% of 2500. Use the up and down arrow keys to move through the results, Enter to choose, and Escape to close.