Anonymous functions, closures and capture

A checkout takes one of three discount codes on an order of 120,000 paise: none, 10 % off or a flat 15,000 off. A case on the code in every place that needs a price repeats the same branches; keep each rule as a value instead, a function in a map, and one line can pick the rule and run it. Elixir makes such values with fn and with the capture operator &, and calls them with a dot.

Below, you will make and call anonymous functionsanonymous function: A function value without a name, made with fn … end or with &, such as fn x -> x * 2 end. A name bound to one is called with a dot, double.(5), and the value can be stored, passed and returned like any other., give one of them several clauses, see what a closureclosure: A function that keeps the values its names had when it was made. After rate = 10, fn p -> p * rate end keeps 10 even when rate is bound to 50 later, and it carries that value into every process that runs it. keeps from the code around it and what that costs, write captures, and pass functions around as data. The programs below ran on Elixir 1.20.4 and Erlang/OTP 29.1.1.

fn makes a function, and a dot calls it

fn x -> x * 2 end is a function with no name: one argument x, an arrow, and the body. Bind it to a name, and call it by putting a dot between the name and the parentheses. The session below does that, then gets three calls wrong on purpose:

from session import shell

# fn ... end makes a function value; a dot calls it. Each wrong call fails in its own way.
shell([
    "double = fn x -> x * 2 end",
    "double.(5)",
    "double(5)",
    "double.(1, 2)",
    "is_function(double, 1)",
    "price = 5",
    "price.(1)",
    "System.halt()",
])

$ iex
Erlang/OTP 29 [erts-17.1] [source] [64-bit] [smp:18:18] [ds:18:18:10] [async-threads:1] [jit]

Interactive Elixir (1.20.4) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> double = fn x -> x * 2 end
#Function<42.130099583/1 in :erl_eval.expr/6>
iex(2)> double.(5)
10
iex(3)> double(5)
error: undefined function double/1 (there is no such import)
└─ iex:3

** (CompileError) cannot compile code (errors have been logged)

iex(3)> double.(1, 2)
** (BadArityError) #Function<42.130099583/1 in :erl_eval.expr/6> with arity 1 called with 2 arguments (1, 2)
    (stdlib 8.1) erl_eval.erl:1014: :erl_eval.do_apply/6
    iex:3: (file)
iex(3)> is_function(double, 1)
true
iex(4)> price = 5
5
iex(5)> price.(1)
** (BadFunctionError) expected a function, got: 5
    iex:5: (file)
iex(5)> System.halt()

IEx printed the function as #Function<42.130099583/1 in :erl_eval.expr/6>: a value like any other, made by the evaluator that runs IEx, and the /1 is its arityarity: The number of arguments a function takes, written after a slash: String.upcase/1 takes one argument, and an anonymous function printed as #Function<…/2 …> takes two., the number of arguments it takes. double.(5) returned 10. Without the dot, double(5) did not even compile: Elixir looked for a named function double/1 and found none. The Elixir guide gives the reason for the dot: with it, is_atom.(:foo) can never be mistaken for the named function is_atom(:foo), so nobody reading the code has to search for a variable of that name.

The two failed calls each raised their own error. double.(1, 2) passed two arguments to a function of arity 1 and raised BadArityError; price.(1) tried to call 5 and raised BadFunctionError, “expected a function, got: 5”. is_function(double, 1) checks both things at once: that the value is a function, and that it takes exactly one argument.

One function can have several clauses, all of the same arity

An anonymous function can hold several clauses, each with its own pattern and guard, tried from the top exactly as the clauses of case are. This module keeps one such function in describe and hands it to Enum.map/2, which calls it once for every reply in the list:

# One anonymous function with three clauses, used by Enum.map/2 on every reply.
defmodule Checkout do
  def messages(replies) do
    describe = fn
      {:ok, amount} when amount > 0 -> "paid #{amount}"
      {:ok, 0} -> "nothing to pay"
      {:error, reason} -> "failed: #{reason}"
    end

    Enum.map(replies, describe)
  end
end

IO.inspect(Checkout.messages([{:ok, 250}, {:ok, 0}, {:error, :card_declined}]))
IO.inspect(Checkout.messages([{:ok, 100}, :pending]))

# Every clause of one anonymous function must take the same number of arguments.
add = fn
  x -> x
  x, y -> x + y
end

from session import run

# The second list holds a reply that fits none of the clauses; arity_mix.exs does not compile.
run("""
!elixir checkout.exs
!elixir arity_mix.exs
""")

$ elixir checkout.exs
["paid 250", "nothing to pay", "failed: card_declined"]
** (FunctionClauseError) no function clause matching in anonymous fn/1 in Checkout.messages/1

    The following arguments were given to anonymous fn/1 in Checkout.messages/1:

        # 1
        :pending

    anonymous fn/1 in Checkout.messages/1
    (elixir 1.20.4) lib/enum.ex:1725: Enum."-map/2-lists^map/1-1-"/2
    (elixir 1.20.4) lib/enum.ex:1725: Enum."-map/2-lists^map/1-1-"/2
    checkout.exs:15: (file)
[exit status 1]
$ elixir arity_mix.exs
    error: cannot mix clauses with different arities in anonymous functions
    │
  2 │ add = fn
    │       ^
    │
    └─ arity_mix.exs:2:7

** (CompileError) arity_mix.exs: cannot compile file (errors have been logged)
[exit status 1]

{:ok, 250} fitted the first clause, whose guard wants an amount above 0. {:ok, 0} failed that guard and fitted the second clause, and {:error, :card_declined} the third. In the second list, :pending fitted no clause, so the call raised FunctionClauseErrorFunctionClauseError: The error a call raises when the number of arguments is right but none of the function's clauses fits them. The message names the function, such as anonymous fn/1 in Checkout.messages/1, and lists the arguments it was given.: “no function clause matching in anonymous fn/1 in Checkout.messages/1”, which names the function by its arity and the function that made it, followed by the argument it was given. The stack trace shows that Enum.map/2 made the call.

arity_mix.exs stopped before it ran: “cannot mix clauses with different arities in anonymous functions”. The guide states the rule: every clause of one anonymous function takes the same number of arguments, because a function value has one arity. Follow one call through every check it passes before its body runs:

A call meets each check in turn: a function, then the arity, then a clause that fits. The error tells you which check failed, so BadFunctionError, BadArityError and FunctionClauseError each point to a different mistake.

A closure keeps the values it saw

A function made with fn may use names from the code around it. This script makes a discount function that uses rate, then binds rate again before calling it:

# A closure keeps the values its names had when it was made.
rate = 10
discount = fn paise -> paise - div(paise * rate, 100) end
rate = 50

IO.puts("rate is now #{rate}")
IO.puts("discount.(120_000) = #{discount.(120_000)}")

# A name bound inside a function stays inside it.
total = 0
Enum.each([300, 450, 250], fn amount -> total = total + amount end)
IO.puts("total after Enum.each: #{total}")

total = Enum.reduce([300, 450, 250], 0, fn amount, sum -> sum + amount end)
IO.puts("total after Enum.reduce: #{total}")

from session import run

run("""
elixir closures.exs
""")

$ elixir closures.exs
    warning: variable "total" is unused (there is a variable with the same name in the context, use the pin operator (^) to match on it or prefix this variable with underscore if it is not meant to be used)
    │
 11 │ Enum.each([300, 450, 250], fn amount -> total = total + amount end)
    │                                         ~
    │
    └─ closures.exs:11:41

rate is now 50
discount.(120_000) = 108000
total after Enum.each: 0
total after Enum.reduce: 1000

The function returned 108000, 10 % off, although rate was 50 by the time it ran. When fn made the function, it kept the value that rate had at that moment, 10. A function that keeps values from where it was made is a closure, and the guide says such functions “close over their scope”. Binding rate again later gives the script’s name a new value; it does not reach into the function:

The second half of the script shows the same rule from the inside. total = total + amount inside the function binds a new total that exists only for that one call, so Enum.each/2 left the outer total at 0. The compiler saw it before the script ran and warned that the inner total is unused, with “a variable with the same name in the context”.

The guide puts the rule plainly: “A variable assigned inside a function does not affect its surrounding environment.”

A closure carries what it captured, wherever it goes

What a closure keeps travels with it: into a list, into another process, and across the network to another node. That is cheap for a number and expensive for a large value. This module makes two closures over the same data, one that needs the whole list of 10,000 prices and one that needs only its length, then measures each one:

# What a closure carries: the values it captured, copied wherever the function goes.
defmodule Carry do
  def run do
    prices = Enum.to_list(1..10_000)
    count = length(prices)

    first_price = fn -> hd(prices) end
    how_many = fn -> count end

    IO.inspect(Function.info(how_many, :env), label: "how_many captured")
    IO.puts("first_price: #{:erlang.external_size(first_price)} bytes as data")
    IO.puts("how_many: #{:erlang.external_size(how_many)} bytes as data")
    IO.puts("a process running first_price: #{memory_of(first_price)} bytes")
    IO.puts("a process running how_many: #{memory_of(how_many)} bytes")
  end

  # Starts a process that runs `fun` and waits, then asks the VM how much memory that process uses.
  defp memory_of(fun) do
    parent = self()

    pid =
      spawn(fn ->
        fun.()
        send(parent, :ready)

        receive do
          :stop -> :ok
        end
      end)

    receive do
      :ready -> :ok
    end

    {:memory, bytes} = Process.info(pid, :memory)
    send(pid, :stop)
    bytes
  end
end

Carry.run()

from session import run

run("""
elixir carry.exs
""")

$ elixir carry.exs
how_many captured: {:env, [10000]}
first_price: 49321 bytes as data
how_many: 85 bytes as data
a process running first_price: 230280 bytes
a process running how_many: 2624 bytes

Function.info(how_many, :env) shows what how_many captured: the single number 10000. The Function documentation calls :env the function’s environment, the outer variables it uses, and its note on Function.info/1 says to use it for debugging only. first_price uses prices, so it captured the whole list. :erlang.external_size/1, the size of a value in the format used to send it to another node, gave 49,321 bytes for first_price against 85 for how_many.

The last two lines start a process that runs each function, which is what Task.async/1 and spawn/1 do, and ask the VM how much memory that process uses. The process with first_price took 230,280 bytes, the one with how_many 2,624, about 88 times less, close to the 327 words, 2,616 bytes, that the Erlang efficiency guide gives for a newly spawned process. Each process has its own heap, and the guide lists the terms passed to a new process among the data that is copied, so every process started with first_price gets its own copy of the list.

The habit that follows: work out what a function needs before you make it, such as count = length(prices) here, and let the closure capture that small value instead of the data it came from.

A capture names a function or writes a short one

Writing fn s -> String.upcase(s) end only to pass String.upcase/1 along is noise. The capture operatorcapture operator: &, which turns a named function into a function value, as in &String.upcase/1, or builds a short function from one expression with numbered arguments, as in &(&1 * 3), where &1 stands for the first argument. & turns a named function into a function value with &String.upcase/1, its module, name and arity. It also builds a short function from an expression with numbered arguments, where &1 stands for the first argument and &2 for the second:

from session import shell

# & captures a named function by name and arity, or builds a short function from &1, &2 ...
# The last two captures are refused: one skips &1, the other holds a capture inside a capture.
shell([
    "upcase = &String.upcase/1",
    "upcase.(\"pune\")",
    "Function.info(upcase, :type)",
    "Enum.map([\"pune\", \"agra\"], &String.upcase/1)",
    "triple = &(&1 * 3)",
    "triple.(14)",
    "Function.info(&(&1 + &2), :arity)",
    "greet = &\"Good #{&1}\"",
    "greet.(\"morning\")",
    "Enum.map([1, 2, 3], &{&1, &1 * &1})",
    "&(&2 + 1)",
    "&(&1 + &(&2))",
    "System.halt()",
])

$ iex
Erlang/OTP 29 [erts-17.1] [source] [64-bit] [smp:18:18] [ds:18:18:10] [async-threads:1] [jit]

Interactive Elixir (1.20.4) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> upcase = &String.upcase/1
&String.upcase/1
iex(2)> upcase.("pune")
"PUNE"
iex(3)> Function.info(upcase, :type)
{:type, :external}
iex(4)> Enum.map(["pune", "agra"], &String.upcase/1)
["PUNE", "AGRA"]
iex(5)> triple = &(&1 * 3)
#Function<42.130099583/1 in :erl_eval.expr/6>
iex(6)> triple.(14)
42
iex(7)> Function.info(&(&1 + &2), :arity)
{:arity, 2}
iex(8)> greet = &"Good #{&1}"
#Function<42.130099583/1 in :erl_eval.expr/6>
iex(9)> greet.("morning")
"Good morning"
iex(10)> Enum.map([1, 2, 3], &{&1, &1 * &1})
[{1, 1}, {2, 4}, {3, 9}]
iex(11)> &(&2 + 1)
error: capture argument &2 cannot be defined without &1 (you cannot skip arguments, all arguments must be numbered)
└─ iex:11

** (CompileError) cannot compile code (errors have been logged)

iex(11)> &(&1 + &(&2))
error: nested captures are not allowed. You cannot define a function using the capture operator & inside another function defined via &. Got invalid nested capture: & &2
└─ iex:11

** (CompileError) cannot compile code (errors have been logged)

iex(11)> System.halt()

&String.upcase/1 printed as itself, and Function.info/2 called it :external: a reference to a function defined in a module, as the Function documentation describes it. &(&1 * 3) is a function of arity 1, and &(&1 + &2) of arity 2, because the highest numbered argument decides the arity. The guide says &(&1 + 1) is exactly the same as fn x -> x + 1 end, and the same shorthand built a string and a tuple in the session:

The last two captures of the session were refused. &(&2 + 1) skips &1, and the compiler wants every argument up to the highest one numbered. &(&1 + &(&2)) puts a capture inside a capture, and the compiler answered that “nested captures are not allowed”. For anything longer than a short expression, write fn, which can also name its arguments and have clauses.

Functions are values: keep them, pass them, return them

A function is a value, so everything that holds values can hold functions. Here are the discount rules of the opening as a map from codes to functions. One rule is a fn, one a capture, and one is made by another function, percent_off/1, which returns a closure over the percentage it was given:

# Discount rules kept as functions: stored in a map, made by a function, passed to Enum.map/2.
defmodule Discounts do
  def percent_off(percent) do
    fn paise -> paise - div(paise * percent, 100) end
  end

  def rules do
    %{
      "NONE" => fn paise -> paise end,
      "TEN" => percent_off(10),
      "FLAT150" => &max(&1 - 15_000, 0)
    }
  end

  def apply_code(code, paise) do
    case Map.fetch(rules(), code) do
      {:ok, rule} -> {:ok, rule.(paise)}
      :error -> {:error, :unknown_code}
    end
  end
end

for code <- ["NONE", "TEN", "FLAT150", "HALF"] do
  IO.inspect(Discounts.apply_code(code, 120_000), label: code)
end

IO.inspect(Enum.map([50_000, 120_000], Discounts.percent_off(20)), label: "20% off each")

from session import run

run("""
elixir discounts.exs
""")

$ elixir discounts.exs
NONE: {:ok, 120000}
TEN: {:ok, 108000}
FLAT150: {:ok, 105000}
HALF: {:error, :unknown_code}
20% off each: [40000, 96000]

Every code took the same path through apply_code/2: find the rule, call it with a dot, wrap the result. “TEN” gave 108000 and “FLAT150” 105000, while “HALF”, which the map does not hold, gave {:error, :unknown_code}. The last line passed the function that percent_off(20) returned straight to Enum.map/2, which applied 20 % off to each price. Step through one lookup:

Map.fetch/2 is what keeps the missing code safe. rules()["HALF"] would return nil, and nil.(120_000) would raise BadFunctionError, the error price.(1) raised. A new rule is now one more entry in the map, and no case anywhere has to change.

Practise: compose and partial

Write two functions that build functions. Fun.compose/1 takes a list of one-argument functions and returns one function that applies them all, the first one first: compose([f, g]) does what fn x -> g.(f.(x)) end does, and compose([]) gives its argument back. Fun.partial/2 takes a two-argument function and a first argument and returns a function of the second: partial(fun, a).(b) gives fun.(a, b). Copy the two files into an empty folder:

defmodule Fun do
  @moduledoc "Building new functions out of other functions."

  @doc """
  One function that applies every function of `funs` in turn, the first one first: compose([f, g]) does what
  fn x -> g.(f.(x)) end does. With no functions, the value comes back unchanged.
  """
  def compose(_funs) do
    # Drop the underscore once you use the argument.
    :todo
  end

  @doc """
  A one-argument function that calls the two-argument function `fun` with `first` as its first argument:
  partial(fun, a).(b) gives fun.(a, b).
  """
  def partial(_fun, _first) do
    :todo
  end
end

Code.require_file("fun.ex", __DIR__)
ExUnit.start(seed: 0)

defmodule FunTest do
  use ExUnit.Case

  test "compose applies the functions from the first to the last" do
    add_one_then_tenfold = Fun.compose([&(&1 + 1), &(&1 * 10)])
    assert add_one_then_tenfold.(2) == 30
  end

  test "compose works with captures of named functions" do
    clean = Fun.compose([&String.trim/1, &String.upcase/1])
    assert clean.("  pune ") == "PUNE"
  end

  test "compose of no functions gives the value back" do
    assert Fun.compose([]).(:same) == :same
  end

  test "partial fixes the first argument" do
    from_ten = Fun.partial(&-/2, 10)
    assert from_ten.(3) == 7
  end

  test "partial returns a function of one argument" do
    assert is_function(Fun.partial(&+/2, 1), 1)
  end

  test "partial works with a named function" do
    repeat_ab = Fun.partial(&String.duplicate/2, "ab")
    assert repeat_ab.(3) == "ababab"
  end
end

Run elixir fun_test.exs. The starter returns :todo instead of a function, so five tests fail with BadFunctionError, and the one that asks is_function/2 fails its assertion:

from session import run

# The tests against the starter, which builds no functions yet.
run("""
cd practice
!elixir fun_test.exs
""")

$ cd practice
$ elixir fun_test.exs
Running ExUnit with seed: 0, max_cases: 36



  1) test compose applies the functions from the first to the last (FunTest)
     fun_test.exs:7
     ** (BadFunctionError) expected a function, got: :todo
     code: assert add_one_then_tenfold.(2) == 30
     stacktrace:
       fun_test.exs:9: (test)



  2) test compose works with captures of named functions (FunTest)
     fun_test.exs:12
     ** (BadFunctionError) expected a function, got: :todo
     code: assert clean.("  pune ") == "PUNE"
     stacktrace:
       fun_test.exs:14: (test)



  3) test compose of no functions gives the value back (FunTest)
     fun_test.exs:17
     ** (BadFunctionError) expected a function, got: :todo
     code: assert Fun.compose([]).(:same) == :same
     stacktrace:
       fun_test.exs:18: (test)



  4) test partial fixes the first argument (FunTest)
     fun_test.exs:21
     ** (BadFunctionError) expected a function, got: :todo
     code: assert from_ten.(3) == 7
     stacktrace:
       fun_test.exs:23: (test)



  5) test partial returns a function of one argument (FunTest)
     fun_test.exs:26
     Expected truthy, got false
     code: assert is_function(Fun.partial(&+/2, 1), 1)
     arguments:

         # 1
         :todo

         # 2
         1

     stacktrace:
       fun_test.exs:27: (test)



  6) test partial works with a named function (FunTest)
     fun_test.exs:30
     ** (BadFunctionError) expected a function, got: :todo
     code: assert repeat_ab.(3) == "ababab"
     stacktrace:
       fun_test.exs:32: (test)


Finished in 0.02 seconds (0.02s on load, 0.00s async, 0.00s sync)

Result: 0/6 passed
Failed: 6 tests
[exit status 2]

For compose/1, return a fn of one value that runs Enum.reduce/3 over the list, with the value as the starting accumulator and each function applied to the result so far; an empty list then leaves the value as it was. For partial/2, return a fn of the second argument that calls the function with both. Both answers are closures: the functions they return keep funs, or fun and first, from the call that made them. A finished version passes all six tests:

from session import run

# The same tests against finished compose/1 and partial/2.
run("""
cd finished
cp ../practice/fun_test.exs .
elixir fun_test.exs
""")

$ cd finished
$ cp ../practice/fun_test.exs .
$ elixir fun_test.exs
Running ExUnit with seed: 0, max_cases: 36

......
Finished in 0.02 seconds (0.02s on load, 0.00s async, 0.00s sync)

Result: 6 passed

To finish, a self-check on calls, clauses, closures, what closures carry, captures and functions as data. Whichever of those your answers show as shakiest is asked about first, with every choice explained right after you make it:

Interview questions

Warm-up: what is a closure in Elixir, and what does it capture?

A closure is a function that keeps values from the place where it was made. When fn creates a function that uses a name from the code around it, such as rate in fn paise -> paise - div(paise * rate, 100) end, the function keeps the value that name had at that moment, 10 here. Binding rate again later, to 50, gives the outer name a new value and leaves the function’s copy alone, so the call still takes 10 % off. The rule works in the other direction too: a name bound inside the function, such as total = total + amount, stays inside that call. Because it keeps values, a closure over a large list carries the whole list into every process that runs it, so capture the small value you need, such as the list’s length.

Two questions from code that passes functions around:

Key takeaways

• fn makes a function value, and a name bound to one is called with a dot, double.(5); without the dot Elixir looks for a named function instead.
• A call fails with BadFunctionError when the value is not a function, BadArityError when the number of arguments differs, and FunctionClauseError when no clause fits.
• An anonymous function may have several clauses with patterns and guards, tried from the top, and all of them take the same number of arguments.
• A closure keeps the values its names had when it was made: binding a name again later changes nothing in it, and a name bound inside it stays inside.
• What a closure captures travels with it, so a closure over a 10,000-item list took 230,280 bytes in a new process against 2,624 for one over the list’s length.
• &Mod.fun/arity captures a named function and &(&1 * 3) writes a short one, and functions are values that maps can hold, Enum can call and functions can return.