Behaviours and dynamic dispatch

A library sends due-date reminders by SMS today, and wants email and a phone app next year: three ways of doing one job that the code deciding whom to remind should not care about. A behaviour writes that job down as a list of functions, deliver/2 among them, that every notifier module promises to define, and the compiler warns when a module breaks the promise.

Below, you will declare the behaviour and implement it, read the warnings for the usual slips, choose the notifier while the program runs, test reminders without sending any, see that GenServer is a behaviour too, and choose between a behaviour, a protocol and a plain function. Every run below used Elixir 1.20.4 and Erlang/OTP 29.1.1.

A behaviour lists the functions a module promises to define

The typespecs reference puts it in one sentence: “A behaviour module defines a set of functions and macros (referred to as callbacks) that callback modules implementing that behaviour must export”. Each callbackcallback: A function that a behaviour declares with @callback and that each module adopting the behaviour defines, unless it is listed in @optional_callbacks; the code that uses the behaviour calls it back. A missing one gets a warning when the module compiles. is declared with @callback and a type specification. Notifier declares two, and makes the second optional with @optional_callbacks, since not every channel has a length limit:

defmodule Notifier do
  @moduledoc "Sends a short message to a member of the library."

  @doc "Delivers body to the member at the address to."
  @callback deliver(to :: String.t(), body :: String.t()) :: :ok | {:error, term()}

  @doc "The longest body this channel takes, in characters."
  @callback max_length() :: pos_integer()

  @optional_callbacks max_length: 0
end

A callback modulecallback module: A module that adopts a behaviour with @behaviour and defines its callbacks, such as an SMS notifier for a Notifier behaviour, or your own module for GenServer. adopts the behaviour with @behaviour Notifier and defines the callbacks. SmsNotifier keeps the promise; EmailNotifier, written by someone who named the function after what it does, does not:

defmodule SmsNotifier do
  @behaviour Notifier

  @impl true
  def deliver(to, body) do
    IO.puts("SMS to #{to}: #{body}")
    :ok
  end

  @impl true
  def max_length, do: 160
end

# An email notifier whose author named the function after what it does.
defmodule EmailNotifier do
  @behaviour Notifier

  def send_mail(to, body) do
    IO.puts("Mail to #{to}: #{body}")
    :ok
  end
end

from session import run

# The behaviour and a module that keeps its promise; then a module that does not.
run("""
cd notify
elixirc -o out notifier.ex sms.ex
elixir -pa out -e 'IO.inspect(Notifier.behaviour_info(:callbacks)); IO.inspect(Notifier.behaviour_info(:optional_callbacks))'
elixir -pa out -e 'SmsNotifier.deliver("Asha", "Gora is due back in 3 days.")'
elixirc -o out_email notifier.ex email.ex
""")

$ cd notify
$ elixirc -o out notifier.ex sms.ex
$ elixir -pa out -e 'IO.inspect(Notifier.behaviour_info(:callbacks)); IO.inspect(Notifier.behaviour_info(:optional_callbacks))'
[max_length: 0, deliver: 2]
[max_length: 0]
$ elixir -pa out -e 'SmsNotifier.deliver("Asha", "Gora is due back in 3 days.")'
SMS to Asha: Gora is due back in 3 days.
$ elixirc -o out_email notifier.ex email.ex
    warning: function deliver/2 required by behaviour Notifier is not implemented (in module EmailNotifier)
    │
  2 │ defmodule EmailNotifier do
    │ ~~~~~~~~~~~~~~~~~~~~~~~~~~
    │
    └─ email.ex:2: EmailNotifier (module)

@callback and @optional_callbacks built Notifier.behaviour_info/1, which listed both callbacks and the optional one. SmsNotifier.deliver/2 sent its message. Compiling email.ex printed “function deliver/2 required by behaviour Notifier is not implemented (in module EmailNotifier)”, and nothing about max_length/0, because it is optional. Step through the check:

@impl makes the compiler check each callback by name and arity

The missing-callback check cannot tell a callback written wrongly from a helper that happens to look like one. @impl true before a function says “this is a callback”, and the compiler checks the claim. The Module documentation lists the values: “You may pass either false, true, or a specific behaviour to @impl.” Three versions of EmailNotifier with one slip each, compiled one by one:

# The callback's name is right, but it takes one argument instead of two.
defmodule EmailNotifier do
  @behaviour Notifier

  @impl true
  def deliver(message) do
    IO.puts("Mail: #{inspect(message)}")
    :ok
  end
end

# One callback is marked with @impl, the other is not.
defmodule EmailNotifier do
  @behaviour Notifier

  @impl true
  def max_length, do: 5_000

  def deliver(to, body) do
    IO.puts("Mail to #{to}: #{body}")
    :ok
  end
end

# @impl on a helper that no behaviour declares.
defmodule EmailNotifier do
  @behaviour Notifier

  @impl true
  def deliver(to, body) do
    IO.puts("Mail to #{to}: #{tidy(body)}")
    :ok
  end

  @impl true
  def tidy(body), do: String.trim(body)
end

from session import run

# Three slips in EmailNotifier, each compiled with the behaviour on its own.
run("""
cd notify
elixirc -o out_arity notifier.ex email_arity.ex
elixirc -o out_unmarked notifier.ex email_unmarked.ex
elixirc -o out_helper notifier.ex email_helper.ex
""")

$ cd notify
$ elixirc -o out_arity notifier.ex email_arity.ex
    warning: function deliver/2 required by behaviour Notifier is not implemented (in module EmailNotifier)
    │
  2 │ defmodule EmailNotifier do
    │ ~~~~~~~~~~~~~~~~~~~~~~~~~~
    │
    └─ email_arity.ex:2: EmailNotifier (module)

    warning: got "@impl true" for function deliver/1 but no behaviour specifies such callback. The known callbacks are:

      * Notifier.deliver/2 (function)
      * Notifier.max_length/0 (function)

    │
  6 │   def deliver(message) do
    │   ~~~~~~~~~~~~~~~~~~~~~~~
    │
    └─ email_arity.ex:6: EmailNotifier (module)
$ elixirc -o out_unmarked notifier.ex email_unmarked.ex
    warning: module attribute @impl was not set for function deliver/2 callback (specified in Notifier). This either means you forgot to add the "@impl true" annotation before the definition or that you are accidentally overriding this callback
    │
  8 │   def deliver(to, body) do
    │   ~~~~~~~~~~~~~~~~~~~~~~~~
    │
    └─ email_unmarked.ex:8: EmailNotifier (module)
$ elixirc -o out_helper notifier.ex email_helper.ex
    warning: got "@impl true" for function tidy/1 but no behaviour specifies such callback. The known callbacks are:

      * Notifier.deliver/2 (function)
      * Notifier.max_length/0 (function)

    │
 12 │   def tidy(body), do: String.trim(body)
    │   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    │
    └─ email_helper.ex:12: EmailNotifier (module)

The second warning comes from a rule the Module documentation states: “If you mark one function with @impl, you must mark all other callbacks for that behaviour as @impl.” So once a module uses @impl, every callback carries the mark and a function without it is not a callback, and a reader of the module sees the contract at a glance. @impl true also “automatically marks the function as @doc false”, because the behaviour documents its callbacks.

A module is a value, so the caller can choose it

A module name such as SmsNotifier is an atom, so it can be passed around, stored and put in a variable, and notifier.deliver(name, body) calls deliver/2 on whichever module the variable holds. The typespecs reference says “Behaviours are useful because you can pass modules around as arguments”. That is dynamic dispatchdynamic dispatch: Calling a function on a module chosen while the program runs, as in notifier.deliver(to, body), where notifier holds a module name passed in as an argument or read from configuration., and Reminders.send_due/2 takes the module as an argument whose default reads the application environment:

defmodule Reminders do
  @body "Your book is due back in 3 days."

  # The notifier is an argument. When the caller gives none, the :library
  # application's environment decides, read each time the function runs.
  def send_due(names, notifier \\ notifier()) do
    for name <- names, do: {name, notifier.deliver(name, @body)}
  end

  defp notifier, do: Application.get_env(:library, :notifier, SmsNotifier)
end

from session import run

# One function, three ways of choosing the notifier; then a module name spelt wrong.
run("""
cd notify
elixirc -o out notifier.ex sms.ex log.ex reminders.ex
elixir -pa out -e 'IO.inspect(Reminders.send_due(["Asha", "Dev"]))'
elixir -pa out -e 'IO.inspect(Reminders.send_due(["Asha"], LogNotifier))'
elixir -pa out -e 'Application.put_env(:library, :notifier, LogNotifier); IO.inspect(Reminders.send_due(["Dev"]))'
elixir -e 'IO.inspect(is_atom(SmsNotifier)); IO.inspect(SmsNotifier == :"Elixir.SmsNotifier")'
elixirc -o out_typo typo.ex
!elixir -pa out -e 'Application.put_env(:library, :notifier, SmsNotifer); Reminders.send_due(["Asha"])'
""")

$ cd notify
$ elixirc -o out notifier.ex sms.ex log.ex reminders.ex
$ elixir -pa out -e 'IO.inspect(Reminders.send_due(["Asha", "Dev"]))'
SMS to Asha: Your book is due back in 3 days.
SMS to Dev: Your book is due back in 3 days.
[{"Asha", :ok}, {"Dev", :ok}]
$ elixir -pa out -e 'IO.inspect(Reminders.send_due(["Asha"], LogNotifier))'
[log] Asha: Your book is due back in 3 days.
[{"Asha", :ok}]
$ elixir -pa out -e 'Application.put_env(:library, :notifier, LogNotifier); IO.inspect(Reminders.send_due(["Dev"]))'
[log] Dev: Your book is due back in 3 days.
[{"Dev", :ok}]
$ elixir -e 'IO.inspect(is_atom(SmsNotifier)); IO.inspect(SmsNotifier == :"Elixir.SmsNotifier")'
true
true
$ elixirc -o out_typo typo.ex
    warning: SmsNotifer.deliver/2 is undefined (module SmsNotifer is not available or is yet to be defined). Make sure the module name is correct and has been specified in full (or that an alias has been defined)
    │
  3 │   def direct(name), do: SmsNotifer.deliver(name, "hi")
    │                                    ~
    │
    └─ typo.ex:3:36: Typo.direct/1
    └─ typo.ex:7:14: Typo.through_a_variable/1
$ elixir -pa out -e 'Application.put_env(:library, :notifier, SmsNotifer); Reminders.send_due(["Asha"])'
** (UndefinedFunctionError) function SmsNotifer.deliver/2 is undefined (module SmsNotifer is not available). Make sure the module name is correct and has been specified in full (or that an alias has been defined)
    SmsNotifer.deliver("Asha", "Your book is due back in 3 days.")
    reminders.ex:7: anonymous fn/2 in Reminders.send_due/2
    (stdlib 8.1) erl_eval.erl:1048: :erl_eval.do_apply/7
[exit status 1]

Without an argument and with nothing configured, the default answered SmsNotifier. Given LogNotifier, the same line of code called LogNotifier.deliver/2. After Application.put_env/3 set the notifier, a call without an argument used it, because notifier/0 reads the environment each time it runs; the lesson on module attributes showed why a value read into an attribute would have stayed as it was when the module compiled. is_atom(SmsNotifier) was true, and the name equals the atom :"Elixir.SmsNotifier". Step through the calls:

The last two commands show what dynamic dispatch costs. A misspelt module written in the code, SmsNotifer, was found when typo.ex compiled, even through a variable, because the compiler could see the name. The same misspelt name set in the configuration was found only when a reminder was sent, as UndefinedFunctionError. Check configured module names when the application starts, so a bad one fails at once instead of at the first reminder.

A test double is one more implementation of the behaviour

Because send_due/2 calls whatever notifier it is given, a test can give it one that sends nothing. A test doubletest double: A module that stands in for a real one in tests: it implements the same behaviour but records or fakes what it does, such as a notifier that sends each message to the test process instead of delivering it. here is a module that adopts Notifier and, instead of delivering, tells the test what it was asked to deliver. TestNotifier sends a message to self(): it runs in the process that called it, the test’s own, so the message lands in the test’s mailbox:

Code.require_file("notify/notifier.ex", __DIR__)
Code.require_file("notify/reminders.ex", __DIR__)
ExUnit.start(seed: 0)

# A notifier for tests: it delivers nothing, and tells the test process what it
# was asked to deliver. It runs in the process that calls it, here the test's.
defmodule TestNotifier do
  @behaviour Notifier

  @impl true
  def deliver(to, body) do
    send(self(), {:delivered, to, body})
    :ok
  end
end

# A notifier whose channel is down.
defmodule DownNotifier do
  @behaviour Notifier

  @impl true
  def deliver(_to, _body), do: {:error, :unreachable}
end

defmodule RemindersTest do
  use ExUnit.Case, async: true

  test "one reminder for each name, with the due-date message" do
    Reminders.send_due(["Asha", "Dev"], TestNotifier)
    assert_received {:delivered, "Asha", "Your book is due back in 3 days."}
    assert_received {:delivered, "Dev", _body}
    refute_received {:delivered, _, _}
  end

  test "each name comes back with what its notifier answered" do
    assert Reminders.send_due(["Asha"], DownNotifier) == [{"Asha", {:error, :unreachable}}]
  end
end

from session import run

run("""
elixir reminders_test.exs
""")

$ elixir reminders_test.exs
Running ExUnit with seed: 0, max_cases: 36

..
Finished in 0.5 seconds (0.4s on load, 0.02s async, 0.00s sync)

Result: 2 passed

Both tests passed, and no reminder went anywhere. The first checked that each name got exactly one reminder, with assert_received for each and refute_received for any third; the second used a double whose channel is down and checked that the error came back with the name. Step through the first test:

The module goes in as an argument, so each test chooses its own and async: true is safe. Setting the notifier with Application.put_env/3 instead would change the environment for every test running at the same moment, and the ExUnit documentation says async “should be enabled only if tests do not change any global state”. Mox, a third-party library on Hex, builds such doubles from a behaviour for you; its documentation says “You can only create mocks based on behaviours”.

GenServer is a behaviour, and use GenServer adopts it

The behavioursbehaviour: The generic part of a kind of process, such as gen_server or supervisor, written once in OTP. Your callback module supplies only the specific part. More in “The BEAM, Elixir and Erlang/OTP explained” of OTP use exactly this mechanism. GenServer is generic code for a server process, and your module supplies the callbacks; GenServer.behaviour_info/1 lists them:

# GenServer is a behaviour: its callbacks, the optional ones, and what is left.
callbacks = GenServer.behaviour_info(:callbacks)
optional = GenServer.behaviour_info(:optional_callbacks)
IO.inspect(Enum.sort(callbacks), label: "callbacks")
IO.inspect(Enum.sort(callbacks -- optional), label: "required")

# A GenServer callback module without init/1.
defmodule Counter do
  use GenServer

  @impl true
  def handle_call(:get, _from, count), do: {:reply, count, count}
end

from session import run

# The callbacks of GenServer, then a callback module that leaves out the required one.
run("""
cd server
elixir callbacks.exs
elixirc -o out counter.ex
elixir -pa out -e 'IO.inspect(Counter.__info__(:attributes)[:behaviour], label: "behaviour")'
""")

$ cd server
$ elixir callbacks.exs
callbacks: [
  code_change: 3,
  format_status: 1,
  format_status: 2,
  handle_call: 3,
  handle_cast: 2,
  handle_continue: 2,
  handle_info: 2,
  init: 1,
  terminate: 2
]
required: [init: 1]
$ elixirc -o out counter.ex
    warning: function init/1 required by behaviour GenServer is not implemented (in module Counter).

    We will inject a default implementation for now:

        def init(init_arg) do
          {:ok, init_arg}
        end

    You can copy the implementation above or define your own that converts the arguments given to GenServer.start_link/3 to the server state.

    │
  2 │ defmodule Counter do
    │ ~~~~~~~~~~~~~~~~~~~~
    │
    └─ counter.ex:2: Counter (module)
$ elixir -pa out -e 'IO.inspect(Counter.__info__(:attributes)[:behaviour], label: "behaviour")'
behaviour: [GenServer]

behaviour_info/1 listed nine callbacks, and all but init/1 are optional. The GenServer documentation counts eight, because format_status/2 is deprecated in favour of format_status/1:

use GenServer wrote @behaviour GenServer into Counter, as its attributes showed, so the compiler checked Counter as it checked EmailNotifier; the GenServer documentation adds that it also defines child_spec/1, so the module “can be used as a child in a supervision tree”. Counter has no init/1, and the warning came with a default that Elixir injected for now, {:ok, init_arg}, and the advice to write one.

Because almost every GenServer callback is optional, @impl true matters more here than for Notifier. A server whose handle_call/3 is misspelt handle_cal/3 gets no missing-callback warning, since handle_call/3 may be left out, and the GenServer documentation says such a server “will fail if a call is performed against it”. With @impl true before it, the compiler reports the name at once, as it reported tidy/1. The OTP module of this track builds servers on these callbacks.

Choose by what decides the code: the value, the caller or the configuration

A protocol, a behaviour and a plain function all let one call run different code; they differ in who decides. The design anti-patterns guide adds a rule for libraries: the application environment “is a global state”, so options belong in arguments, and configuration fits only to swap a component that “must behave the exact same”, which “library authors can even enforce” with a behaviour. Answer the questions about the code you need to vary:

In short: a protocol when the type of the value decides, as Weight.grams/1 and Enumerable do; a behaviour when the caller or the deployment decides among modules that each provide several functions, passed as an argument wherever tests must run at the same time; and a plain function argument when only one function varies, as with Enum.map/2.

Practise: a cache with two stores

A cache keeps computed values in a store, and the store may be memory or a folder on disk. The Storage behaviour is written, with put/3 and get/2; MemoryStorage keeps a map, DiskStorage writes one file for each key, and Cache.fetch/4 takes the adapter as an argument. Copy the two files into an empty folder:

# A cache that can keep its values in memory or on disk. The Storage behaviour is
# written; finish the two adapters and Cache.fetch/4.
defmodule Storage do
  @doc "Stores value under key in store, and returns the store to use next."
  @callback put(store :: term(), key :: String.t(), value :: binary()) :: term()

  @doc "The value stored under key, or :error when there is none."
  @callback get(store :: term(), key :: String.t()) :: {:ok, binary()} | :error
end

defmodule MemoryStorage do
  # The store is a map from keys to values.
  @behaviour Storage

  @impl true
  def put(store, _key, _value), do: store

  @impl true
  def get(_store, _key), do: :error
end

defmodule DiskStorage do
  # The store is the path of a folder; each value is a file named after its key.
  @behaviour Storage

  @impl true
  def put(dir, _key, _value), do: dir

  @impl true
  def get(_dir, _key), do: :error
end

defmodule Cache do
  # The value under key in store, through adapter: {value, store}. When there is
  # none, compute.() makes it, the adapter stores it, and the new store comes back.
  def fetch(_adapter, store, _key, compute), do: {compute.(), store}
end

Code.require_file("storage.ex", __DIR__)
ExUnit.start(seed: 0)

defmodule StorageTest do
  use ExUnit.Case, async: true
  # Each test gets an empty folder of its own, under tmp/ beside this file.
  @moduletag :tmp_dir

  # A compute function that tells the test process each time it runs.
  defp compute do
    fn ->
      send(self(), :computed)
      "Premchand"
    end
  end

  test "memory: a value comes back under its key" do
    store = MemoryStorage.put(%{}, "gora", "Rabindranath Tagore")
    assert MemoryStorage.get(store, "gora") == {:ok, "Rabindranath Tagore"}
    assert MemoryStorage.get(store, "godaan") == :error
  end

  test "disk: a value comes back under its key", %{tmp_dir: dir} do
    assert DiskStorage.put(dir, "gora", "Rabindranath Tagore") == dir
    assert DiskStorage.get(dir, "gora") == {:ok, "Rabindranath Tagore"}
    assert DiskStorage.get(dir, "godaan") == :error
  end

  test "fetch through memory computes the value once" do
    {"Premchand", store} = Cache.fetch(MemoryStorage, %{}, "godaan", compute())
    {"Premchand", _store} = Cache.fetch(MemoryStorage, store, "godaan", compute())
    assert_received :computed
    refute_received :computed
  end

  test "fetch through disk computes the value once", %{tmp_dir: dir} do
    {"Premchand", ^dir} = Cache.fetch(DiskStorage, dir, "godaan", compute())
    {"Premchand", ^dir} = Cache.fetch(DiskStorage, dir, "godaan", compute())
    assert_received :computed
    refute_received :computed
  end
end

Run elixir storage_test.exs. The starter’s placeholders store nothing and compute every time, so all four tests fail:

from session import run

# The tests against the starter: the adapters have no functions yet.
run("""
cd practice
!elixir storage_test.exs
""")

$ cd practice
$ elixir storage_test.exs
Running ExUnit with seed: 0, max_cases: 36



  1) test memory: a value comes back under its key (StorageTest)
     storage_test.exs:17
     Assertion with == failed
     code:  assert MemoryStorage.get(store, "gora") == {:ok, "Rabindranath Tagore"}
     left:  :error
     right: {:ok, "Rabindranath Tagore"}
     stacktrace:
       storage_test.exs:19: (test)



  2) test disk: a value comes back under its key (StorageTest)
     storage_test.exs:23
     Assertion with == failed
     code:  assert DiskStorage.get(dir, "gora") == {:ok, "Rabindranath Tagore"}
     left:  :error
     right: {:ok, "Rabindranath Tagore"}
     stacktrace:
       storage_test.exs:25: (test)



  3) test fetch through memory computes the value once (StorageTest)
     storage_test.exs:29
     Unexpectedly received message :computed (which matched :computed)
     code: refute_received :computed
     stacktrace:
       storage_test.exs:33: (test)



  4) test fetch through disk computes the value once (StorageTest)
     storage_test.exs:36
     Unexpectedly received message :computed (which matched :computed)
     code: refute_received :computed
     stacktrace:
       storage_test.exs:40: (test)


Finished in 0.6 seconds (0.2s on load, 0.4s async, 0.00s sync)

Result: 0/4 passed
Failed: 4 tests
[exit status 2]

MemoryStorage needs two calls of the Map module, one to put and one to fetch. DiskStorage writes the value to Path.join(dir, key) and reads it back, turning a missing file into :error. Cache.fetch/4 asks adapter.get/2 first; on :error it calls compute.(), stores the value with adapter.put/3 and returns it with the new store. The same fetch/4 then works with both adapters, and with any third one that keeps the promise. A finished version passes all four:

from session import run

# The same tests against finished adapters and Cache.fetch/4.
run("""
cd finished
cp ../practice/storage_test.exs .
elixir storage_test.exs
""")

$ cd finished
$ cp ../practice/storage_test.exs .
$ elixir storage_test.exs
Running ExUnit with seed: 0, max_cases: 36

....
Finished in 0.6 seconds (0.3s on load, 0.3s async, 0.00s sync)

Result: 4 passed

Then work through the self-check: it starts from the idea your answers show you know least, asks one question at a time, and explains each choice as soon as you make it:

Interview questions

Warm-up: what is the difference between a protocol and a behaviour in Elixir?

Both let one call run different code, but they choose it differently. A protocol dispatches on data: Weight.grams(item) runs the implementation for the type of item, so a new type joins by implementing the protocol, anywhere. A behaviour is a contract for modules: @callback lists the functions a module promises to define, the compiler warns when one is missing, and the caller picks the module, through an argument or the configuration, and calls notifier.deliver(to, body). So a protocol fits when the kind of value decides, as for Enumerable or String.Chars; a behaviour fits when the caller or the deployment decides, as for a GenServer’s callback module or an adapter that tests replace with a double.

Two questions from code that swaps its parts:

Key takeaways

• A behaviour lists its callbacks with @callback, and every module that says @behaviour is checked against the list when it compiles, with optional callbacks left out of the check.
• @impl true makes the compiler check each callback’s name and arity, and once one function uses it, every callback of the module must.
• A module name is an atom, so notifier.deliver(to, body) runs whichever module the caller passed or the configuration names, read while the code runs; a misspelt name from the configuration fails only then.
• A test double is one more callback module, passed as an argument, which keeps global state out of the tests and makes them safe to run with async: true.
• OTP’s behaviours work the same way: use GenServer adopts GenServer, whose only required callback is init/1.
• Choose a protocol when the value’s type decides, a behaviour when the caller or the deployment picks among modules with several functions, and a function argument when only one function varies.