Testing Bash scripts with Bats

A Bash script gets edited long after it was written, and one changed line can break it with nothing to say so until it runs for real. Three Bats tests are enough to pin down what a small greet function must do: bats --formatter tap greet.bats prints 1..3, then one ok line for each behaviour that still holds. Bats-core 1.14.0 gives you the parts of a suite worth keeping: @test blocks that read like sentences, a run helper that captures a command’s status and output, setup and teardown around every test, and a fresh process with its own empty folder for each one.

Below you will write a @test block for a greet function, read $status and $output from run, isolate tests with setup, teardown and $BATS_TEST_TMPDIR, and stub curl offline by putting a fake first on PATH. The scripts have a main guardmain guard: The test at the end of a script, if [[ ${BASH_SOURCE[0]} == "$0" ]]; then main "$@"; fi, which runs main only when the file is executed as a program and skips it when a shell sources the file. More in “Script structure: main, source and libraries” so a test file can sourcesourcing: Running a file's commands in the current shell with source file or . file, so its assignments, functions and cd stay after the file returns; return N inside the file ends it early. More in “Script structure: main, source and libraries” them without main running.

A @test block is one test; a file prints TAP

A BatsBats: A testing framework for Bash scripts (bats-core, MIT licence). A .bats file is Bash plus @test blocks, which Bats turns into functions and runs one by one; its TAP report has a plan line and one ok or not ok line per test. file is Bash with one extra form, @test "name" { ... }, which marks a test; Bats turns each block into a function before it runs it. Bats-core 1.14.0 reads the file, counts the @test blocks, prints a TAPTest Anything Protocol: A plain-text format for test results that most CI systems read: a plan line of the form 1..N, then one ok or not ok line per test, with optional # diagnostic lines. Bats prints it whenever its output goes to a file, a pipe or a CI log, and with --formatter tap. plan line, runs each block in order and prints ok N name or not ok N name for every one. The suite below tests a tiny greet.sh:

#!/usr/bin/env bash
# greet.sh: prints "Namaste, NAME!" with "friend" as the default.
# A test file can source this script without main running, because the
# main guard at the bottom only calls main when the file is run directly.

greet() {
  local name=${1:-friend}
  printf 'Namaste, %s!\n' "$name"
}

main() {
  greet "$@"
}

if [[ ${BASH_SOURCE[0]} == "$0" ]]; then
  main "$@"
fi

#!/usr/bin/env bats
# Tests for greet.sh. Each @test block is one test; run executes a command and
# records its exit status, standard output and the lines of that output.

setup() {
  source "${BATS_TEST_DIRNAME}/greet.sh"
}

@test "greet with no name uses friend" {
  run greet
  [ "$status" -eq 0 ]
  [ "$output" = "Namaste, friend!" ]
}

@test "greet with a name uses that name" {
  run greet Priya
  [ "$status" -eq 0 ]
  [ "$output" = "Namaste, Priya!" ]
}

@test "running the script prints the greeting" {
  run bash "${BATS_TEST_DIRNAME}/greet.sh" Meera
  [ "$status" -eq 0 ]
  [ "${lines[0]}" = "Namaste, Meera!" ]
}

Run the file with bats --formatter tap greet.bats. TAP is what Bats prints whenever its output goes to a file, a pipe or a CI log; the flag makes that choice explicit, and bats -p asks for the pretty view instead, a tick per test and a “3 tests, 0 failures” line. The recorded run:

from session import files, run

# Runs the Bats suite for greet.sh, the two files shown above. --formatter tap asks for
# TAP whatever the output is connected to, so this record reads the same on every machine
# and in every CI log; bats -p would ask for the pretty view instead.
files("greet.sh", "greet.bats")
run("""
bats --formatter tap greet.bats
""")

$ bats --formatter tap greet.bats
1..3
ok 1 greet with no name uses friend
ok 2 greet with a name uses that name
ok 3 running the script prints the greeting

The plan line 1..3 says how many tests will run, and each ok line is one passing test. A failing test starts with not ok, and the # lines under it give the test file and the line where it stopped, such as (in test file greet.bats, line 18), then the command that failed. When the failure happens inside a shell functionshell function: A named group of commands defined with name() { …; } or function name { …; }, called like a command. It runs in the current shell, shares its variables unless local is used, and returns a status with return. More in “Functions: arguments, local and return status”, they name that function and its line first. TAP goes back to t/TEST, the test script that came with Perl 1. Bats writes the older form, without a version line: --formatter tap13 adds TAP version 13 for a reader that wants one, and --formatter junit prints JUnit XML for a CI system that reads that format instead.

run captures status and output, so assertions are plain Bash

Bats runs every @test with set -e, so a command that ends with a non-zero exit statusexit status: The number from 0 to 255 every command ends with, kept by the shell in $?. 0 means success; Bash uses 126 for found but not executable and 127 for not found. More in “Your first script: shebang, chmod and running it” ends the test with not ok. That is a problem for the commands you are testing, because a command under test may be meant to fail. run cmd ... solves it: Bats runs the command in a subshell, saves its exit status in $status, its combined output in $output and that output split by newline in the array ${lines[@]}, leaving out empty lines unless you add --keep-empty-lines, and then returns 0 itself, so the next assertiontest assertion: A check in a test that states what must be true for the test to pass. In Bats it is an ordinary command such as [ "$status" -eq 0 ], and its non-zero exit status ends the test with not ok. runs. The assertions are ordinary [ ... ] tests on those variables. Because the command ran in a subshell, a variable it sets is gone once run returns; call a function directly when that variable is what you want to test.

The greet.bats tests show the three shapes you need often: [ "$status" -eq 0 ] for the exit code, [ "$output" = "Namaste, friend!" ] for the whole output, and [ "${lines[0]}" = "Namaste, Meera!" ] for one line of many. For a command whose standard errorstandard error: Descriptor 2, where a command writes its warnings and error messages, kept apart from its results so that they still show when standard output goes to a file or a pipe. Also called stderr. More in “Redirection: stdin, stdout, stderr, /dev/null” is the point, add --separate-stderr: Bats-core 1.5.0 added it, and test 2 of the stub-curl suite below uses it. Step through that test to see what run keeps when the command fails on purpose:

After run, these are the variables a test reads, with the assertion each one usually feeds:

setup, teardown and a fresh folder isolate every test

Bats runs each @test block in a fresh Bash process of its own, so variables, the current folder and traps from one test cannot leak into the next: a cd / in test 1 still leaves test 2 in the folder bats was started from. setup runs before every test and teardown runs after, in that same process; setup_file and teardown_file run once around the whole file, in a process of their own, so only what setup_file exports reaches the tests. The environment variableenvironment variable: A variable that has been exported, so every program the shell starts receives a copy of it in its environment, the list of name=value pairs a process begins with. More in “Variables, export and the environment” $BATS_TEST_TMPDIR names a new, empty folder made for the current test: what other tests wrote in their own folders is not in it, and Bats removes all those folders when the run ends. Test isolationtest isolation: The property that one test's actions cannot change another test's result. Bats runs each @test in a fresh process of its own with a new, empty $BATS_TEST_TMPDIR, so variables, the current folder and traps do not carry over from one test to the next. in Bats covers the process and that folder; a file written anywhere else is still there for the next test.

Walk through the lifecycle for the first @test of greet.bats:

The source line in step 4 is what the main guard is for. source "${BATS_TEST_DIRNAME}/greet.sh" reads the file into the test’s shell so you can call greet directly, but greet.sh’s own [[ ${BASH_SOURCE[0]} == "$0" ]] is false there ($0 is Bats’s own bats-exec-test program), so main does not run. See sourcing and script structure for the template every script in this module uses.

Stub an external command with a fake first on PATH

A command stubcommand stub: A fake program with a real command's name, placed first on PATH so the shell finds it instead of the real one. A stub lets a test decide what the script sees from curl, date or any external tool, so the suite needs no network and gives the same result every run. is a fake program with the real one’s name, placed first on PATH so the shell finds it instead: for a name that is not a function or a builtin, Bash searches the folders of PATH in order and runs the first executable file with that name. For a script that calls curl, the stub writes a seeded body to the file it was told to and prints a seeded status code, so the test proves the script’s own logic without touching the network. The stub lives in $BATS_TEST_TMPDIR/bin so each test’s PATH is clean when the next one starts; the real curl is still on PATH, behind the stub.

#!/usr/bin/env bash
# download.sh URL: fetches URL with curl. Prints the body on a 2xx response.
# On any other HTTP status it writes a message to standard error and exits 1.
# A transport error from curl exits 2.

download() {
  local url=$1 tmp http
  tmp=$(mktemp)
  http=$(curl --silent --output "$tmp" --write-out '%{http_code}' "$url") || {
    rm -f "$tmp"
    return 2
  }
  if [[ $http != 2* ]]; then
    rm -f "$tmp"
    printf 'download: %s returned %s\n' "$url" "$http" >&2
    return 1
  fi
  cat "$tmp"
  rm -f "$tmp"
}

main() {
  download "$@"
}

if [[ ${BASH_SOURCE[0]} == "$0" ]]; then
  main "$@"
fi

#!/usr/bin/env bats
# Tests download.sh offline. A fake curl, first on PATH, writes a seeded body
# and status, so the suite runs without touching the network.

bats_require_minimum_version 1.5.0

setup() {
  bindir="${BATS_TEST_TMPDIR}/bin"
  mkdir -p "$bindir"
  cat > "$bindir/curl" <<'STUB'
#!/usr/bin/env bash
# Fake curl: --output writes MOCK_CURL_BODY to that file; --write-out prints
# MOCK_CURL_STATUS. Only the arguments download.sh uses are read.
set -euo pipefail
out=
while [ $# -gt 0 ]; do
  case $1 in
    --silent)    shift ;;
    --output)    out=$2; shift 2 ;;
    --write-out) shift 2 ;;
    *)           shift ;;
  esac
done
printf '%s' "${MOCK_CURL_BODY-OK}" > "$out"
printf '%s' "${MOCK_CURL_STATUS-200}"
STUB
  chmod +x "$bindir/curl"
  PATH=$bindir:$PATH
  source "${BATS_TEST_DIRNAME}/download.sh"
}

@test "a 2xx response prints the body" {
  MOCK_CURL_BODY=hello MOCK_CURL_STATUS=200 run download http://example.test/
  [ "$status" -eq 0 ]
  [ "$output" = "hello" ]
}

@test "a 500 goes to standard error and the exit status is 1" {
  MOCK_CURL_STATUS=500 run --separate-stderr download http://example.test/
  [ "$status" -eq 1 ]
  [ -z "$output" ]
  [ "$stderr" = "download: http://example.test/ returned 500" ]
}

@test "the curl the script finds is this test's stub" {
  [ "$(command -v curl)" = "$BATS_TEST_TMPDIR/bin/curl" ]
}

from session import files, run

# Runs stub-curl.bats, shown above, against download.sh. The fake curl it writes is the
# only curl the tests reach, so the run needs no network.
files("download.sh", "stub-curl.bats")
run("""
bats --formatter tap stub-curl.bats
""")

$ bats --formatter tap stub-curl.bats
1..3
ok 1 a 2xx response prints the body
ok 2 a 500 goes to standard error and the exit status is 1
ok 3 the curl the script finds is this test's stub

The three tests show what offline testing is for: the body test proves download returns a 2xx body as it is; the 500 test proves the error message goes to standard error and the exit status is 1; the third test is a test fixturetest fixture: A known starting state built before a test and torn down after it: a seeded file, a stub command on PATH or an environment variable. In Bats, setup builds a per-test fixture and setup_file a per-file one. check, in which command -v curl must name the stub in this test’s own $BATS_TEST_TMPDIR/bin, so a broken PATH line fails a test whose name says what went wrong, while the other two would only show the symptom. The bats_require_minimum_version 1.5.0 line at the top says the file needs Bats 1.5.0 or later, the release that added run --separate-stderr. The function itself arrived in Bats 1.7.0, which also warns about a run flag used without it; an older Bats does not know the function, so the call fails instead of letting the suite pass by mistake.

Prove the suite catches a regression with mutation testingmutation testing: An audit of a test suite's strength: change one small thing in the code under test and see which tests fail. A mutant that no test catches is a hole in the suite; a stronger assertion or a new test is needed.: change one character of download.sh, run bats stub-curl.bats, and watch a test fail. Flip the comparison from 2* to 3*: the 2xx test goes from ok to not ok on its [ "$status" -eq 0 ] line, because the function now treats success as an error, while the other two stay ok. A mutant that leaves every test ok shows a hole in the suite: add the test, or the assertion, that fails on it.

Run Bats in CI next to the static checks already on the file, so a broken test and a shell bug are both found before the next pull request: shellcheck download.sh && shfmt -i 2 -ci -d download.sh && bats stub-curl.bats. The -i 2 -ci flags match the two-space layout of these files; a bare shfmt -d would ask for tabs and fail the job. The lessons on ShellCheck and style with shfmt set those up; the performance lesson times code with the shell’s time, and time bats stub-curl.bats times the whole suite the same way.

Practise: predict output, pick the fix, read a TAP line

Each question shows a Bats fragment, a TAP line or a scenario you have met in a shell test. A wrong choice names the rule it broke:

Interview questions

Warm-up (fresher to 2–5 years): What do $status and $output hold after run, and why not just call the command directly in a @test?

run CMD... tells Bats to execute the command, save its exit status in $status, its combined standard output and standard error in $output, and that output split by newline in the array ${lines[@]}, with empty lines left out, and then return 0 itself. The return 0 is the point: Bats runs a @test with set -e, so a bare greet Priya that returns 1 would end the test on that line with “not ok” before any assertion ran, and the test could never prove that a failure is the right answer for bad input. With run, the status is data: [ "$status" -eq 1 ] or [ "$status" -eq 0 ] is an ordinary [ ] test like any other. For a command whose standard error is the thing you want to check, run --separate-stderr (Bats 1.5 and up) gives $output and $stderr separately.

Two harder questions follow, one about tests that pass alone but fail in the suite, one about testing a script that calls the network. Explain your reasoning before you open each first step:

Key takeaways

• A Bats file is Bash with @test "name" { ... } blocks; bats --formatter tap FILE prints a plan line and one ok or not ok line per test, in the Test Anything Protocol that CI systems read.
• run CMD... runs the command in a subshell and keeps its exit status in $status, its combined output in $output and its non-empty lines in ${lines[@]}, so an assertion is an ordinary [ ... ] test; run --separate-stderr adds $stderr for error messages.
• Every @test runs in a fresh process with a new, empty $BATS_TEST_TMPDIR; setup and teardown run around each test, and setup_file and teardown_file run once per file.
• Test a function by sourcing its script, which needs a main guard that stops main from running under a different $0; test a command by running the script itself.
• Stub a command by putting a fake first on PATH in $BATS_TEST_TMPDIR/bin, so a script that calls curl can be tested offline, and prove the suite with a mutant that must turn at least one test not ok.