POSIX sh vs Bash: spotting and fixing bashisms

A script that starts with #!/bin/sh runs well on your Mac and goes wrong on an Ubuntu server. Twelve Bash-only features were tried in this lesson under dash, the shell behind sh on Ubuntu: five stopped with an error and seven carried on with exit status 0 and the wrong result.

/bin/sh means POSIX sh, the shell language the standard defines, and Bash is POSIX sh plus many extras that other shells do not have. You spot those extras by running the script under dash and reading it with shellcheck -s sh, and you fix each one by writing its POSIX form. You will see which shell a script really gets, try the twelve extras that break, meet three behaviours that differ without any error, rewrite a script and check it, and pick the first line that fits.

Which shell a script gets, and what POSIX sh promises

Two machines can read the same #!/bin/sh script with two different programs. The shebangshebang: The first line of a script, #! followed by the interpreter to run it, such as #!/usr/bin/env bash. The kernel reads it when the file is run by its path; to Bash it is a comment. More in “Your first script: shebang, chmod and running it” on the first line names the interpreter, and sh is only a name that each system points at a shell of its choice. Step through what happens between the first line and the shell that reads it:

A POSIX shellPOSIX shell: A shell that implements the shell command language the POSIX standard defines for the sh utility. Bash and dash both do; Bash adds many extras the standard does not require. is any shell that implements the language the standard defines. Bash and dashdash: The Debian Almquist shell, a small POSIX sh implementation with few extras. On Ubuntu its package provides /usr/bin/sh, so it is the usual strict test of a script that claims to be portable. both do, and only Bash adds a long list of extras. Debian policy lets authors of /bin/sh scripts assume POSIX.1-2017 plus a short list, and nothing more: echo -n printing no newline, -a and -o in test, local, and numeric signals for kill and trap. A bashismbashism: A feature of Bash that POSIX does not require, such as arrays, [[ ]], {1..3} or the function keyword. A script that uses one can fail under dash or any other plain POSIX sh. is an extra outside that set. The next session starts one script body three ways and prints which shell ran it:

from session import run, write

# One script body, three first lines. The kernel starts the interpreter the first line names, so BASH_VERSION and
# POSIXLY_CORRECT show which shell really ran it. Recorded on a Mac, where /bin/sh is Apple's Bash 3.2.57 started as sh;
# on Debian or Ubuntu the first line would name dash and the script would print "not bash".
BODY = """echo "BASH_VERSION=${BASH_VERSION:-not bash}"
echo "POSIXLY_CORRECT=${POSIXLY_CORRECT:-unset}"
"""
write("by-sh.sh", "#!/bin/sh\n" + BODY)
write("by-env.sh", "#!/usr/bin/env bash\n" + BODY)
write("by-bin.sh", "#!/bin/bash\n" + BODY)
run("""
chmod +x by-sh.sh by-env.sh by-bin.sh
./by-sh.sh
./by-env.sh
./by-bin.sh
# the same file as by-sh.sh, started by dash: the first line is only a comment now
dash by-sh.sh
# the Mac's /bin/bash is too old for associative arrays
!/bin/bash -c 'declare -A seen'
bash -c 'declare -A seen=([a]=1); echo "${seen[a]}"'
""")

$ chmod +x by-sh.sh by-env.sh by-bin.sh
$ ./by-sh.sh
BASH_VERSION=3.2.57(1)-release
POSIXLY_CORRECT=y
$ ./by-env.sh
BASH_VERSION=5.3.20(1)-release
POSIXLY_CORRECT=unset
$ ./by-bin.sh
BASH_VERSION=3.2.57(1)-release
POSIXLY_CORRECT=unset
# the same file as by-sh.sh, started by dash: the first line is only a comment now
$ dash by-sh.sh
BASH_VERSION=not bash
POSIXLY_CORRECT=unset
# the Mac's /bin/bash is too old for associative arrays
$ /bin/bash -c 'declare -A seen'
/bin/bash: line 0: declare: -A: invalid option
declare: usage: declare [-afFirtx] [-p] [name[=value] ...]
[exit status 2]
$ bash -c 'declare -A seen=([a]=1); echo "${seen[a]}"'
1

The #!/bin/sh file printed BASH_VERSION=3.2.57(1)-release and POSIXLY_CORRECT=y. On this Mac sh is a Bash started under the name sh, and Bash then enters POSIX mode, though it keeps its extras. The #!/usr/bin/env bash file found Bash 5.3.20, the first one on PATH. The #!/bin/bash file ran the old 3.2.57 again, this time with POSIXLY_CORRECT unset. Then dash by-sh.sh started the same file: the shell named on the command line wins, the first line is only a comment to dash, and BASH_VERSION was empty, so the script printed not bash. That answer is what #!/bin/sh gives on Ubuntu, where the dash package provides /usr/bin/sh. The last two commands show why the version matters: the Mac’s /bin/bash rejected declare -A for an associative array with invalid option and status 2, and Bash 5.3.20 accepted it.

What the Mac accepted says nothing about Ubuntu, so the sections that follow treat dash as the reference reader.

Twelve Bash features that dash does not have

Each of twelve one-line scripts uses one Bash feature. A small runner starts every file with Bash 5.3.20 and then with dash 0.5.13.5 and prints both outputs and both exit statuses, so you can read them side by side. The $RANDOM line prints set when the variable has a value, so the record does not change from run to run:

from session import run, write

# Twelve one-line Bash-only features, each in its own file, run under GNU Bash 5.3.20 and under dash 0.5.13.5 by one
# small runner that prints both outputs and both exit statuses. The $RANDOM line prints "set" when $RANDOM has a value,
# so the recorded output does not change from run to run.
write("b01.sh", 'a=(one two three); echo "${a[1]}"\n')
write("b02.sh", 'x=abc; if [[ $x == a* ]]; then echo match; else echo no match; fi\n')
write("b03.sh", "echo {1..3}\n")
write("b04.sh", "function bye { echo bye; }; bye\n")
write("b05.sh", 'echo "x=1" > lib.sh; source ./lib.sh; echo "x=[$x]"\n')
write("b06.sh", 's=abcdef; echo "${s:1:3}"\n')
write("b07.sh", 'cat <<< "hello"\n')
write("b08.sh", 'echo "RANDOM is [${RANDOM:+set}]"\n')
write("b09.sh", 'printf "%q\\n" "a b"\n')
write("b10.sh", 'printf "a:b:c" | { read -r -d : first; echo "[$first]"; }\n')
write("b11.sh", 'x=a; if [ "$x" == a ]; then echo equal; else echo different; fi\n')
write("b12.sh", 'echo hi &> out.txt; wait; echo "out.txt holds: [$(cat out.txt)]"\n')
write("both.sh", r"""for f in "$@"; do
  printf '=== %s\n' "$(cat "$f")"
  for sh in bash dash; do
    out=$($sh "$f" 2>&1); rc=$?
    printf '%s (status %s):\n' "$sh" "$rc"
    printf '%s\n' "$out" | sed 's/^/  /'
  done
done
""")
run("""
echo "bash $BASH_VERSION"
bash both.sh b01.sh b02.sh b03.sh b04.sh b05.sh b06.sh
bash both.sh b07.sh b08.sh b09.sh b10.sh b11.sh b12.sh
""")

$ echo "bash $BASH_VERSION"
bash 5.3.20(1)-release
$ bash both.sh b01.sh b02.sh b03.sh b04.sh b05.sh b06.sh
=== a=(one two three); echo "${a[1]}"
bash (status 0):
  two
dash (status 2):
  b01.sh: 1: Syntax error: "(" unexpected
=== x=abc; if [[ $x == a* ]]; then echo match; else echo no match; fi
bash (status 0):
  match
dash (status 0):
  b02.sh: 1: [[: not found
  no match
=== echo {1..3}
bash (status 0):
  1 2 3
dash (status 0):
  {1..3}
=== function bye { echo bye; }; bye
bash (status 0):
  bye
dash (status 2):
  b04.sh: 1: Syntax error: "}" unexpected
=== echo "x=1" > lib.sh; source ./lib.sh; echo "x=[$x]"
bash (status 0):
  x=[1]
dash (status 0):
  b05.sh: 1: source: not found
  x=[]
=== s=abcdef; echo "${s:1:3}"
bash (status 0):
  bcd
dash (status 2):
  b06.sh: 1: Bad substitution
$ bash both.sh b07.sh b08.sh b09.sh b10.sh b11.sh b12.sh
=== cat <<< "hello"
bash (status 0):
  hello
dash (status 2):
  b07.sh: 1: Syntax error: redirection unexpected
=== echo "RANDOM is [${RANDOM:+set}]"
bash (status 0):
  RANDOM is [set]
dash (status 0):
  RANDOM is []
=== printf "%q\n" "a b"
bash (status 0):
  a\ b
dash (status 2):
  b09.sh: 1: printf: %q: invalid directive
=== printf "a:b:c" | { read -r -d : first; echo "[$first]"; }
bash (status 0):
  [a]
dash (status 0):
  b10.sh: 1: read: Illegal option -d
  []
=== x=a; if [ "$x" == a ]; then echo equal; else echo different; fi
bash (status 0):
  equal
dash (status 0):
  b11.sh: 1: [: a: unexpected operator
  different
=== echo hi &> out.txt; wait; echo "out.txt holds: [$(cat out.txt)]"
bash (status 0):
  out.txt holds: [hi]
dash (status 0):
  hi
  out.txt holds: []

Bash printed the expected result for all twelve. dash did not do the same for any of them, and it failed in two different ways. Five stopped with status 2: the array assignment (Syntax error: "(" unexpected), the function keyword, ${s:1:3} (Bad substitution), the here-string <<< (redirection unexpected) and printf %q (invalid directive). The other seven finished with status 0 and a wrong result. [[ was reported as not found, so the if took its else branch. {1..3} stayed the literal text {1..3}. source was not found and left x empty. $RANDOM expanded to nothing. read -d printed Illegal option -d and left the variable empty. == inside [ ] was unexpected operator, which made the test false. And &> was read as echo hi & followed by > out.txt, so hi went to the terminal and the file stayed empty.

Eleven of them have a POSIX form, and each form ran correctly in dash when tried on its own; printf %q has none:

Newer POSIX editions add some features of this list: the read page of POSIX.1-2024 defines -d. dash 0.5.13.5 still printed Illegal option -d, so a feature in the standard is not yet a feature in every shell.

Each of the seven shows itself only in what was printed or written, which is why the checks later in this lesson read the output.

Three places where dash and Bash differ without an error

Some differences have no bashism in them. The code is valid POSIX sh, and the shells simply behave differently, because the standard leaves the behaviour open or Bash chose a different default. The next session runs three small files under Bash 5.3.20, under bash --posix and under dash 0.5.13.5:

from session import run, write

# Three things dash does differently when a script uses only POSIX syntax: echo reads backslash escapes, set -e is
# passed into $( ), and $(( )) does not follow a variable whose value is another variable's name. A small runner prints
# GNU Bash 5.3.20, bash --posix and dash 0.5.13.5 for each file.
write("q1.sh", "echo 'one\\ntwo'\nprintf '%s\\n' 'one\\ntwo'\n")
write("q2.sh", 'set -e\nx=$(false; echo after)\necho "x is [$x]"\necho "script went on"\n')
write("q3.sh", "n=count\ncount=7\necho $(( n + 1 ))\n")
write("three.sh", r"""for f in "$@"; do
  printf '=== %s\n' "$f"
  for sh in "bash" "bash --posix" "dash"; do
    out=$($sh "$f" 2>&1); rc=$?
    printf '%s (status %s):\n' "$sh" "$rc"
    if [ -n "$out" ]; then printf '%s\n' "$out" | sed 's/^/  /'; else echo '  (no output)'; fi
  done
done
""")
run("""
cat q1.sh
bash three.sh q1.sh
cat q2.sh
bash three.sh q2.sh
cat q3.sh
bash three.sh q3.sh
""")

$ cat q1.sh
echo 'one\ntwo'
printf '%s\n' 'one\ntwo'
$ bash three.sh q1.sh
=== q1.sh
bash (status 0):
  one\ntwo
  one\ntwo
bash --posix (status 0):
  one\ntwo
  one\ntwo
dash (status 0):
  one
  two
  one\ntwo
$ cat q2.sh
set -e
x=$(false; echo after)
echo "x is [$x]"
echo "script went on"
$ bash three.sh q2.sh
=== q2.sh
bash (status 0):
  x is [after]
  script went on
bash --posix (status 1):
  (no output)
dash (status 1):
  (no output)
$ cat q3.sh
n=count
count=7
echo $(( n + 1 ))
$ bash three.sh q3.sh
=== q3.sh
bash (status 0):
  8
bash --posix (status 0):
  8
dash (status 2):
  q3.sh: 3: Illegal number: count

First, echo. POSIX leaves the result implementation-defined when an operand contains a backslash. Bash printed one\ntwo on one line with echo and with printf '%s\n'. dash read the escape in echo and printed two lines, and printf kept it as written. Bash needs its xpg_echo option to read escapes the same way. Print with printf '%s\n', as the printf and echo lesson shows.

Second, set -e in a command substitution. With x=$(false; echo after) Bash by default turns -e off inside the substitution, so echo after still runs, x becomes after and the script goes on. dash keeps -e on, the substitution stops at false, and the script ended with status 1 and no output. POSIX describes the dash behaviour. Bash gets it with bash --posix too, which switches on the inherit_errexitinherit_errexit: A Bash shopt option, added in 4.4: when switched on, set -e is passed down into every command substitution $(...). Without it, Bash runs each $(...) in a subshell without errexit, so x=$(false; echo after) sets x=after and the script runs on; dash inherits errexit into command substitutions by default. More in “set -e, set -u and pipefail: what they really do” option, and the second run ended with status 1. Step through that one assignment to see the point where the two shells part:

Third, arithmetic. With n=count and count=7, Bash evaluated $(( n + 1 )) as 8 by looking through n to count, in default and in POSIX mode. dash answered Illegal number: count. POSIX only asks for variables whose value is a valid integer, and an expansion fails when the shell does not recognise a variable’s contents, so the Bash result is an extension.

A rewrite is done when ShellCheck is quiet and both shells agree

The best check is the strictest reader. The session below holds a short Bash script with a brace range, a [[ ]] test and source, and its rewrite in POSIX sh. ShellCheck reads each with -s sh, Bash and dash run them, and cmp compares the two outputs byte for byte. The end of the session tries two things that are easy to get wrong:

from session import run, write

# A short Bash script (a brace list, a [[ ]] test, source) and its POSIX sh rewrite. ShellCheck reads both with
# -s sh, then Bash 5.3.20 and dash 0.5.13.5 run each one. The last part shows two things that are easy to get wrong:
# bash --posix still accepts the Bash script, and ShellCheck 0.11.0 still flags $'...' in sh.
write("settings.sh", "MODE=fast\n")
write("jobs.bash", """#!/bin/bash
source ./settings.sh
for n in {1..3}; do
  if [[ $MODE == fast && $n == 2 ]]; then
    echo "skip $n"
    continue
  fi
  echo "run $n in $MODE mode"
done
""")
write("jobs.sh", """#!/bin/sh
. ./settings.sh
for n in 1 2 3; do
  if [ "$MODE" = fast ] && [ "$n" = 2 ]; then
    echo "skip $n"
    continue
  fi
  echo "run $n in $MODE mode"
done
""")
write("tab.sh", """#!/bin/sh
tab=$'\\t'
printf '[%s]\\n' "$tab"
""")
run("""
# the Bash script first: ShellCheck in sh mode flags source, the brace list and [[ ]]
!shellcheck -s sh -x jobs.bash
bash jobs.bash
dash jobs.bash
# the rewrite: clean in sh mode, and both shells print the same lines
shellcheck -s sh -x jobs.sh && echo "shellcheck: no findings"
bash jobs.sh > bash.out
dash jobs.sh > dash.out
cat dash.out
cmp bash.out dash.out && echo "bash and dash printed the same bytes"
# bash --posix is not dash: it still runs the Bash script
bash --posix jobs.bash | head -n 2
# POSIX.1-2024 allows $'...': dash runs it, ShellCheck 0.11.0 still flags it in sh
dash tab.sh
!shellcheck -s sh tab.sh
""")

# the Bash script first: ShellCheck in sh mode flags source, the brace list and [[ ]]
$ shellcheck -s sh -x jobs.bash

In jobs.bash line 2:
source ./settings.sh
^------------------^ SC3046 (warning): In POSIX sh, 'source' in place of '.' is undefined.
^------------------^ SC3051 (warning): In POSIX sh, 'source' in place of '.' is undefined.


In jobs.bash line 3:
for n in {1..3}; do
         ^----^ SC3009 (warning): In POSIX sh, brace expansion is undefined.


In jobs.bash line 4:
  if [[ $MODE == fast && $n == 2 ]]; then
     ^----------------------------^ SC3010 (warning): In POSIX sh, [[ ]] is undefined.
              ^-- SC3014 (warning): In POSIX sh, == in place of = is undefined.
                            ^-- SC3014 (warning): In POSIX sh, == in place of = is undefined.

For more information:
  https://www.shellcheck.net/wiki/SC3009 -- In POSIX sh, brace expansion is u...
  https://www.shellcheck.net/wiki/SC3010 -- In POSIX sh, [[ ]] is undefined.
  https://www.shellcheck.net/wiki/SC3014 -- In POSIX sh, == in place of = is ...
[exit status 1]
$ bash jobs.bash
run 1 in fast mode
skip 2
run 3 in fast mode
$ dash jobs.bash
jobs.bash: 2: source: not found
jobs.bash: 4: [[: not found
run {1..3} in  mode
# the rewrite: clean in sh mode, and both shells print the same lines
$ shellcheck -s sh -x jobs.sh && echo "shellcheck: no findings"
shellcheck: no findings
$ bash jobs.sh > bash.out
$ dash jobs.sh > dash.out
$ cat dash.out
run 1 in fast mode
skip 2
run 3 in fast mode
$ cmp bash.out dash.out && echo "bash and dash printed the same bytes"
bash and dash printed the same bytes
# bash --posix is not dash: it still runs the Bash script
$ bash --posix jobs.bash | head -n 2
run 1 in fast mode
skip 2
# POSIX.1-2024 allows $'...': dash runs it, ShellCheck 0.11.0 still flags it in sh
$ dash tab.sh
[	]
$ shellcheck -s sh tab.sh

In tab.sh line 2:
tab=$'\t'
    ^---^ SC3003 (warning): In POSIX sh, $'..' is undefined.

For more information:
  https://www.shellcheck.net/wiki/SC3003 -- In POSIX sh, $'..' is undefined.
[exit status 1]

ShellCheck read jobs.bash as sh and named source (SC3046 and SC3051), the brace list (SC3009), [[ ]] (SC3010) and == (SC3014). Run in dash, the same file printed source: not found and [[: not found, looped once over the text {1..3}, and exited with status 0. The rewrite changed three things: a plain . ./settings.sh, a list 1 2 3 and two [ ] tests joined by &&. ShellCheck reported no findings, and the output of Bash and dash was the same bytes.

Here is the order of the checks and what each one can and cannot see:

Debian policy names checkbashismscheckbashisms: A Debian devscripts tool that scans a script meant for /bin/sh and warns about features POSIX does not require. Debian policy names it as a way to find violations. for this job. Its manual page says it is based on a check of Debian’s lintian tool, so it is a second reader to run next to ShellCheck. It comes with the Debian devscripts package, so it was not part of the recorded sessions.

bash --posix jobs.bash ran the Bash script and printed its first lines, which is why it is not a check for bashisms. The last two commands show a case where the standard has moved ahead of the tools. POSIX.1-2024 allows $'\t', and dash ran it, but ShellCheck 0.11.0 still reported SC3003 on it, and its page says the rule applied to POSIX.1-2017. Keep the warning if the script must also run under older shells. Add a # shellcheck disable=SC3003 comment above that one line when every shell you target has the feature.

The first line should say what the script needs

The first line states what the script needs. Answer the three questions below about a script of your own and see which first lines would run it wrongly; each one that is ruled out says why:

In words, the rule has three steps. If the script must run on machines you do not control and it passes the checks of the last section, keep it POSIX and start it with #!/bin/sh. If it needs a Bashism, a newer Bash, or just the comfort of arrays and [[ ]], say so in the first line instead of hoping that sh is Bash. Choose #!/usr/bin/env bash to run the first Bash on PATH, which found 5.3.20 in the session, or #!/bin/bash when you want one fixed path, which started 3.2.57, too old for the associative array of the first session. The .sh ending of a file name chooses nothing, and a script started as dash file ignores its first line.

Practise: bashisms and first lines

Each question gives a short script, a shell or a message, like the ones recorded above. Decide what dash or Bash does, then check the answer against the recorded output:

Interview questions

Warm-up (fresher to 2–5 years): what makes a feature a bashism, and how would you find one before it reaches a server?

A bashism is a feature of Bash that the POSIX shell language does not require, such as arrays, [[ ]], {1..3}, source, ${s:1:3} and <<<. It matters only when the script claims to be plain sh, because dash then rejects or mishandles it, sometimes with exit status 0. To find one, run shellcheck -s sh script.sh, which names the construct, and run the script under dash script.sh, reading the output and not only the status. checkbashisms from the Debian devscripts package is a second reader. If the script really needs a Bashism, name Bash on the first line instead.

Two harder questions take the same skills to cases this lesson does not show: a root check in a container entrypoint that dash lets through, and a first line with shell options that one system accepts and another refuses. Note the first command you would try on each before you read its suggested first step:

Key takeaways

• /bin/sh names a role that each system fills with its own shell: dash on Ubuntu, and Bash 3.2.57 started as sh in the Mac session, so a script that asks for sh may rely only on POSIX sh.
• All twelve Bash features tried in this lesson failed under dash, five with an error and status 2 and seven with status 0 and a wrong result, so the output matters as much as the exit status.
• Most bashisms have a POSIX form: case or [ ], a written-out list or a counter, . for source, cmd > file 2>&1 for &>, and printf '%s\n' instead of echo for data.
• dash also differs on valid code: echo reads backslash escapes, set -e reaches into $( ), and $(( )) will not look through a variable that holds another name.
• Check a script with shellcheck -s sh and with dash script.sh; bash --posix keeps every Bash extra, so it proves nothing about dash.
• Use #!/bin/sh only for a script that passes those checks; otherwise name Bash on the first line and say that it is needed.