Lists done right: arrays, mapfile and read -a

cmd="grep -i -e 'disk full' app.log" holds 30 characters of a sound search, yet running it as $cmd makes grep look for 'disk, apostrophe included, in two files, one of which does not exist, and the search ends with status 2. Kept in a Bash array, the same command finds both lines of the log that mention a full disk.

Lists of arguments, lines and fields break the same way whenever they go through a string. You will store commands as arrays, build argument lists from the options that apply and print them before they run, load a file or a command’s output with mapfile, split one line with read -a, join an array for printing, and tell which of these forms the Bash 3.2 of a Mac lacks.

Quotes kept in a string are just characters, so a command needs an array

The session keeps a search of app.log, a four-line backup log, in a variable and runs it three ways: as $cmd, as "$cmd" and from an array. Watch the lines that printf '[%s]\n' prints: each holds one argument between brackets.

from session import run, write

write("app.log", "09:00 INFO backup started\n09:02 ERROR disk full on /data\n09:03 WARN Disk Full soon on /logs\n09:05 INFO backup done\n")
write("search-string.sh", "#!/usr/bin/env bash\n# search-string.sh: a grep command kept in a string\ncmd=\"grep -i -e 'disk full' app.log\"\n$cmd\n")

run(r"""
cat app.log
cmd="grep -i -e 'disk full' app.log"; echo "${#cmd} characters"
!$cmd
printf '[%s]\n' $cmd
!"$cmd"
cmd=(grep -i -e 'disk full' -- app.log)
printf '[%s]\n' "${cmd[@]}"
"${cmd[@]}"
declare -p cmd
cat search-string.sh
!shellcheck search-string.sh
""")

$ cat app.log
09:00 INFO backup started
09:02 ERROR disk full on /data
09:03 WARN Disk Full soon on /logs
09:05 INFO backup done
$ cmd="grep -i -e 'disk full' app.log"; echo "${#cmd} characters"
30 characters
$ $cmd
grep: full': No such file or directory
[exit status 2]
$ printf '[%s]\n' $cmd
[grep]
[-i]
[-e]
['disk]
[full']
[app.log]
$ "$cmd"
bash: grep -i -e 'disk full' app.log: command not found
[exit status 127]
$ cmd=(grep -i -e 'disk full' -- app.log)
$ printf '[%s]\n' "${cmd[@]}"
[grep]
[-i]
[-e]
[disk full]
[--]
[app.log]
$ "${cmd[@]}"
09:02 ERROR disk full on /data
09:03 WARN Disk Full soon on /logs
$ declare -p cmd
declare -a cmd=([0]="grep" [1]="-i" [2]="-e" [3]="disk full" [4]="--" [5]="app.log")
$ cat search-string.sh
#!/usr/bin/env bash
# search-string.sh: a grep command kept in a string
cmd="grep -i -e 'disk full' app.log"
$cmd
$ shellcheck search-string.sh

In search-string.sh line 3:
cmd="grep -i -e 'disk full' app.log"
     ^----------------------------^ SC2089 (warning): Quotes/backslashes will be treated literally. Use an array.


In search-string.sh line 4:
$cmd
^--^ SC2090 (warning): Quotes/backslashes in this variable will not be respected.

For more information:
  https://www.shellcheck.net/wiki/SC2089 -- Quotes/backslashes will be treate...
  https://www.shellcheck.net/wiki/SC2090 -- Quotes/backslashes in this variab...
[exit status 1]

Unquoted, $cmd was splitword splitting: The step in which Bash cuts the unquoted results of parameter expansion, command substitution and arithmetic expansion into separate words at the characters of IFS. More in “Word splitting and IFS: why "$var" needs quotes” at its spaces into six words, and two of them still held the single quotes: 'disk and full'. grep took 'disk as its pattern, and full' and app.log as two files to search. It printed only “No such file or directory” for full' and ended with status 2. Quoted, "$cmd" stayed one word, so Bash looked for a command named after the whole value and reported “command not found”, status 127.

The cause is the order in which Bash reads a line. It finds quotes, operators and the boundaries of words while it reads what you typed, and only then expands variables. A value that comes out of a variable is split and globbed when it is unquoted, but it is never read again for quotes: the manual’s quote removalThe last step of expansion: Bash deletes the quote characters and backslashes that were typed in the command line, once they have marked where each word begins and ends. deletes only the quote characters that “did not result from” an expansion. Inside a value, quotes are data.

An array keeps the boundaries instead of the text. In cmd=(grep -i -e 'disk full' -- app.log) the quotes are typed in the assignment, so they are quoting: declare -p cmd showed six elements, the fourth disk full with no quote characters left. "${cmd[@]}" then hands each element on as exactly one word, the rule the word splitting and IFS lesson showed for "$@". Such a command arraycommand array: An indexed array that holds a command name and its arguments, one per element, run as "${cmd[@]}" so that each element reaches the command as exactly one word, spaces and quote characters included. is the way to keep a command whose arguments may hold spaces. The -- ends grep’s options, so a pattern or file name that starts with a dash would still be read as an operand, as Guideline 10 of the POSIX utility syntax guidelines asks.

Step through the same command kept both ways, one box per word that grep receives:

The left column of the figure is what ShellCheck warns about. In search-string.sh it marked the assignment with SC2089, “Quotes/backslashes will be treated literally. Use an array.”, and the line that runs $cmd with SC2090.

An argument list grows by the options that apply, and a dry run shows it

A search that should ignore case only when ignore_case=1, and number its lines only when line_numbers=1, needs a list of options that grows by the ones that apply. Start from an empty array and append with +=( ), as in opts=(); [[ $ignore_case == 1 ]] && opts+=(-i). The session builds the list twice, tries the same with a string, and ends by printing the whole command before running it:

from session import run, write

write("app.log", "09:00 INFO backup started\n09:02 ERROR disk full on /data\n09:03 WARN Disk Full soon on /logs\n09:05 INFO backup done\n")

run(r"""
ignore_case=1; line_numbers=0
opts=(); [[ $ignore_case == 1 ]] && opts+=(-i); [[ $line_numbers == 1 ]] && opts+=(-n); declare -p opts
grep "${opts[@]}" -e 'disk full' -- app.log
ignore_case=0; opts=(); [[ $ignore_case == 1 ]] && opts+=(-i); [[ $line_numbers == 1 ]] && opts+=(-n); declare -p opts
grep "${opts[@]}" -e 'disk full' -- app.log
flags=''; echo "flags=[$flags]"
!grep "$flags" -e 'disk full' -- app.log
flags='-i -n'
!grep "$flags" -e 'disk full' -- app.log
grep $flags -e 'disk full' -- app.log
opts=(-i); opts+=-n; declare -p opts
opts=(-i); opts+=(-n); declare -p opts
cmd=(grep "${opts[@]}" -e 'disk full' -- app.log); dry_run=1
if [[ $dry_run == 1 ]]; then printf '[%s]\n' "${cmd[@]}"; else "${cmd[@]}"; fi
dry_run=0; if [[ $dry_run == 1 ]]; then printf '[%s]\n' "${cmd[@]}"; else "${cmd[@]}"; fi
""")

$ ignore_case=1; line_numbers=0
$ opts=(); [[ $ignore_case == 1 ]] && opts+=(-i); [[ $line_numbers == 1 ]] && opts+=(-n); declare -p opts
declare -a opts=([0]="-i")
$ grep "${opts[@]}" -e 'disk full' -- app.log
09:02 ERROR disk full on /data
09:03 WARN Disk Full soon on /logs
$ ignore_case=0; opts=(); [[ $ignore_case == 1 ]] && opts+=(-i); [[ $line_numbers == 1 ]] && opts+=(-n); declare -p opts
declare -a opts=()
$ grep "${opts[@]}" -e 'disk full' -- app.log
09:02 ERROR disk full on /data
$ flags=''; echo "flags=[$flags]"
flags=[]
$ grep "$flags" -e 'disk full' -- app.log
grep: : No such file or directory
app.log:09:02 ERROR disk full on /data
[exit status 2]
$ flags='-i -n'
$ grep "$flags" -e 'disk full' -- app.log
grep: invalid option --
usage: grep [-abcdDEFGHhIiJLlMmnOopqRSsUVvwXxZz] [-A num] [-B num] [-C[num]]
	[-e pattern] [-f file] [--binary-files=value] [--color=when]
	[--context[=num]] [--directories=action] [--label] [--line-buffered]
	[--null] [pattern] [file ...]
[exit status 2]
$ grep $flags -e 'disk full' -- app.log
2:09:02 ERROR disk full on /data
3:09:03 WARN Disk Full soon on /logs
$ opts=(-i); opts+=-n; declare -p opts
declare -a opts=([0]="-i-n")
$ opts=(-i); opts+=(-n); declare -p opts
declare -a opts=([0]="-i" [1]="-n")
$ cmd=(grep "${opts[@]}" -e 'disk full' -- app.log); dry_run=1
$ if [[ $dry_run == 1 ]]; then printf '[%s]\n' "${cmd[@]}"; else "${cmd[@]}"; fi
[grep]
[-i]
[-n]
[-e]
[disk full]
[--]
[app.log]
$ dry_run=0; if [[ $dry_run == 1 ]]; then printf '[%s]\n' "${cmd[@]}"; else "${cmd[@]}"; fi
2:09:02 ERROR disk full on /data
3:09:03 WARN Disk Full soon on /logs

With ignore_case=1 the list was -i and grep found both lines. With both switches off, declare -p showed opts=(), "${opts[@]}" added no word at all, and grep found the one line in lower case. The table sets every form of the session side by side:

The empty word is the subtle row. A quoted expansion is always exactly one word, even when its value is empty, and grep took that empty word for the name of a file: it printed an error for it and put app.log: before the line it found, because it now had two files to search. The string of two options failed the other way: as one word, -i -n made grep read -i and then a space as the next option letter. An empty array has no elements, so its quoted expansion is no word at all, which is exactly what an option that does not apply should be.

+= needs its parentheses for the same reason. The manual says that += with a compound assignment appends new values to the array, while on a plain value it appends to the text, and opts without a subscript means its first element: opts+=-n glued -n onto -i.

The last two lines are a dry run. cmd=(grep "${opts[@]}" -e 'disk full' -- app.log) puts the options into the command, and with dry_run=1 the if printed it with printf '[%s]\n' "${cmd[@]}" instead of running it. printf reuses its format for every argument, so each argument appeared on its own line between brackets, and the space inside [disk full] could not be taken for the end of an argument. With dry_run=0, the same array ran.

mapfile keeps every line whole, and an unquoted command substitution does not

todo.txt holds four lines: renew the domain, * call the bank, an empty line and pay rent. lines-to-array.sh loads it into an array three ways and prints every element between brackets:

#!/usr/bin/env bash
# lines-to-array.sh: the lines of a file into an array, three ways.
file=${1:-todo.txt}

# 1. Unquoted command substitution: the output is split at spaces, tabs
#    and newlines, then every word is globbed (ShellCheck SC2207).
broken=( $(cat "$file") )
echo "broken: ${#broken[@]} elements"
printf '  [%s]\n' "${broken[@]}"

# 2. mapfile: one element per line; -t removes the newline that ends each.
mapfile -t lines < "$file"
echo "mapfile: ${#lines[@]} elements"
printf '  [%s]\n' "${lines[@]}"

# 3. read -a: one line only, split at IFS into words, never globbed.
read -r -a words <<< "${lines[1]}"
echo "read -a on line 2: ${#words[@]} elements"
printf '  [%s]\n' "${words[@]}"

The session runs the script in a folder that also holds notes.md and report.pdf, then tries the options of mapfile one at a time:

from session import run, files, write

files("lines-to-array.sh")
write("todo.txt", "renew the domain\n* call the bank\n\npay rent\n")

run(r"""
touch notes.md report.pdf
bash lines-to-array.sh todo.txt
!shellcheck lines-to-array.sh
mapfile kept < todo.txt; declare -p kept
mapfile -t due < <(grep -v '^pay' todo.txt); declare -p due
printf '%s\n' one two | mapfile -t got; echo "${#got[@]} elements after the pipe"
type -t mapfile readarray
printf 'first\nsecond' > partial.txt; readarray -t p < partial.txt; declare -p p
mapfile -t -d , colours < <(printf %s 'red,green,blue'); declare -p colours
mapfile -t -d , colours <<< 'red,green,blue'; declare -p colours
""")

$ touch notes.md report.pdf
$ bash lines-to-array.sh todo.txt
broken: 12 elements
  [renew]
  [the]
  [domain]
  [lines-to-array.sh]
  [notes.md]
  [report.pdf]
  [todo.txt]
  [call]
  [the]
  [bank]
  [pay]
  [rent]
mapfile: 4 elements
  [renew the domain]
  [* call the bank]
  []
  [pay rent]
read -a on line 2: 4 elements
  [*]
  [call]
  [the]
  [bank]
$ shellcheck lines-to-array.sh

In lines-to-array.sh line 7:
broken=( $(cat "$file") )
         ^------------^ SC2207 (warning): Prefer mapfile or read -a to split command output (or quote to avoid splitting).

For more information:
  https://www.shellcheck.net/wiki/SC2207 -- Prefer mapfile or read -a to spli...
[exit status 1]
$ mapfile kept < todo.txt; declare -p kept
declare -a kept=([0]=$'renew the domain\n' [1]=$'* call the bank\n' [2]=$'\n' [3]=$'pay rent\n')
$ mapfile -t due < <(grep -v '^pay' todo.txt); declare -p due
declare -a due=([0]="renew the domain" [1]="* call the bank" [2]="")
$ printf '%s\n' one two | mapfile -t got; echo "${#got[@]} elements after the pipe"
0 elements after the pipe
$ type -t mapfile readarray
builtin
builtin
$ printf 'first\nsecond' > partial.txt; readarray -t p < partial.txt; declare -p p
declare -a p=([0]="first" [1]="second")
$ mapfile -t -d , colours < <(printf %s 'red,green,blue'); declare -p colours
declare -a colours=([0]="red" [1]="green" [2]="blue")
$ mapfile -t -d , colours <<< 'red,green,blue'; declare -p colours
declare -a colours=([0]="red" [1]="green" [2]=$'blue\n')

broken=( $(cat "$file") ) made 12 elements out of 4 lines. The unquoted command substitutioncommand substitution: $(command), or the older backquoted form: Bash runs the command in a subshell and puts its standard output in place of the substitution, with every trailing newline removed. More in “Command and process substitution” was split at every space and newline, the empty line left nothing behind, and the * was treated as a glob patternglob pattern: A word containing *, ? or [ that Bash replaces with the sorted names of the files it matches; it stays as written when nothing matches, unless nullglob or failglob is set. More in “Globbing: wildcards, dotfiles, extglob, globstar” and replaced by the four file names of the folder. ShellCheck reported the line as SC2207, “Prefer mapfile or read -a to split command output”. mapfile -t lines < "$file" gave one element per line, 4 in all, the empty line as an empty element and the * as a plain character. read -a on line 2 split it into four words and still left the * alone.

The other lines of the session show how mapfile behaves:

• Without -t, each element keeps the newline that ended its line, as $'renew the domain\n' in declare -p kept shows; -t removes it.
• < <(grep -v '^pay' todo.txt) hands mapfile the output of a command through process substitutionprocess substitution: <(command) or >(command): Bash starts the command and puts a file name such as /dev/fd/63 in its place. Reading that file gives the command's output; writing to it feeds the command's input. More in “Command and process substitution”, which the command and process substitution lesson explains.
• printf '%s\n' one two | mapfile -t got left 0 elements. The manual’s Pipelines section runs each command of a pipeline in its own subshellsubshell: A copy of the shell that runs commands apart from it, as Bash does for each command of a pipeline. It starts with the shell's variables, and every change it makes to them is lost when it exits. More in “Reading files line by line with read”, so the array was filled and lost with it, as the lesson on reading files line by line showed for a counting loop.
• type -t called both names builtins: readarray is the same command as mapfile under another name. A last line without a newline, as in partial.txt, is still stored.

-d sets the record delimiterrecord delimiter: The character that ends each record a builtin reads: a newline by default. mapfile -d and read -d choose another one, and an empty delimiter means the NUL byte., the character that ends each element instead of a newline, and came with Bash 4.4. mapfile -t -d , colours < <(printf %s 'red,green,blue') gave three clean elements. Fed by the here-stringhere-string: The [n]<<< word form, a Bash extension: word goes through tilde, parameter, command and arithmetic expansion and quote removal, and the result plus one newline is fed to the command on its standard input. More in “Here-documents and here-strings” <<< 'red,green,blue', the last element was $'blue\n': a here-string adds a newline after its word, and -t removes only the delimiter, here the comma. -d '' ends each element at a NUL byte, the separator of NUL-separated inputNUL-separated input: A list in which each item, usually a file name, ends with a NUL byte instead of a newline, as find -print0 writes it. Bash reads one item per call with read -r -d ''. More in “Reading files line by line with read”, which the word splitting lesson used to keep file names whole, newlines and all.

One line splits into fields with read, and a join needs IFS or printf

mapfile makes one element per line; read -a makes one element per field of a single line. It splits the line at the characters of IFSIFS: The internal field separator: the characters Bash splits unquoted expansions on and read splits lines on; by default a space, a tab and a newline. More in “Word splitting and IFS: why "$var" needs quotes”, as word splitting does, but never expands a *. The session splits three lines and then joins a list of cities in four ways:

from session import run, write

write("two.txt", "red green\nblue\n")

run(r"""
IFS=, read -r -a fields <<< 'Asha Rao,Pune,,* priority'; declare -p fields
read -r -a words <<< '  New Delhi   Pune  '; declare -p words
read -r -a first < two.txt; declare -p first
IFS=, read -r -a f <<< 'a,b,'; echo "${#f[@]} fields"
cities=(Pune Chennai 'New Delhi')
printf '%s\n' "${cities[@]}"
(IFS=,; echo "${cities[*]}")
echo "${cities[*]}"
(IFS=', '; echo "${cities[*]}")
csv() { local IFS=,; echo "$*"; }; csv "${cities[@]}"
none=(); printf '[%s]\n' "${none[@]}"; echo "${#none[@]} elements"
(IFS=,; echo "joined: [${none[*]}]")
""")

$ IFS=, read -r -a fields <<< 'Asha Rao,Pune,,* priority'; declare -p fields
declare -a fields=([0]="Asha Rao" [1]="Pune" [2]="" [3]="* priority")
$ read -r -a words <<< '  New Delhi   Pune  '; declare -p words
declare -a words=([0]="New" [1]="Delhi" [2]="Pune")
$ read -r -a first < two.txt; declare -p first
declare -a first=([0]="red" [1]="green")
$ IFS=, read -r -a f <<< 'a,b,'; echo "${#f[@]} fields"
2 fields
$ cities=(Pune Chennai 'New Delhi')
$ printf '%s\n' "${cities[@]}"
Pune
Chennai
New Delhi
$ (IFS=,; echo "${cities[*]}")
Pune,Chennai,New Delhi
$ echo "${cities[*]}"
Pune Chennai New Delhi
$ (IFS=', '; echo "${cities[*]}")
Pune,Chennai,New Delhi
$ csv() { local IFS=,; echo "$*"; }; csv "${cities[@]}"
Pune,Chennai,New Delhi
$ none=(); printf '[%s]\n' "${none[@]}"; echo "${#none[@]} elements"
[]
0 elements
$ (IFS=,; echo "joined: [${none[*]}]")
joined: []

IFS=, read -r -a fields <<< 'Asha Rao,Pune,,* priority' gave 4 fields: the spaces stayed inside Asha Rao and * priority, the two commas in a row held an empty field, and the * was not expanded. With the default IFS, the spaces at both ends were dropped and a run of spaces counted as one separator, so ' New Delhi Pune ' gave New, Delhi and Pune. read took only the first line of two.txt, and a comma at the very end, as in a,b,, made no empty field after it: 2 fields. -r keeps backslashes as plain characters; the reading files lesson shows what read does to them without it.

The joins printed these lines:

"${cities[*]}" makes a single word of all the elements and puts the first character of IFS between each pair, so IFS has to be set before the expansion and only for as long as the join: inside a subshell, ( … ), or as local IFS=, in a function, which leaves the rest of the script with the default. IFS=', ' still joined with a bare comma. A separator of several characters needs the printf -v pattern of the word splitting lesson, which writes the separator after every element and then removes the last one.

An empty array joins to an empty string, but printf '[%s]\n' "${none[@]}" still printed one [] line, because printf uses its format once even when it gets no argument. A dry run of an empty list therefore looks like one empty argument; ${#none[@]}, 0 here, tells the two apart.

Each array feature has a first Bash version, and the Mac’s own Bash is older than most

A script that runs well on Bash 5.3 can stop on the /bin/bash of a Mac, which reports itself as 3.2.57 here. The session runs the same lines in both:

from session import run, write

write("todo.txt", "renew the domain\n* call the bank\n\npay rent\n")

run(r"""
bash -c 'echo "$BASH_VERSION"'; /bin/bash -c 'echo "$BASH_VERSION"'
bash -c 'set -u; none=(); for c in "${none[@]}"; do echo "$c"; done; echo "loop done, no error"'
!/bin/bash -c 'set -u; none=(); for c in "${none[@]}"; do echo "$c"; done; echo "loop done, no error"'
/bin/bash -c 'set -u; none=(); for c in ${none[@]+"${none[@]}"}; do echo "$c"; done; echo "loop done, no error"'
!/bin/bash -c 'mapfile -t lines < todo.txt'
/bin/bash -c 'lines=(); while IFS= read -r line; do lines+=("$line"); done < todo.txt; echo "${#lines[@]} lines"'
""")

$ bash -c 'echo "$BASH_VERSION"'; /bin/bash -c 'echo "$BASH_VERSION"'
5.3.20(1)-release
3.2.57(1)-release
$ bash -c 'set -u; none=(); for c in "${none[@]}"; do echo "$c"; done; echo "loop done, no error"'
loop done, no error
$ /bin/bash -c 'set -u; none=(); for c in "${none[@]}"; do echo "$c"; done; echo "loop done, no error"'
/bin/bash: none[@]: unbound variable
[exit status 127]
$ /bin/bash -c 'set -u; none=(); for c in ${none[@]+"${none[@]}"}; do echo "$c"; done; echo "loop done, no error"'
loop done, no error
$ /bin/bash -c 'mapfile -t lines < todo.txt'
/bin/bash: mapfile: command not found
[exit status 127]
$ /bin/bash -c 'lines=(); while IFS= read -r line; do lines+=("$line"); done < todo.txt; echo "${#lines[@]} lines"'
4 lines

Under set -unounset: Bash's set -u option (also set -o nounset): the shell ends the script when it expands an unset variable or an unset positional parameter. "$@", "$*" (since Bash 4.1) and an empty array's "${a[@]}" (since Bash 4.4) are exempt; use ${v-default} or ${v:-default} to opt out explicitly where unset is acceptable. More in “set -e, set -u and pipefail: what they really do”, which turns the expansion of an unset variable into an error, Bash 5.3 ran the loop over the empty array none without a single round and went on. Bash 3.2 stopped with none[@]: unbound variable. The CHANGES file of bash-4.4-rc2 records the change: ${a[@]} of an array without elements no longer raises that error under nounset. On 3.2, ${none[@]+"${none[@]}"} expands to nothing when the array is empty and to its elements otherwise. It is the :+ form of the defaults lesson without the colon, so it asks only whether the array is set, and the manual counts an array as set once one of its elements has a value. mapfile did not exist in 3.2, which answered “command not found”, while a while IFS= read -r line loop with lines+=("$line") loaded the 4 lines there too.

Walk through the releases to see when each array feature arrived, and so which forms a script may use for the oldest Bash it has to support:

A script for 3.2 may use +=, printf -v and ${!a[@]}, but not mapfile, associative arrays or ${a[-1]}; one that needs mapfile -d needs 4.4 or newer. To see which Bash runs a script, print $BASH_VERSION, as the session did; the lesson on shells, terminals and Bash versions covers the rest.

Practise: three helpers for lists

Write three functions in a file lists.sh, source it, and try them in a folder that holds a few other files, so that a * which gets expanded by mistake shows at once:

• parse_list FILE ARRAYNAME loads the lines of FILE into the array named ARRAYNAME, leaving out empty lines.
• uniq_preserve_order ARRAYNAME removes repeated elements and keeps the first copy of each where it was.
• join_by SEP ARRAYNAME prints the elements on one line with SEP between them, whatever the length of SEP.

A function that fills or reads an array its caller names needs a nameref, local -n, from the indirection lesson; give it a name no caller will use. Make the input with printf '%s\n' 'New Delhi' Chennai '' '*' 'New Delhi' Pune > cities.txt. Then parse_list cities.txt list; printf '[%s]\n' "${list[@]}" must print [New Delhi], [Chennai], [*], [New Delhi] and [Pune]; after uniq_preserve_order list the same printf prints the four without the second [New Delhi]; join_by ' | ' list prints New Delhi | Chennai | * | Pune; and bash -u -c 'source ./lists.sh; empty=(); join_by , empty; echo "status $?"' prints an empty line, then status 0. To check uniq_preserve_order on a longer list, paste its lines into the duplicate line remover, which also keeps the first copy of every line in place.

The questions mix all five sections, and every answer explains why a choice is right or what it misses:

Interview questions

Warm-up (fresher to 2–5 years): what goes into an array with mapfile -t, and what with read -a?

mapfile -t arr reads its whole input and makes one element per line, with the newline at the end of each line removed. Spaces, a * and empty lines are kept exactly, and a last line without a newline is stored too. read -r -a arr reads one line only and splits it into fields at the characters of IFS: with the default IFS the spaces at its ends disappear and New Delhi becomes two elements, while IFS=, read -r -a arr cuts at commas and keeps an empty field between two of them. Neither expands a * into file names, as arr=( $(cmd) ) does. Both empty the array before they fill it, and both lose it when a pipe feeds them, because they then run in a subshell, so give them < file, < <(command) or a here-string instead.

Two harder questions go further. One is about a team whose job steps are lines of text that a loop runs, and the other about turning a user’s list of file-name patterns into a find command, an empty list included. Before you open either, write down the first command you would type to watch it go wrong:

Key takeaways

• A command kept in a string loses its quoting, because quotes that come out of a variable are only characters; an array keeps each argument as one element, and "${cmd[@]}" hands each one on as one word.
• An argument list starts as an empty array and grows with +=( ), so an option that does not apply leaves no empty argument behind, and printf '[%s]\n' "${cmd[@]}" prints it one argument per line for a dry run.
• mapfile -t loads one element per line, empty lines and * included, from < file or < <(command); an array filled from an unquoted $( ) is split and globbed, and one filled through a pipe is lost in a subshell.
• read -r -a splits a single line into fields at IFS without globbing, and "${a[*]}" joins an array with the first character of IFS, which belongs in a subshell or in a function’s local IFS.
• Array features arrived over many releases, mapfile in 4.0, mapfile -d and empty arrays allowed under set -u in 4.4, so a script for the Bash 3.2 of a Mac needs a while read loop and the ${a[@]+"${a[@]}"} form.