Bash flashcards: the shell, scripts and tools
Recall cards for every Bash lesson: expansion and quoting, tests, loops, pipes, functions, arrays, errors, text tools, processes, portability and production.
482 cards. Read the question, answer it in your head, then open the card to check.
-
What is the difference between a terminal emulator and a shell?
The terminal emulator is the window: it draws characters and sends your keystrokes on, and runs no command itself. The shell, such as Bash, zsh or dash, is the program inside it that reads each command line, expands it and starts the programs it names.
From the lesson Shells, terminals and which Bash you have
-
Why does echo $SHELL not tell you which shell is running a script?
SHELL names your login shell, the one your account starts when you log in. A script can run under a different shell, so ask the running shell itself through BASH_VERSION or BASH_VERSINFO.
From the lesson Shells, terminals and which Bash you have
-
Which checks tell you which Bash you are running, and which version it is?
In an interactive shell $0 gives the name the shell was started with. BASH_VERSION and the BASH_VERSINFO array describe the running shell, bash --version reports the bash found first on PATH, and type -a bash lists every bash on PATH in search order.
From the lesson Shells, terminals and which Bash you have
-
Which Bash does macOS keep at /bin/bash, and how do you get a current one?
Bash 3.2.57. Install Bash 5.3 with Homebrew on macOS, with the package manager on Linux, or through WSL or Git Bash on Windows, and then read its version line.
From the lesson Shells, terminals and which Bash you have
-
Which document defines the sh language, and how do Bash, dash and zsh relate to it?
POSIX.1-2024 defines it. dash implements it closely, Bash implements it and adds many extensions, and zsh, the default login shell of macOS from 10.15, is a separate shell with its own language.
From the lesson Shells, terminals and which Bash you have
-
Into which parts does the command line head -n 5 notes.txt split?
The command name head, the option -n, its option-argument 5, and the operand notes.txt, which is what the command works on.
From the lesson Commands, options, arguments and help
-
What does -- do in rm -- "$file"?
It ends the options. Every word after it is an operand, even one that begins with a dash, so a file called -rf is removed as a file and never read as options.
From the lesson Commands, options, arguments and help
-
Why can rm * go wrong in a folder that holds a file named -n or -rf?
The glob hands each name to rm as an ordinary word, so a name starting with a dash is taken as an option. Write rm -- * or rm ./* so every name stays a file.
From the lesson Commands, options, arguments and help
-
In which order does Bash resolve a command name?
Aliases and keywords first, then functions, then builtins, and last the programs it finds through PATH. type -a NAME lists every match in that order.
From the lesson Commands, options, arguments and help
-
What is the portable test for whether a command exists?
command -v NAME. It prints how the shell would run the name: a path for a program, the bare name for a builtin or function, the definition for an alias. When nothing matches it prints nothing and ends with status 1.
From the lesson Commands, options, arguments and help
-
Where do you look up a builtin, an installed program and a tool's own options?
help NAME documents Bash builtins, man NAME opens the manual page of an installed program, and NAME --help prints the option list that many tools carry themselves.
From the lesson Commands, options, arguments and help
-
How does an absolute path differ from a relative one?
An absolute path starts at / and names the same file from anywhere. A relative path is read from the current folder that pwd prints, so docs/notes.txt means a different file in each folder. A path that starts with ~ begins at your home folder.
From the lesson Files and directories from the command line
-
Can you get back what rm -r deleted?
No. The shell has no recycle bin: rm -r removes the folder and everything in it at once and for good, so look at what a delete will touch before you run it.
From the lesson Files and directories from the command line
-
Why can ls and the glob * miss files that a copy or delete should include?
Names that begin with a dot, such as .env, are hidden from both. List the folder with ls -a before copying or deleting it.
From the lesson Files and directories from the command line
-
How do you keep rm -rf "$dir"/ from widening when dir is empty?
Write "${dir:?}". An empty or unset dir then stops the command with an error instead of turning the path into /. Printing the command with echo first shows what it would remove.
From the lesson Files and directories from the command line
-
Which commands create nested folders, copy a whole tree, and rename a file?
mkdir -p creates a folder together with any missing parents, cp -R copies a tree, and mv both moves and renames.
From the lesson Files and directories from the command line
-
Which commands show what a file holds without an editor?
cat prints it, head and tail show its first and last lines, wc -l counts its lines, file names its type, and less pages through a long one.
From the lesson Files and directories from the command line
-
What does the first line #!/usr/bin/env bash do?
When the file is run by its path, the kernel reads that line and starts env, which finds bash on PATH and gives it the script. To Bash itself the line is only a comment.
From the lesson Your first script: shebang, chmod and running it
-
What does each of bash hello.sh, ./hello.sh and source hello.sh need?
bash hello.sh needs only read permission. ./hello.sh needs execute permission, set with chmod +x, and a shebang that names its interpreter. source hello.sh runs the commands in your current shell, so the variables and folder changes they make stay in it.
From the lesson Your first script: shebang, chmod and running it
-
What do exit statuses 126 and 127 mean?
126 means the command was found but could not be executed, often a missing execute bit. 127 means it was not found at all. 0 is success, and any other value is the program's own choice of how it failed.
From the lesson Your first script: shebang, chmod and running it
-
What does bash -n script.sh check, and what does ShellCheck add?
bash -n reads the whole script for syntax errors without running a single command. ShellCheck goes further and points out likely bugs, such as a variable left unquoted.
From the lesson Your first script: shebang, chmod and running it
-
Why does a script saved with Windows line endings fail with errors such as $'ls\r': command not found?
Every line ends in a carriage return before its newline, and Bash reads that \r as part of the last word, so ls becomes ls\r. Removing the carriage returns, for example with tr -d '\r', repairs the file.
From the lesson Your first script: shebang, chmod and running it
-
Why can a semicolon stored in a variable never start a second command?
Bash cuts the line into tokens, words and operators such as ; | && and >, before it expands anything. A semicolon that arrives later from a variable's value is plain text inside a word.
From the lesson How Bash reads a line: tokens and expansions
-
In which order does Bash expand the words of a command?
Brace expansion first; then tilde, parameter and arithmetic expansion and command substitution together, from left to right; then word splitting of unquoted results; then filename expansion; and quote removal last.
From the lesson How Bash reads a line: tokens and expansions
-
Which results of expansion are split into words and globbed?
Only unquoted ones. A quoted variable such as "$x" stays one word whatever its value holds, which is why quoting a variable keeps spaces and wildcards from changing the command.
From the lesson How Bash reads a line: tokens and expansions
-
How can you see the exact words a command receives after expansion?
Run the script with bash -x, or turn on set -x inside it. Each command is printed after expansion, behind the PS4 prefix, which is + and a space by default.
From the lesson How Bash reads a line: tokens and expansions
-
What does each of single quotes and double quotes protect?
Single quotes keep every character literal: no variable, substitution or backslash works inside them. Double quotes keep spaces and wildcards literal but still let $, backticks and backslash work, so "$name" is one word holding the variable's value.
From the lesson Quoting: single, double, backslash and $'...'
-
How do you put an apostrophe inside a single-quoted string?
Close the quotes, write an escaped quote, and open them again, as in 'it'\''s'. Single quotes themselves can never contain a single quote.
From the lesson Quoting: single, double, backslash and $'...'
-
Can a command tell which kind of quoting you used for its argument?
No. Quote removal is the last step of expansion, so the command sees only the resulting words; printf '[%s]\n' with the same arguments shows exactly what arrived.
From the lesson Quoting: single, double, backslash and $'...'
-
What does $'...' quoting do?
It turns backslash escapes such as \t, \n, \x41 and \u00e9 into the characters they name. POSIX.1-2024 includes it, and both Bash and dash accept it.
From the lesson Quoting: single, double, backslash and $'...'
-
How should text that may contain quotes reach a command you build?
A value a user supplies goes in as a parameter, so no character can change its meaning. Pasted into the text unescaped, its own quotes end the string early; text you must build needs the quotes escaped in the value, not in the format.
From the lesson Quoting: single, double, backslash and $'...'
-
What two extra steps does an unquoted $var go through?
Word splitting at the characters of IFS, so my notes.txt becomes two words, and then globbing, so a value holding * can turn into a list of file names.
From the lesson Word splitting and IFS: why "$var" needs quotes
-
What is IFS, and what does it hold by default?
The internal field separator: the characters Bash splits unquoted expansions on, and read splits lines on. By default it is a space, a tab and a newline.
From the lesson Word splitting and IFS: why "$var" needs quotes
-
How do whitespace and other IFS characters split a value differently?
A run of IFS whitespace counts as one separator, but every other IFS character separates a field on its own: with IFS=: the value x:y::z splits into four words, one of them empty.
From the lesson Word splitting and IFS: why "$var" needs quotes
-
What happens to an empty variable written unquoted as an argument?
It disappears completely: the command gets one argument fewer, and every argument after it moves up one place.
From the lesson Word splitting and IFS: why "$var" needs quotes
-
How do "$@" and "$*" differ?
"$@" passes each argument on as its own word, exactly as received. "$*" makes a single string of them all, separated by IFS's first character. Unquoted, both split again.
From the lesson Word splitting and IFS: why "$var" needs quotes
-
How should a script change IFS?
For one command only, as in IFS=, read -r a b, or inside a subshell. A global change affects every later unquoted expansion in the script.
From the lesson Word splitting and IFS: why "$var" needs quotes
-
Which characters make a word a glob pattern, and what does each match?
* matches any string, ? one character, [...] one character from a set, and [!...] one character not in the set. A leading dot in a name must be matched explicitly.
From the lesson Globbing: wildcards, dotfiles, extglob, globstar
-
What does Bash do by default with a glob that matches no file?
It leaves the pattern as its own text, so ls *.md in a folder without Markdown files gets the word *.md and reports that no such file exists.
From the lesson Globbing: wildcards, dotfiles, extglob, globstar
-
How do nullglob and failglob change an unmatched pattern?
With nullglob it expands to nothing, which suits loops that may have no work. With failglob it is an expansion error and the command does not run, which suits destructive commands.
From the lesson Globbing: wildcards, dotfiles, extglob, globstar
-
What do the globstar and extglob options add?
globstar makes ** match folders at any depth. extglob adds patterns such as !(...) for anything but a pattern and +(...) for one or more; it must be on before Bash reads the line that uses them.
From the lesson Globbing: wildcards, dotfiles, extglob, globstar
-
What does GLOBSORT change in Bash 5.3?
The order of glob matches: they can be sorted by size, by time or by other keys instead of by name. Older versions ignore the variable.
From the lesson Globbing: wildcards, dotfiles, extglob, globstar
-
What does brace expansion do, and when does it run?
It turns a list {a,b} or a range {1..9..2} into several words, copying the text around the braces into each one. It is the first expansion Bash performs.
From the lesson Brace and tilde expansion
-
Why does echo {1..$n} not print a range?
The braces are expanded while the word still holds the text $n, so the range is not valid and stays as written. Use for ((i = 1; i <= n; i++)) or seq 1 "$n" instead.
From the lesson Brace and tilde expansion
-
What can nested and repeated braces build in one command?
Whole folder trees, such as mkdir -p project/{src,docs}/{v1,v2}, and backup names such as cp config{,.bak}. ShellCheck flags braces in sh scripts as SC3009.
From the lesson Brace and tilde expansion
-
Where does a tilde expand, and what do ~+ and ~- mean?
A tilde is expanded only when it begins a word, or follows = or : in an assignment; quoted, it stays a plain tilde. ~+ gives the current folder and ~- the previous one.
From the lesson Brace and tilde expansion
-
What does printf do with more arguments than its format uses?
It reuses the format until every argument is printed, and it never adds a newline you did not write.
From the lesson printf and echo: printing text reliably
-
What do %-20s, %10.2f and %05d do in a printf format?
%-20s left-aligns text in 20 columns, %10.2f prints a number with two decimals in 10 columns, and %05d pads a whole number with zeros to 5 digits.
From the lesson printf and echo: printing text reliably
-
Why is echo unreliable for printing data?
It behaves differently between shells: dash's echo interprets backslashes and prints -e as text, Bash's does not without -e, and both read a value such as -n as an option.
From the lesson printf and echo: printing text reliably
-
What is the reliable way to print the value of a variable?
printf '%s\n' "$var". The value goes into an argument, never into the format, so backslashes, percent signs and leading dashes print as written.
From the lesson printf and echo: printing text reliably
-
What do printf -v and %q do?
printf -v name stores the formatted text in a variable without a subshell. %q prints a value quoted so that the shell can read it back as input.
From the lesson printf and echo: printing text reliably
-
Why should a Bash script add up money in whole paise rather than in rupees with decimals?
Bash arithmetic is integer only, so a decimal amount cannot be summed exactly. Count every amount in paise, add those whole numbers, and print rupees and paise separately with a format such as %d.%02d.
From the lesson printf and echo: printing text reliably
-
Why does city = Pune fail in Bash?
With spaces around the equals sign it is no longer an assignment: Bash runs a command named city with the arguments = and Pune, and answers city: command not found. Write city=Pune.
From the lesson Variables, export and the environment
-
How can a script tell an unset variable from an empty one?
With ${name-word}, which gives word only when name is unset and keeps an empty value as it is. ${name:-word} treats unset and empty alike.
From the lesson Variables, export and the environment
-
What does declare -p name print?
The variable as the declare command that would recreate it, attributes included, which makes it the clearest way to see what a variable is and holds.
From the lesson Variables, export and the environment
-
Which variables does a program started from the shell receive?
Only exported ones, set with export or declare -x. The child gets a copy, so a change it makes never travels back to the parent shell.
From the lesson Variables, export and the environment
-
How do you give one command a variable without changing the shell?
Put the assignment in front of it, as in LC_ALL=C sort file. The variable is in that command's environment only, and the shell's own value stays as it was.
From the lesson Variables, export and the environment
-
Why should a script name its own variables in lower case?
Capitalised names such as PATH and HOME belong to the shell and other programs. Overwriting PATH by accident makes every later program fail with command not found, status 127.
From the lesson Variables, export and the environment
-
Why does "$10" print the first argument followed by a 0?
Bash reads $10 as $1 followed by the digit 0. From the tenth argument on the number needs braces, "${10}", to be one name.
From the lesson Positional and special parameters
-
What do $# and "$@" give inside a script?
$# is the number of arguments it received. "$@" expands to every argument as its own word, exactly as the caller gave them, which is how they are passed on.
From the lesson Positional and special parameters
-
What do shift and set -- do to the argument list?
shift drops the first argument and moves the rest down, lowering $# by one. set -- a b c replaces the whole list with new words.
From the lesson Positional and special parameters
-
Why do option loops look at $1 and call shift?
Each pass reads one option and removes it, so when the loop ends, "$@" holds exactly the arguments it did not consume, ready for the rest of the script.
From the lesson Positional and special parameters
-
Why is $? saved with rc=$? straight after the command it belongs to?
Every command replaces $?, even an echo or a test, so the status of the command you care about is gone after the next line.
From the lesson Positional and special parameters
-
What do $$, BASHPID, $! and $- hold?
$$ is the main shell's process ID, the same inside subshells; BASHPID is the ID of the process running now; $! is the ID of the last background job; and $- lists the shell's current option flags.
From the lesson Positional and special parameters
-
When do ${v-x} and ${v:-x} give different results?
Only when v is set to the empty string: ${v-x} keeps the empty value, because the variable is set, while ${v:-x} treats empty like unset and gives x.
From the lesson Defaults and required values: ${v:-x}, ${v:?}
-
What does : "${PORT:=8080}" do at the top of a script?
It assigns 8080 to PORT when PORT is unset or empty, and the colon command ignores the result, so every default of the script can be set in one place.
From the lesson Defaults and required values: ${v:-x}, ${v:?}
-
How can an expansion add a flag only when a variable has a value?
${verbose:+--verbose} expands to --verbose when verbose is set and non-empty, and to nothing otherwise; left unquoted, the empty case adds no argument.
From the lesson Defaults and required values: ${v:-x}, ${v:?}
-
How does ${dir:?} protect rm -rf -- "${dir:?}"/*?
When dir is unset or empty the expansion prints an error and a non-interactive shell stops, so the command never runs and can never widen to the root folder.
From the lesson Defaults and required values: ${v:-x}, ${v:?}
-
What does the colon change in -, =, + and ? expansions?
In each form the colon widens unset to unset or empty: without it, an empty variable counts as having a value; with it, empty is treated like unset.
From the lesson Defaults and required values: ${v:-x}, ${v:?}
-
What do ${#s} and ${s:offset:length} give?
${#s} is the number of characters in s. ${s:offset:length} takes length characters starting offset characters in; a negative offset counts from the end.
From the lesson String operations with parameter expansion
-
Why must ${s: -5} have a space before the minus sign?
Without it, ${s:-5} is the default form, which gives 5 only when s is unset or empty. The space makes it a substring of the last five characters.
From the lesson String operations with parameter expansion
-
What do #, ##, % and %% remove in a parameter expansion?
# and ## remove a matching prefix, % and %% a matching suffix. One sign removes the shortest match of the pattern, two signs the longest.
From the lesson String operations with parameter expansion
-
How do you get the file name and the folder of a path without basename and dirname?
${path##*/} removes everything up to the last slash, leaving the name, and ${path%/*} removes the last slash and what follows, leaving the folder. Neither starts a process.
From the lesson String operations with parameter expansion
-
What does an ampersand mean in the replacement of ${s//pat/text} since Bash 5.2?
The matched text, because the patsub_replacement option is on by default: ${s//at/[&]} turns cat into c[at]. Escape it as \& for a literal ampersand.
From the lesson String operations with parameter expansion
-
Which expansions change the case of a value, and which version added them?
${s^^} makes it upper case, ${s,,} lower case and ${s^} capitalises the first character. They need Bash 4.0 or later, and POSIX sh has none of them.
From the lesson String operations with parameter expansion
-
What does ${!name} expand to?
The value of the variable whose name name holds: Bash reads name, takes its value as a second variable's name and expands that one instead.
From the lesson Indirection, namerefs and ${var@Q} transforms
-
How do you list the variables whose names start with a prefix?
${!prefix*} expands to the names of every variable that begins with prefix.
From the lesson Indirection, namerefs and ${var@Q} transforms
-
How can a function fill an array that its caller names?
The caller passes the array's name, and the function declares local -n out=$1. Every assignment to out then writes the caller's variable.
From the lesson Indirection, namerefs and ${var@Q} transforms
-
What goes wrong when a nameref has the same name as the caller's variable?
The nameref then names itself: Bash warns of a circular name reference and writes a global variable of that name, so a caller's local copy is left unchanged. Give namerefs names a caller is unlikely to use, such as _out.
From the lesson Indirection, namerefs and ${var@Q} transforms
-
Why use namerefs or indirection rather than eval for names chosen at run time?
eval runs its argument as code, so a value with shell syntax in it runs as a command. Namerefs and indirection only look names up and never run a value.
From the lesson Indirection, namerefs and ${var@Q} transforms
-
What does ${var@Q} produce?
The value quoted so that the shell reads it back unchanged: a value of it's here comes out as 'it'\''s here', which shows exactly what the variable holds.
From the lesson Indirection, namerefs and ${var@Q} transforms
-
What kind of numbers does Bash arithmetic use?
Signed 64-bit whole numbers. There are no decimals, and one more than 9223372036854775807 wraps around to the most negative value without an error.
From the lesson Integer arithmetic: $(( )), (( )) and let
-
What do $((-7 / 2)) and $((-7 % 3)) give in Bash?
-3 and -1. Bash cuts the fraction off towards zero, so -3.5 becomes -3, and the result of % is negative here because -7, the number divided, is negative.
From the lesson Integer arithmetic: $(( )), (( )) and let
-
Why does $((08)) fail with value too great for base?
A leading 0 makes Bash read the number as octal, and 8 is not an octal digit. Zero-padded text such as days and months needs 10# in front, as in $((10#$day + 1)).
From the lesson Integer arithmetic: $(( )), (( )) and let
-
When does (( expression )) succeed as a command?
When the expression is non-zero; it fails, with status 1, when the expression is zero. That lets it act as a condition in if and while.
From the lesson Integer arithmetic: $(( )), (( )) and let
-
Why can ((count++)) stop a set -e script?
When count is 0 the expression's value is 0, so the command fails and set -e ends the script. count=$((count + 1)) is an assignment, which always succeeds.
From the lesson Integer arithmetic: $(( )), (( )) and let
-
How should a Bash script handle fractions and money?
Hand fractions to awk, which computes in floating point, and keep money in whole hundredths, split so that the shares add up exactly to the total.
From the lesson Integer arithmetic: $(( )), (( )) and let
-
Which exit status means success, and where does the shell keep the last one?
Only 0 means success; every value from 1 to 255 is a failure. The status of the most recent command is in $?, which changes after every command, while true and : always succeed.
From the lesson Exit status, &&, || and command lists
-
What do the statuses 1, 2, 126, 127 and 128 plus n usually mean?
1 is a plain failure and 2 a misuse, such as a bad option; 126 is a command that could not execute and 127 one that was not found; 128 plus n is a command that signal n stopped.
From the lesson Exit status, &&, || and command lists
-
Which statuses does grep end with?
0 when a line matched, 1 when nothing matched and 2 for an error such as a missing file, so a script can tell no match from a failure.
From the lesson Exit status, &&, || and command lists
-
Which numbers should a script choose for its own exit codes?
Values from 1 to 125, set with exit n, so they never collide with 126, 127 and the 128 plus n statuses of signals.
From the lesson Exit status, &&, || and command lists
-
When does the next command run after ;, && and ||?
After ; it always runs, after && only when the previous command succeeded, and after || only when it failed. ! in front of a command inverts its status.
From the lesson Exit status, &&, || and command lists
-
Why is a && b || c not the same as if a; then b; else c; fi?
The operators group from the left, so it means (a && b) || c: c runs when a fails, but also when a succeeds and b fails. Real branches belong in if.
From the lesson Exit status, &&, || and command lists
-
What status does a command started with & give back at once?
0, straight away, whatever the job will do. Its real status comes from wait once the job ends.
From the lesson Exit status, &&, || and command lists
-
What are test and [ ], and how do they report their result?
One command under two names. It ends with 0 for true, 1 for false and more than 1 for an error, every operand and operator must be a separate word, and [ needs ] as its last argument.
From the lesson test and [ ]: file, string and number checks
-
Which operators of [ ] compare strings, compare integers and check files?
= and != compare strings; -eq, -ne, -lt, -le, -gt and -ge compare integers; -e, -f, -d and -s check one file, and -nt, -ot and -ef compare two.
From the lesson test and [ ]: file, string and number checks
-
What happens to [ $x = "" ] and [ -n $x ] when x is empty?
The empty value disappears before [ runs. The first becomes [ = "" ] and fails with unary operator expected, status 2; the second becomes [ -n ], which is true because the word -n is not empty.
From the lesson test and [ ]: file, string and number checks
-
Why should every variable inside [ ] be quoted?
"$x" is always exactly one argument, so the test means what it says even when the value is empty or contains spaces. Unquoted, an empty value vanishes and one with spaces splits into several words.
From the lesson test and [ ]: file, string and number checks
-
How do portable scripts join two tests?
With && and || between separate [ ] commands. POSIX.1-2024 removed -a, -o and grouping from test, and dash refuses ==, so portable tests use = for strings.
From the lesson test and [ ]: file, string and number checks
-
Why does [ apple > banana ] not compare the two words?
An unquoted > is a redirection: the test checks the single word apple and creates a file named banana. Inside [ ], < and > compare strings only when quoted or escaped.
From the lesson test and [ ]: file, string and number checks
-
Why are values inside [[ ]] never split or globbed?
[[ ]] is a keyword that Bash parses itself, not a command whose arguments are expanded first. Inside it, && and || combine tests, and < and > compare text, not numbers.
From the lesson [[ ]]: patterns, regex and BASH_REMATCH
-
What does the right side of == mean inside [[ ]]?
A glob pattern, so [[ $file == *.log ]] matches any name ending in .log. Quoting the right side, or a part of it, makes that part literal text.
From the lesson [[ ]]: patterns, regex and BASH_REMATCH
-
Where in the string does [[ $s =~ regex ]] look for a match?
Anywhere, unless the expression is anchored with ^ and $. The right side is a POSIX extended regular expression.
From the lesson [[ ]]: patterns, regex and BASH_REMATCH
-
Where do the capture groups of an =~ match end up?
In the array BASH_REMATCH: element 0 holds the whole match and elements 1 onwards the text of each group in round brackets.
From the lesson [[ ]]: patterns, regex and BASH_REMATCH
-
Why does [[ $s =~ "$re" ]] not use re as a regular expression?
Quoting makes the right side literal text, so the whole pattern is matched as a plain string. Store the expression in a variable and leave the variable unquoted after =~.
From the lesson [[ ]]: patterns, regex and BASH_REMATCH
-
What happens to [[ ]] in a script run by dash?
dash has no [[ ]], so the line fails, and ShellCheck reports it as SC3010 in sh scripts. Portable scripts keep to [ ] and case.
From the lesson [[ ]]: patterns, regex and BASH_REMATCH
-
What does if actually test?
The exit status of any command list: if grep -q ERROR app.log; then runs grep and branches on its status. Brackets are needed only when the condition is a test, such as [ -f "$file" ].
From the lesson if, elif, else and case statements
-
How do elif and else decide which branch runs?
if runs its first list; when it fails, each elif list is tried in turn, and else runs only when none of them succeeded.
From the lesson if, elif, else and case statements
-
In which order does case try its patterns?
From the top, and it runs only the first match, so specific patterns go first and *) at the end catches everything else. When nothing matches, case ends with status 0.
From the lesson if, elif, else and case statements
-
Which pattern forms can a case clause use?
Globs, alternatives joined with | such as y | yes, and bracket expressions such as [!0-9] or classes such as [[:upper:]], all of which work in every POSIX shell.
From the lesson if, elif, else and case statements
-
What do ;& and ;;& do at the end of a case clause?
;& runs the next clause's commands without testing its pattern, and ;;& goes on testing the patterns that follow. Bash has both since 4.0; POSIX.1-2024 has ;&, but dash 0.5.13.5 refuses it.
From the lesson if, elif, else and case statements
-
When is an associative array better than a long case?
When the branches only map a key to a value, such as a service name to its port: one lookup replaces many clauses that each set a variable.
From the lesson if, elif, else and case statements
-
How does for name in words decide how many passes it makes?
It expands the words once, then runs the body once for each word left, so quoting decides the passes: "$var" and "${array[@]}" keep every item whole.
From the lesson for loops: lists, globs, ranges and C-style
-
What does a for loop written for arg; do ... done loop over?
The positional parameters, exactly as for arg in "$@" would: one pass per argument, each kept whole, empty ones included. POSIX defines the short form, so dash accepts it too.
From the lesson for loops: lists, globs, ranges and C-style
-
Why is a glob the right list for a loop over files?
It gives the loop each matching name as one sorted word that is never split, even when the name holds spaces; quote "$f" wherever the body uses it.
From the lesson for loops: lists, globs, ranges and C-style
-
What does for f in *.csv do in a folder with no CSV files, and how is it handled?
It makes one pass with f set to the pattern itself, *.csv. Skip that pass with [[ -e $f ]] || continue, or turn on shopt -s nullglob so the loop does not run at all.
From the lesson for loops: lists, globs, ranges and C-style
-
When do you use a brace range and when the C-style for (( ))?
A brace range such as {1..10} needs bounds typed into the script. The C-style for (( i = 1; i <= n; i++ )) takes variables and any step. dash prints brace ranges as written and refuses the C-style form.
From the lesson for loops: lists, globs, ranges and C-style
-
Is seq part of the shell?
No. seq is an outside program, and POSIX does not define it, so a loop that relies on it depends on the system having it.
From the lesson for loops: lists, globs, ranges and C-style
-
Why are for f in $(ls) and for line in $(cat file) wrong?
The output is split at spaces, tabs and newlines and then globbed, so names and lines break into words. Loop over files with a glob and over lines with while read, as ShellCheck's SC2045 and SC2013 advise.
From the lesson for loops: lists, globs, ranges and C-style
-
What does the loop variable hold after a for loop ends?
Its last value, and in the C-style form one step past the last pass. A for ... in loop that makes no pass never sets it, and inside a function it should be declared local.
From the lesson for loops: lists, globs, ranges and C-style
-
How do while and until differ?
Both run a test command before every pass. while repeats the body as long as the test succeeds, until as long as it fails, so until cmd reads as while not cmd.
From the lesson while, until, break, continue and select
-
Which status does a while or until loop end with?
The status of the last body command it ran, or 0 when the body never ran.
From the lesson while, until, break, continue and select
-
What do break, continue and break 2 do?
break ends the innermost loop and continue starts its next pass at once. With a number, break 2 or continue 2 acts on the loop around it.
From the lesson while, until, break, continue and select
-
How does the shell write a loop whose exit test sits at the end?
while : with a break inside the body. The colon always succeeds, so only the break ends the loop; the shell has no do-while.
From the lesson while, until, break, continue and select
-
What does a sound retry loop need?
A limit on attempts, a pause that grows after each failure, no pause after the last attempt, and the last attempt's status at the end so the caller can see the failure.
From the lesson while, until, break, continue and select
-
How does a polling loop avoid waiting for ever?
It checks the state at a fixed interval and also stops at a deadline, for example one counted with Bash's SECONDS variable.
From the lesson while, until, break, continue and select
-
What does select do with the answer it reads?
It prints a numbered menu on standard error, prompted by PS3, and reads a line into REPLY. A listed number sets the loop variable to that word, any other text empties it, and an empty line shows the menu again.
From the lesson while, until, break, continue and select
-
When does a select loop end, and which shells have it?
It goes on until a break, or until input ends, when it finishes with status 1. Bash has it; POSIX sh and dash do not.
From the lesson while, until, break, continue and select
-
Why do careful read loops write IFS= and -r?
A plain read trims spaces and tabs at both ends of the line and treats backslashes as escapes. IFS= keeps the edges and -r keeps every backslash, in Bash and in dash alike.
From the lesson Reading files line by line with read
-
What is the loop that reads a file one line at a time, each line as written?
while IFS= read -r line; do ...; done < file. It reads one line per pass and keeps its spaces and backslashes.
From the lesson Reading files line by line with read
-
What does read do with a last line that has no newline?
It still stores the line but returns 1, so the loop would skip it. Adding || [[ -n $line ]] to the test gives that line its pass.
From the lesson Reading files line by line with read
-
Why is a count made inside cmd | while read ... lost after done?
Each part of a pipeline runs in a subshell, so the loop's variables vanish when it ends. Feed the loop with done < file or done < <(command) instead, or set lastpipe in a script.
From the lesson Reading files line by line with read
-
How does read split a line between several variable names?
At the characters of IFS, one field per name, and the last name takes the rest of the line, delimiters included.
From the lesson Reading files line by line with read
-
How do you read file names safely when one may contain a newline?
Separate them with NUL bytes: find -print0 writes them that way, and read -r -d '' reads one name per call. dash 0.5.13.5 does not have read -d.
From the lesson Reading files line by line with read
-
Which three descriptors does every command start with?
0 for standard input, where it reads; 1 for standard output, for results; and 2 for standard error, for warnings and errors. In a terminal, 1 and 2 both go to the screen.
From the lesson Redirection: stdin, stdout, stderr, /dev/null
-
How do > file and >> file differ, and when is the file opened?
> empties the file and writes it, >> appends to it, and 2> and 2>> do the same for standard error. The shell carries out every redirection before the command starts, so a folder that does not exist stops the command before it runs.
From the lesson Redirection: stdin, stdout, stderr, /dev/null
-
Why does cmd > file 2>&1 put both streams in the file while cmd 2>&1 > file does not?
Redirections run from left to right, and 2>&1 copies where descriptor 1 points at that moment. In the second form, standard error copies the terminal or pipe before standard output moves to the file.
From the lesson Redirection: stdin, stdout, stderr, /dev/null
-
Why should a script that sh may run avoid &> file?
It is Bash's short form of > file 2>&1. dash reads it as & followed by > file: it runs the command in the background and empties the file. Bash 3.2 also lacks &>>, so portable scripts write the long forms.
From the lesson Redirection: stdin, stdout, stderr, /dev/null
-
What does a redirection to /dev/null hide?
Only the text: /dev/null discards what is written to it and reads as empty, but the command's exit status is still there for if, && and ||.
From the lesson Redirection: stdin, stdout, stderr, /dev/null
-
How do you test whether a file contains a match without printing anything?
grep -q prints nothing and reports through its status, 0 for a match, and it can stop at the first match instead of reading the whole file.
From the lesson Redirection: stdin, stdout, stderr, /dev/null
-
What do set -C and >| do?
set -C, noclobber, makes > refuse to overwrite an existing regular file. >| overrides it for one redirection, while >>, new files and /dev/null still work.
From the lesson Redirection: stdin, stdout, stderr, /dev/null
-
What does tee do, and how do you keep it from emptying its files?
It copies its standard input to its standard output and to every file it names. It empties those files first unless -a makes it append.
From the lesson Redirection: stdin, stdout, stderr, /dev/null
-
Which status does a pipeline return by default?
Its last command's, so false | true returns 0 and the failure of false is lost unless you look for it.
From the lesson Pipelines, PIPESTATUS and pipefail
-
Why does a variable set inside a pipeline stage vanish afterwards?
The stages run at the same time as separate processes, each in a subshell, so whatever a stage sets is gone when the pipeline ends.
From the lesson Pipelines, PIPESTATUS and pipefail
-
Which stream does | carry, and how do you send standard error through too?
Only standard output. |& or 2>&1 | adds standard error to what the next command reads.
From the lesson Pipelines, PIPESTATUS and pipefail
-
How do you keep every stage's status of a pipeline?
Copy the PIPESTATUS array on the same line, before anything else runs, with p=("${PIPESTATUS[@]}"), and read the copy, because the next command replaces it.
From the lesson Pipelines, PIPESTATUS and pipefail
-
What does set -o pipefail change?
The pipeline returns the status of its rightmost command that failed, or 0 when every stage succeeded. Bash has had it since 3.0, and POSIX.1-2024 and dash 0.5.13.5 have it too.
From the lesson Pipelines, PIPESTATUS and pipefail
-
Why can a healthy pipeline such as cmd | head -n 1 report 141 under pipefail?
When the reader stops early, the writer is killed by SIGPIPE, signal 13, and Bash reports 128 plus 13. Decide from PIPESTATUS, stage by stage, where a 141 is expected.
From the lesson Pipelines, PIPESTATUS and pipefail
-
What does a here-document feed to a command?
Every line between <<word and a line that holds only word, as the command's standard input.
From the lesson Here-documents and here-strings
-
How do <<EOF, <<'EOF' and <<-EOF differ?
<<EOF expands variables, command substitutions and arithmetic in the body. Quoting any part of the word, as in <<'EOF', keeps the body literal. <<-EOF strips leading tabs, never spaces, from each line and from the closing delimiter.
From the lesson Here-documents and here-strings
-
What must a script do before it renders a config file from variables with cat > file <<EOF?
Check each caller's value for newlines and quote it for the file's language, because Bash inserts every value as it is. A template is never rendered with eval.
From the lesson Here-documents and here-strings
-
What does a here-string such as wc -c <<< abc feed to the command?
The word followed by a newline that Bash adds, so wc -c <<< abc printed 4 and wc -c <<< '' printed 1.
From the lesson Here-documents and here-strings
-
Can a sh script use <<<?
No. The here-string is a Bash extension: dash answers Syntax error: redirection unexpected, and ShellCheck flags <<< as SC3011 in a script whose shebang is #!/bin/sh.
From the lesson Here-documents and here-strings
-
What does exec do when it is given only redirections?
It runs no program: it changes the running shell's own descriptors, and the change stays in force for every later command of the script.
From the lesson File descriptors and exec redirections
-
How do you open, use and close descriptor 3 for a trace file?
exec 3> trace.log opens it, emptying the file first; echo text >&3 writes one command's output through it; and exec 3>&- closes it.
From the lesson File descriptors and exec redirections
-
Why should a script close the descriptors it opened with exec?
Every program it starts inherits them. Close them when done, and start a program that must not have one with a redirection such as 3>&-.
From the lesson File descriptors and exec redirections
-
How do you move standard output for a while and put it back?
Save it first with exec 5>&1, and restore it by copying back with exec 1>&5 5>&-. It returns to wherever standard output pointed when it was saved, the terminal or the caller's file.
From the lesson File descriptors and exec redirections
-
What does the {var}> form of redirection do?
Bash 4.1 and later choose an unused descriptor of 10 or more and store its number in var. It stays open after its command until exec {var}>&- closes it; Bash 3.2 rejects the form.
From the lesson File descriptors and exec redirections
-
How can one loop read two files at the same time?
Give the second file a descriptor of its own, such as 3 or 4, and point a second read at it with -u 3 or with <&4, while the loop's first read takes standard input. <> opens a file for reading and writing without emptying it.
From the lesson File descriptors and exec redirections
-
What does $(cmd) drop from the command's output?
Every newline at its end, however many there are; newlines inside the output stay. Quote the result so word splitting leaves it whole.
From the lesson Command and process substitution
-
How do you capture output with its trailing newlines intact?
Print a sentinel character after the command inside the same substitution, then remove that one character from the variable afterwards.
From the lesson Command and process substitution
-
Which status does x=$(cmd) leave, and why is local x=$(cmd) different?
A plain assignment keeps the status of its substitution, so x=$(cmd) || ... sees a failure. local, export, declare and readonly report their own status, so declare first and assign on the next line.
From the lesson Command and process substitution
-
Why is $( ) preferred to backquotes?
It nests and keeps backslashes as written, where backquotes need escapes for both. $(< file) also reads a file in Bash itself without starting cat.
From the lesson Command and process substitution
-
What does <(cmd) put on the command line?
A path like /dev/fd/63 that stands for the command's output, so diff and comm can compare two commands' output without temporary files. dash rejects the form.
From the lesson Command and process substitution
-
What do Bash 5.3's ${ cmd; } and ${| cmd; } capture?
${ cmd; } captures the output in the current shell, so the command's changes to variables persist. ${| cmd; } expands to the value the command leaves in REPLY. Older shells reject both.
From the lesson Command and process substitution
-
Which way of defining a function works in every POSIX shell?
name() { ...; }. The form function name { ...; } is not POSIX: Bash and ksh accept it, dash stops with a syntax error, and ShellCheck flags it as SC2113 in sh scripts.
From the lesson Functions: arguments, local and return status
-
What do $1, "$@" and $0 mean inside a function?
$1 to $# and "$@" are the function's own arguments. $0 stays the script's name, FUNCNAME[0] gives the function's name, and the caller's arguments come back after the function returns.
From the lesson Functions: arguments, local and return status
-
What does local do, and what happens without it?
local keeps a variable inside the function and the functions it calls. A variable changed without local is the global one, so the function overwrites the caller's value.
From the lesson Functions: arguments, local and return status
-
Can a function read the local variables of the function that called it?
Yes. Bash uses dynamic scoping: a called function sees, and can change, the locals of its caller.
From the lesson Functions: arguments, local and return status
-
What status does a function return, with and without return?
Without return, whatever status the final command of its body left. return N sets N and ends the function, but only 0 to 255 survive: return 256 arrives as 0 and return 300 as 44.
From the lesson Functions: arguments, local and return status
-
When does a function used in if succeed?
When it returns status 0. return prints nothing, so the caller tests the function directly, as in if is_valid "$x"; then.
From the lesson Functions: arguments, local and return status
-
What does FUNCNEST do?
It caps how deeply function calls may nest, from Bash 4.2. With FUNCNEST=5 the sixth nested call was refused: Bash gave up the whole command that began the recursion, $? was 1, and the next line of the script still ran. Without a cap, runaway recursion runs until the stack is used up and the shell crashes.
From the lesson Functions: arguments, local and return status
-
Why can return not hand text back to the caller?
return only sets the exit status, one number from 0 to 255. A function hands data back by printing it, and the caller collects the print with v=$(fn) while still reading the status.
From the lesson Returning values from functions
-
What is lost when a function runs inside v=$(fn)?
Everything except its printed bytes and its status: the capture runs the function in a subshell, so a counter it raises or a folder it changes to is gone when the copy ends.
From the lesson Returning values from functions
-
How can a function fill a variable without running in a subshell?
Through an output parameter: the caller names the variable, and the function writes it with printf -v "$1" '%s' "$value" (Bash 3.1), a local -n nameref (Bash 4.3), or an agreed global such as REPLY.
From the lesson Returning values from functions
-
Why must a function that returns data print its messages with >&2?
Standard output is its return value. A progress line or warning printed there is captured along with the value, and the caller's next test on it fails.
From the lesson Returning values from functions
-
Why does if v=$(fn); then work, while local v=$(fn) hides a failure?
A plain assignment keeps the status of the command it captured. local reports its own status, which is 0, so the failure is lost: ShellCheck reports it as SC2155. Declare first, then assign.
From the lesson Returning values from functions
-
What layout makes a Bash script easy to read and to test?
Shebang, constants, functions, a main function, and at the end a main guard that runs main "$@" only when the file is executed.
From the lesson Script structure: main, source and libraries
-
What does source file, or . file, do differently from running the file?
It runs the file's commands in the current shell, so its assignments, functions and cd stay after it returns. return N ends the sourced file early; exit N ends the caller's shell.
From the lesson Script structure: main, source and libraries
-
Where does source look for a file given by a bare name?
In PATH first, because sourcepath is on by default, and, outside POSIX mode, in the current folder. A name with a slash is opened against the current folder.
From the lesson Script structure: main, source and libraries
-
How does a script find its own folder to load a library from it?
script_dir=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd), run before any cd. BASH_SOURCE[0] holds the path the running file was opened by.
From the lesson Script structure: main, source and libraries
-
When is [[ ${BASH_SOURCE[0]} == "$0" ]] true?
When the file is run as a program, with bash file.sh or by its path: both sides are then the file's path. When the file is sourced, $0 belongs to the shell that sources it, so the test is false and main does not run.
From the lesson Script structure: main, source and libraries
-
How do you let ShellCheck follow a file the script sources?
Run it with -x so it opens sourced files, and add a # shellcheck source=... directive when the path is built from a variable; otherwise it prints SC1091.
From the lesson Script structure: main, source and libraries
-
Which permissions does a sourced library need?
Only read permission. The script that is run by its path needs read and execute.
From the lesson Script structure: main, source and libraries
-
How do you get one element of an indexed array, count its elements and append one?
${a[2]} reads the element at subscript 2, counting from 0; ${#a[@]} counts them; and a+=(x) appends one.
From the lesson Indexed arrays: create, slice and loop
-
What does ${a[@]:start:count} give?
A copy of count elements beginning at subscript start, which is how a queue can be rotated or a run of items taken.
From the lesson Indexed arrays: create, slice and loop
-
What does ${a[-1]} give, and which Bash versions accept it?
The last element. Reading a negative subscript works from Bash 4.2 and assigning one from 4.3; the Bash 3.2 of macOS answers bad array subscript.
From the lesson Indexed arrays: create, slice and loop
-
What does unset 'a[1]' leave behind?
A sparse array: the other elements keep their subscripts, so the indices have a gap. "${!a[@]}" lists the indices that are set, and a copy b=("${a[@]}") renumbers from 0.
From the lesson Indexed arrays: create, slice and loop
-
When should a loop go over "${!a[@]}" instead of counting from 0?
When the array may be sparse. "${!a[@]}" gives the subscripts that really have values, while counting 0 to ${#a[@]} - 1 misses or invents elements after an unset.
From the lesson Indexed arrays: create, slice and loop
-
Why should unset 'a[i]' be quoted?
Unquoted, a[1] is a glob pattern. With a file named a1 in the folder it matched that name, so the shell ran unset a1 and the array kept every element; the quotes make unset see the subscript.
From the lesson Indexed arrays: create, slice and loop
-
How do "${a[@]}" and "${a[*]}" expand, and does sh have arrays?
"${a[@]}" gives one word per element and "${a[*]}" joins them with the first character of IFS. Arrays are a Bash feature: dash rejects them and ShellCheck flags them as SC3030 in a #!/bin/sh script.
From the lesson Indexed arrays: create, slice and loop
-
What goes wrong when an associative array is assigned without declare -A?
Bash makes an indexed array and reads each word key as arithmetic, where an unset name is 0, so every key lands on element 0 and the map collapses into one element.
From the lesson Associative arrays: key-value maps
-
How do you test whether a key exists in an associative array?
[[ -v m[$k] ]], which is true when the element is set, whatever its value. Reading ${m[$k]} cannot tell a missing key from a key whose value is empty.
From the lesson Associative arrays: key-value maps
-
How should a key be removed from an associative array?
unset 'm[$k]', in single quotes. An unquoted key is a glob, and a double-quoted one that came from outside the script is evaluated a second time.
From the lesson Associative arrays: key-value maps
-
In which order does a loop over "${!m[@]}" give the keys?
In no defined order, so sort the keys, for example with printf '%s\n' "${!m[@]}" | sort under LC_ALL=C, whenever output is read or compared.
From the lesson Associative arrays: key-value maps
-
How do you count or group values by key in a single pass?
(( count[$key]++ )) tallies and map[$key]+=value groups, because an element that was never set reads as 0 or as the empty string.
From the lesson Associative arrays: key-value maps
-
How can a flat associative array hold two-level data?
With a composite key that joins the parts with a separator, such as m[row,col].
From the lesson Associative arrays: key-value maps
-
Why does a command kept in a string lose its quoting?
Quote characters stored in a variable's value are plain characters by the time the value is expanded; they quote nothing. Keep the command and its arguments as array elements and run it as "${cmd[@]}", so each element reaches it as one word.
From the lesson Lists done right: arrays, mapfile and read -a
-
How do you build an argument list whose options depend on settings?
Start with an empty array and add each option that applies with +=( ). An option that does not apply leaves no empty argument behind.
From the lesson Lists done right: arrays, mapfile and read -a
-
How can a script show the exact arguments it is about to run?
printf '[%s]\n' "${cmd[@]}" prints each argument on its own line between brackets, so a space inside one, as in [disk full], cannot be taken for the end of an argument.
From the lesson Lists done right: arrays, mapfile and read -a
-
What does mapfile -t arr load?
One element per line, with the newline removed, keeping spaces, a * and empty lines exactly, and a last line without a newline too. Feed it with < file or < <(command), never through a pipe.
From the lesson Lists done right: arrays, mapfile and read -a
-
What is wrong with arr=($(command))?
The unquoted output is split at IFS and then globbed, so a line with spaces becomes several elements and a * turns into file names.
From the lesson Lists done right: arrays, mapfile and read -a
-
What does read -r -a arr do with a line?
It splits that single line into fields at IFS, without globbing, one field per element: with the default IFS, New Delhi becomes two elements.
From the lesson Lists done right: arrays, mapfile and read -a
-
How do you join an array's elements with a comma?
Set IFS to a comma for the join alone, in a subshell or as a function's local IFS, and expand "${a[*]}", which joins with IFS's first character.
From the lesson Lists done right: arrays, mapfile and read -a
-
Which array features need a newer Bash than the 3.2 of a Mac?
mapfile needs 4.0, and mapfile -d and empty arrays allowed under set -u need 4.4. For Bash 3.2, use a while read loop and the ${a[@]+"${a[@]}"} form.
From the lesson Lists done right: arrays, mapfile and read -a
-
When does set -e end a script?
When a command returns a non-zero status that nothing handles, outside the spots where errexit looks away. It does not handle errors for you: it only stops the script.
From the lesson set -e, set -u and pipefail: what they really do
-
Where does set -e look away from a failure?
In an if, while or until condition and the functions it calls; in every command of a && or || list but the last; after !; in every pipeline stage but the last, unless pipefail is on; inside a command substitution, unless inherit_errexit is on; and in local or export with a substitution.
From the lesson set -e, set -u and pipefail: what they really do
-
Does set -e reach inside $(...)?
Not by default: Bash runs each command substitution without -e. shopt -s inherit_errexit, from Bash 4.4, passes it down.
From the lesson set -e, set -u and pipefail: what they really do
-
Why does local x=$(cmd) let a failing cmd slip past set -e?
The status that counts is local's own, which is 0. Declare the variable on one line and assign it on the next, as ShellCheck's SC2155 asks.
From the lesson set -e, set -u and pipefail: what they really do
-
What does set -u catch, and which expansions are exempt?
Expanding an unset variable, which also catches typos in names. "$@", "$*" and, since Bash 4.4, an empty array's "${a[@]}" are exempt, and ${v-default} opts out for a value that may be missing.
From the lesson set -e, set -u and pipefail: what they really do
-
Under pipefail, what does a status of 141 from a pipeline usually mean?
A reader closed its end of the pipe early, and the kernel sent SIGPIPE to the writer before it. Any other failed stage reports its own non-zero status.
From the lesson set -e, set -u and pipefail: what they really do
-
Is set -euo pipefail with an ERR trap enough error handling?
No. It is a safety net against unexpected failures; failures you expect still need their own if ! cmd, cmd || ... or an explicit status check.
From the lesson set -e, set -u and pipefail: what they really do
-
What does trap 'action' SIGNAL do?
It registers action to run when SIGNAL, or one of the shell events EXIT, ERR, RETURN and DEBUG, reaches the shell. The action is one shell word.
From the lesson Traps and cleanup: EXIT, ERR and signals
-
Why should a trap action be in single quotes?
So its variables expand when the trap runs, not when it is set; double quotes freeze the values of the moment, the slip ShellCheck names SC2064.
From the lesson Traps and cleanup: EXIT, ERR and signals
-
When does an EXIT trap run in Bash, and when can it not?
However the shell ends: the last line, exit, a set -e stop, or an INT or TERM. It never runs after SIGKILL, which no process can catch, and dash runs it on none of these signals.
From the lesson Traps and cleanup: EXIT, ERR and signals
-
Which status should a script exit with after trapping INT or TERM?
The handler does not exit on its own, so write exit 130 for INT and exit 143 for TERM, which is 128 plus the signal number, and the caller sees the right code.
From the lesson Traps and cleanup: EXIT, ERR and signals
-
How much work belongs in a signal handler?
Very little: set a flag and let the main loop act on it. Bash runs a trap only between commands, so a foreground sleep 60 delays the handler; start a long wait as sleep 60 & wait $!, because wait returns at once when a trapped signal arrives.
From the lesson Traps and cleanup: EXIT, ERR and signals
-
Why does an ERR trap not fire inside functions by default?
Functions do not inherit it unless set -E is on. RETURN and DEBUG traps need set -T for the same reason.
From the lesson Traps and cleanup: EXIT, ERR and signals
-
What happens to traps in a subshell, and how do you list them?
A subshell resets every trap except ignored signals, so a trap set inside ( ... ) fires when that subshell ends. trap -p lists the current traps in Bash and POSIX.1-2024, but dash 0.5.13.5 rejects -p.
From the lesson Traps and cleanup: EXIT, ERR and signals
-
Why do error messages go to standard error?
Standard output is the data the next command or file receives. With errors on standard error, script | jq still gives jq clean data while the error stays on the screen.
From the lesson Error-handling patterns for scripts
-
What do the die and warn helpers of a script do?
die writes error: and the message to standard error and exits with the status in rc, 1 by default, so rc=2 die "wrong arguments" marks a usage error. warn writes warning: and the message to standard error and returns without exiting.
From the lesson Error-handling patterns for scripts
-
Where should a script read a command's status?
Where the decision is made: if cmd, cmd || ..., ! cmd, or rc=0; cmd || rc=$? when the number matters. Testing $? after another command has run is ShellCheck's SC2181 slip.
From the lesson Error-handling patterns for scripts
-
Why does cmd; rc=$? never record a failure in a set -e script?
The failing cmd stops the script before rc=$? runs. Write rc=0; cmd || rc=$? so the failure is handled and its status kept.
From the lesson Error-handling patterns for scripts
-
What should a script check before its first side effect?
The argument count, the input and every command it needs, so a broken call stops before it leaves any partial state behind.
From the lesson Error-handling patterns for scripts
-
Which exit codes should a script choose for its own failures?
2 for a usage error, as Bash's builtins use it, and 3 and up for failures of its own, kept at 125 or below, since callers read 126, 127 and 128 plus N as could not run or killed by a signal.
From the lesson Error-handling patterns for scripts
-
How should a script handle failures in a batch of items?
Keep going, collect the failed items in an array, print a summary at the end and then exit non-zero, so the caller can retry only the failures.
From the lesson Error-handling patterns for scripts
-
What does set -x print, and how do you trace only part of a script?
Every simple command, after expansion, on standard error just before it runs, behind the PS4 prefix. set -x and set +x bracket a block; bash -x script traces the whole run.
From the lesson Debugging: bash -x, PS4, trap DEBUG and logs
-
How do you make each trace line name its file, line and function?
Set PS4='+ ${BASH_SOURCE##*/}:${LINENO}:${FUNCNAME[0]:-main}: ' in single quotes, so the expansions fire at trace time rather than when PS4 is set.
From the lesson Debugging: bash -x, PS4, trap DEBUG and logs
-
How do you send the trace to its own file instead of the screen?
Open a descriptor for the file with exec 3> trace.log and point the trace at it with BASH_XTRACEFD=3 before set -x; the user sees the normal output while the trace lands in trace.log.
From the lesson Debugging: bash -x, PS4, trap DEBUG and logs
-
What does bash -n find, and what is new about it in Bash 5.3?
Syntax errors, without running a single command. Bash 5.3 names the line of the opening construct, so an unexpected end of file points at the block that was never closed.
From the lesson Debugging: bash -x, PS4, trap DEBUG and logs
-
Which slips does ShellCheck catch that the parser misses?
Among others SC2086 for unquoted expansions, SC2155 for local with a substitution, SC2064 for a double-quoted trap, SC2164 for cd without || exit, and SC2181 for testing $? instead of the command.
From the lesson Debugging: bash -x, PS4, trap DEBUG and logs
-
What do you do when a long script fails and the trace is too long to read?
Shrink the bug: copy the failing block and the input that breaks it into scratch files, and trace that pair; a DEBUG trap can narrate each step when lint and trace are not enough.
From the lesson Debugging: bash -x, PS4, trap DEBUG and logs
-
In getopts ':vo:' opt, what do the two colons mean?
The colon after o means -o takes an argument, which getopts puts in OPTARG. The leading colon switches getopts to silent mode, so the script reports bad options itself.
-
How does a getopts loop read the options?
One option per call: each call puts the next option letter in the named variable. At -- or at the first word that does not start with a dash it returns 1, so the loop ends and a -v after an operand stays an operand.
-
Why does a getopts loop end with shift $((OPTIND - 1))?
OPTIND is the index of the next word getopts would read, and the positional parameters still hold the options. The shift removes them, so "$@" holds only the operands.
-
What does silent mode change about errors in getopts?
getopts prints no message itself: an unknown option sets the variable to ? and a missing argument sets it to :, with the offending letter in OPTARG, so the script writes its own usage error.
-
How does a script accept long options such as --output=file?
With a hand-written while loop over $1, a case on the word and shift, handling --name, --name=value and --name value; getopts reads short options only.
-
How do you pass a file named -v to a script that parses options?
Put -- before it, as in ./script -- -v: the double dash ends the options, so the next word is an operand.
-
Why do portable scripts avoid the external getopt command?
It comes in two incompatible versions, so a script that must run on both macOS and Linux keeps to the getopts builtin and a case-and-shift loop.
-
Which stream and status should --help use?
Standard output with exit 0, because the user asked for it: that is what makes tool --help | grep work.
-
Which stream and status should a usage error use?
Standard error with a non-zero status, 2 by convention for a tool that was called wrongly, so the message survives a pipe and stays out of the data.
-
When should a script print colour codes?
Only when [[ -t 1 ]] is true, standard output is a terminal, and NO_COLOR is unset or empty; never into files or pipes.
-
In which order should a tool read its settings?
Defaults first, then the config file, then environment variables, then flags, so each later source overrides the earlier ones.
-
Why should a script not source its config file?
Sourcing runs the file as code. Read key=value lines with read instead and accept only the keys the script expects.
-
Which three output controls make a tool easier to trust?
-v for extra information, -q to suppress informational messages, never errors, and --dry-run, which shows what a destructive tool would do without doing it.
-
What does -r change in read -r name?
It keeps backslashes as typed. Without it, read treats a backslash as an escape and removes it, so C:\new\report arrives mangled; IFS= read -r line also keeps leading and trailing spaces.
-
Where does read -p print its prompt, and when?
On standard error, and only when standard input is a terminal, so in printf 'x\n' | ./script the prompt is skipped and a piped run stays quiet.
-
How does a prompt give an empty answer a default?
Read the answer, then use ${name:-default}, which takes the default when the answer is empty.
-
What two values does each pass of a select loop give?
REPLY, the raw line typed, and the loop variable, the menu word that number names. A number out of range leaves the word empty, so the default branch reports a wrong answer; break ends the loop.
-
How should a script behave when nobody is at a terminal?
[[ -t 0 ]] is false in cron, CI and pipes. A script that may run unattended takes a --yes flag and reports a missing one on standard error instead of waiting for an answer.
-
What does read -t do when the time runs out, and where does it fit?
It returns a status above 128, so the script can fall back to a default. That suits a choice with a safe default, not a destructive action.
-
How do cut -f2 and cut -c2 differ?
cut -f2 takes the second delimited field, with a tab as the default delimiter and -d to set another. cut -c2 takes the second character of each line, which suits fixed-width data.
From the lesson cut, tr, head, tail, wc, paste and friends
-
Why does cut -d, give wrong columns on some CSV files?
cut sees only the delimiter character, not the quoting, so a comma inside a quoted field splits that field in two. Quoted CSV needs a parser that understands quotes.
From the lesson cut, tr, head, tail, wc, paste and friends
-
What does tr do, and where does it read its input?
It transforms or deletes single characters, and -s squeezes repeated characters into one. It reads only standard input, so a file goes in with < file.
From the lesson cut, tr, head, tail, wc, paste and friends
-
How do you print a file without its header line?
tail -n +2 file starts at line 2. head -n N and tail -n N take N lines from the start or the end.
From the lesson cut, tr, head, tail, wc, paste and friends
-
Why can wc -l report one line fewer than the file seems to have?
wc -l counts newline characters, so a last line without a trailing newline is not counted.
From the lesson cut, tr, head, tail, wc, paste and friends
-
How do paste and join differ?
paste joins files side by side line by line, with -d for the separator and -s to turn columns into rows. join merges two sorted files on a matching key field.
From the lesson cut, tr, head, tail, wc, paste and friends
-
Which sort options sort a file on its second column as numbers?
sort -t sep -k2,2n: -k2,2 closes the key at field 2, n compares numbers, and -t sets the separator when it is not whitespace. A plain sort compares whole lines as text.
From the lesson sort and uniq: counting and top-N reports
-
Why write -k2,2 rather than -k2?
-k2 runs from field 2 to the end of the line, so when field 2 ties, later fields quietly decide the order. -k2,2 sorts on field 2 alone.
From the lesson sort and uniq: counting and top-N reports
-
Why does uniq need a sort before it?
uniq compares each line only with the line before it, so it drops adjacent duplicates and nothing else. Sorting first puts every copy of a line next to the others.
From the lesson sort and uniq: counting and top-N reports
-
What do uniq -c, uniq -d and uniq -u show?
-c prefixes each group with its count, -d prints only lines that repeat, and -u only lines that appear once.
From the lesson sort and uniq: counting and top-N reports
-
What pipeline gives the N most frequent lines, the same way every time?
Count with sort file | uniq -c, then order the counts with sort -k1,1nr -k2,2 and keep the first N lines with head -n N. The second sort is numeric and reversed on the count and breaks ties by name.
From the lesson sort and uniq: counting and top-N reports
-
What do LC_ALL=C, sort -V and sort -h change?
LC_ALL=C makes sort compare bytes, which is predictable and fast. -V orders version numbers as a person would, v1.2 before v1.10, and -h reads sizes such as 512, 1K and 2M.
From the lesson sort and uniq: counting and top-N reports
-
How do grep, grep -E and grep -F read their pattern?
Plain grep reads a basic regular expression, -E an extended one, and -F plain text, so a dot or a bracket in the pattern matches itself.
From the lesson grep: searching text and files
-
What do grep -i, -v, -w, -n, -c and -o do?
-i ignores case, -v prints lines that do not match, -w matches whole words, -n adds line numbers, -c counts matching lines, and -o prints only the matched text.
From the lesson grep: searching text and files
-
How do you see the lines around each match?
-A N prints N lines after each match, -B N before and -C N both; runs of context that do not touch are separated by a -- line.
From the lesson grep: searching text and files
-
How do you search only the Python files of a repository and skip .git?
grep -r PATTERN --include='*.py' --exclude-dir='.git' . walks the tree, keeps files whose names match the glob and skips the whole folder.
From the lesson grep: searching text and files
-
Why must a script treat grep's status 2 differently from 1?
1 means no line matched; 2 means an error such as a missing file or permission. A test that only checks grep -q quietly takes the else branch on a wrong path, which hides a bug.
From the lesson grep: searching text and files
-
How do you give grep several patterns, or one that starts with a dash?
Each -e PATTERN adds one, and they act as OR; -f FILE reads one pattern per line; and -- ends the options, so -e or -- lets a pattern such as -v be searched for.
From the lesson grep: searching text and files
-
What separates a basic from an extended regular expression?
The quoting rule: ( ) and { are literal in BRE and need a backslash to become metacharacters, as + ? and | do in GNU and BSD tools, while in ERE they are metacharacters and need a backslash to be literal.
From the lesson Regular expressions for grep, sed and awk
-
Which tools read BRE and which read ERE by default?
grep and sed read BRE; grep -E, sed -E, awk and Bash's [[ string =~ regex ]] read ERE.
From the lesson Regular expressions for grep, sed and awk
-
What does a bracket expression match, and what are POSIX classes?
[set] matches one character of the set and [^set] one that is not. Classes such as [[:digit:]], [[:alpha:]] and [[:space:]] go inside brackets, work in both dialects and follow the locale.
From the lesson Regular expressions for grep, sed and awk
-
How do you anchor a match and count repetitions?
^ and $ pin a match to the start or end of the line; *, +, ? and {n,m} count repetitions, with the braces written \{n,m\} in BRE and {n,m} in ERE.
From the lesson Regular expressions for grep, sed and awk
-
What do round brackets and \1 do in a regular expression?
The brackets capture what a group matched, and \1 to \9 read it back, so \(a\)\1 matches aa in a BRE. POSIX defines back-references for BRE; GNU and BSD grep -E and sed -E also accept (a)\1 in an ERE.
From the lesson Regular expressions for grep, sed and awk
-
Which branch of (a|ab) matches the text ab in grep, sed and awk?
The longer one, ab: POSIX regular expressions pick the leftmost-longest match. Perl-style engines, as in Python and JavaScript, pick the first branch that works, here a.
From the lesson Regular expressions for grep, sed and awk
-
What do the flags of s/old/new/ change?
Without a flag only the first match on each line changes; g changes every match, a number only the Nth, i ignores case, and p with -n prints only the lines that changed.
From the lesson sed: substitute, delete and print
-
How does a sed replacement reuse what matched?
& writes the whole match back, and \1 to \9 write the text of the groups captured with \( \), or with ( ) under -E. A literal & or backslash needs its own backslash.
From the lesson sed: substitute, delete and print
-
How do you make a sed command act only on some lines?
Put an address before it: a line number, $ for the last line, /regex/, a range such as 2,5, or any of them followed by ! to mean every other line.
From the lesson sed: substitute, delete and print
-
How does sed act as a small grep or print a range of lines?
-n turns off the automatic printing, so sed -n '/re/p' prints only matching lines and sed -n '2,3p' prints lines 2 and 3. d deletes the line and starts the next cycle.
From the lesson sed: substitute, delete and print
-
Why is sed -i not portable between Linux and macOS?
GNU sed takes sed -i 's/x/y/' file, BSD sed needs sed -i '' 's/x/y/' file. A portable script writes to a temporary file and moves it over the original only when sed succeeded.
From the lesson sed: substitute, delete and print
-
What are sed's two buffers, and when does each clear?
The pattern space holds the current line and is cleared before each new line is read. The hold space starts empty and keeps its value until the script changes it, which lets a script remember earlier lines.
From the lesson sed beyond s///: hold space and multiline
-
What do h, H, g, G and x do?
h copies the pattern space to the hold space and H appends to it; g copies the hold space back and G appends it; x swaps the two.
From the lesson sed beyond s///: hold space and multiline
-
How do N, P and D let sed work across lines?
N appends the next line to the pattern space after a newline, P prints up to the first newline, and D deletes up to it and restarts the cycle.
From the lesson sed beyond s///: hold space and multiline
-
What do b and t do in a sed script?
b label jumps every time, and t label jumps only when a substitution succeeded since the last input line, so :loop, s/--/-/ and t loop on three lines repeat until no match is left. BSD sed needs each label on a line of its own.
From the lesson sed beyond s///: hold space and multiline
-
How do you append, insert or change lines portably with sed?
a, i and c add a line after, add one before, or replace the line; their one-line form is GNU-only, so portable scripts write the text after a backslash and a newline.
From the lesson sed beyond s///: hold space and multiline
-
How does awk run a program of pattern { action } pairs?
For each record it reads, it tests every pattern and runs the actions whose pattern matched, then goes on to the next record. BEGIN runs once before the input and END once after it.
From the lesson awk: fields, patterns and actions
-
What are $0, $1, $NF, NF and NR in awk?
$0 is the whole record and $1 the first field; NF is the number of fields, so $NF is the last one; NR counts records across all the input and FNR within the current file.
From the lesson awk: fields, patterns and actions
-
Which three ways set awk's field separator in time for the first record?
-F sep on the command line, -v FS=sep, or BEGIN { FS = sep } inside the program. All three take effect before the first record is read, so field 1 of line 1 is cut correctly.
From the lesson awk: fields, patterns and actions
-
Why does setting OFS not change what a plain print writes?
print writes $0 as it was read. OFS goes between fields in print $1, $2, and an assignment such as $1 = $1 rebuilds $0 with the current OFS.
From the lesson awk: fields, patterns and actions
-
Which kinds of pattern can choose the records of an awk action?
BEGIN and END, a /regex/, any expression, a range start, end of two patterns, and any of them negated with !; patterns combine with && and || as in C.
From the lesson awk: fields, patterns and actions
-
Why can "10" < "9" be true in awk?
A string constant on one side forces a byte-wise compare of text, where 1 sorts before 9. Fields that look like numbers compare as numbers, and adding +0 settles the ambiguity.
From the lesson awk: fields, patterns and actions
-
How does awk group lines by key in one pass?
Its arrays are associative: count[$2]++ counts lines per value of field 2, sum[$2] += $3 adds up field 3 per key, and an END block prints the totals.
From the lesson awk programs: arrays, joins and reports
-
How do you make an awk join print only the lines whose key was found?
Test membership with (k in a): FNR==NR { a[$1]; next } ($2 in a) { print } prints a line of the second file only when its key was loaded, while a bare lookup gives an empty value for a missing key.
From the lesson awk programs: arrays, joins and reports
-
In which order does for (k in array) visit the keys?
POSIX leaves it unspecified, and gawk and mawk differ, so pipe the output to sort or use gawk's PROCINFO["sorted_in"] for a fixed order.
From the lesson awk programs: arrays, joins and reports
-
How does the FNR==NR idiom join two files in awk?
While FNR equals NR, awk is still reading the first file, so FNR==NR { name[$1] = $2; next } loads it into an array; the second file then looks each key up. Missing keys give empty strings, and a repeated key overwrites.
From the lesson awk programs: arrays, joins and reports
-
What do print > "file", print >> "file" and close() do in awk?
> empties the file once per run and then keeps writing to it, >> always appends, and print | "cmd" feeds a command. close() flushes and closes it, which must come before reading a file the program wrote.
From the lesson awk programs: arrays, joins and reports
-
How are longer awk programs and their functions written?
In a file run with awk -f prog.awk; a function's local variables are declared as extra parameters after its real ones.
From the lesson awk programs: arrays, joins and reports
-
How does find decide what to print?
It evaluates its expression on every path of the tree: tests pick paths, operators combine them, and the implicit action -print applies only when the expression has no action of its own.
From the lesson find and xargs: working with many files
-
How do you make find skip node_modules entirely?
Put the pruned path on the left of -o and a trailing -print on the right: find . -path ./node_modules -prune -o -type f -name '*.js' -print never enters the folder.
From the lesson find and xargs: working with many files
-
When do you use -exec cmd {} + and when -exec cmd {} \;?
{} + passes many paths to each run of the command, so far fewer processes start; {} \; runs the command once per path, for a command that takes one file at a time.
From the lesson find and xargs: working with many files
-
How should a clean-up with find -delete be run?
Twice: first with -print in place of -delete, read the list, then run the same line with -delete placed last. -delete switches on -depth, which makes -prune do nothing.
From the lesson find and xargs: working with many files
-
How do you pass file names with spaces or newlines from find to another command?
Separate them with NUL bytes: find ... -print0 | xargs -0 cmd, or a while IFS= read -r -d '' loop. A plain find | xargs splits names at blanks.
From the lesson find and xargs: working with many files
-
What do xargs -n, -I {}, -P and -r do?
-n sets how many items go to each command, -I {} puts each item where {} stands, -P N runs up to N commands at once, and -r stops GNU xargs from running the command once on empty input.
From the lesson find and xargs: working with many files
-
How long may one command line be?
Up to the kernel's ARG_MAX bytes, which getconf ARG_MAX prints; find -exec {} + and xargs batch the paths so each command stays under it.
From the lesson find and xargs: working with many files
-
How do . and .[] differ in jq?
. emits its input unchanged, so jq '.' pretty-prints a file. .[] emits each element of an array, or each value of an object, as its own value, so the next stage sees one at a time.
From the lesson jq: query and reshape JSON from the shell
-
What do | and , do between jq filters?
| passes each value from the left filter into the right one; , runs two filters on the same input and joins their outputs.
From the lesson jq: query and reshape JSON from the shell
-
What do jq's -r, -c, -s and -n flags change?
-r prints strings without their quotes, for the shell; -c prints each result on one line; -s reads every input value into one array; and -n runs the program once with null as input, so jq -n '{a:1}' builds JSON without reading anything.
From the lesson jq: query and reshape JSON from the shell
-
What do select, map and group_by do?
select keeps the values for which a test is true, map transforms every element of an array, and group_by, like sort_by and unique_by, returns results sorted by the key.
From the lesson jq: query and reshape JSON from the shell
-
How should a shell value reach a jq filter?
Through --arg name value, which gives a string, or --argjson name JSON, which gives a parsed value. Pasting the variable into the filter text breaks silently when the value contains jq syntax.
From the lesson jq: query and reshape JSON from the shell
-
What does jq -e change about the exit status?
The status then describes the last output: 0 for a value that is neither null nor false, 1 for null or false, and 4 when nothing came out, which makes if jq -e '.field' file a presence test.
From the lesson jq: query and reshape JSON from the shell
-
Which jq format strings make output safe for the shell?
@csv, @tsv, @sh and @uri format values as CSV, TSV, shell-quoted words and URL parts; with -r they come out as plain lines a loop can read.
From the lesson jq: query and reshape JSON from the shell
-
Why do cut -d, and awk -F, break on real CSV files?
CSV has no single standard: RFC 4180 defines quoting, and a quoted field may hold commas, quotes or newlines that a plain split on commas cuts apart.
From the lesson CSV, TSV and YAML from the shell
-
What does gawk --csv do?
It parses CSV the way RFC 4180 quotes it, from gawk 5.3: a comma inside quotes stays in its field, a doubled quote becomes one, and a newline inside a quoted field does not end the record.
From the lesson CSV, TSV and YAML from the shell
-
Why does IFS=$'\t' read -r a b c lose an empty TSV field?
A tab is a whitespace character of IFS, so consecutive tabs count as one separator and the fields after the empty one shift left. awk -F'\t' keeps empty fields as empty strings.
From the lesson CSV, TSV and YAML from the shell
-
Why should a script read YAML with yq rather than grep and sed?
YAML is a tree with anchors, multi-line values and indentation rules; line tools see only lines, so they give wrong results or break the file silently.
From the lesson CSV, TSV and YAML from the shell
-
How should a script read a .env or INI file it did not write?
Never source it, because sourcing runs the file as code. Read each line with IFS='=' read -r key value instead, which only splits the text.
From the lesson CSV, TSV and YAML from the shell
-
Which two shapes store a timestamp without ambiguity?
UTC epoch seconds, one integer, and an ISO 8601 string in UTC ending in Z. A local time without a zone leaves the reader guessing which zone was meant.
From the lesson Dates, times and non-integer numbers
-
How does GNU date convert between epoch seconds and an ISO 8601 UTC string?
date -u -d @N +'%FT%TZ' turns an epoch into the string, and date -u -d STR +'%s' reads the string back as an epoch. BSD date on macOS does the same with -r and -ju -f.
From the lesson Dates, times and non-integer numbers
-
How can Bash print the time without starting date?
printf '%(fmt)T' formats a time inside Bash. EPOCHSECONDS and EPOCHREALTIME give the wall clock, and Bash 5.3's BASH_MONOSECONDS a monotonic clock that NTP steps and DST changes do not move.
From the lesson Dates, times and non-integer numbers
-
Why is a local wall-clock time not always one instant?
In a zone with daylight saving time, one local hour is missing in spring and one is repeated in autumn, so a local reading can mean two UTC instants or none. Store UTC and convert at the edge.
From the lesson Dates, times and non-integer numbers
-
How does jq convert between ISO 8601 strings and epoch numbers?
fromdateiso8601 turns a UTC string into an epoch and todate turns it back; strftime formats an epoch for a person and leaves the stored field alone.
From the lesson Dates, times and non-integer numbers
-
How should a shell script add up rupee amounts exactly?
Store them as integer paise, add in integers, and divide by 100 only when a person reads the total. Other decimals go through awk or bc, because Bash arithmetic is integer only.
From the lesson Dates, times and non-integer numbers
-
How do ( list ) and { list; } differ?
( list ) runs in a subshell, so every variable, cd and option change is lost when it ends. { list; } runs in the current shell and keeps every change.
From the lesson Subshells and the execution environment
-
Why does { echo hi } fail as a command group?
The closing brace is a reserved word that Bash recognises only where a command could start, so it needs a semicolon or a newline before it: { echo hi; }.
From the lesson Subshells and the execution environment
-
Which constructs run their commands in a subshell without any parentheses?
Each stage of a pipeline, except the last one when lastpipe is on and job control is off; a command substitution; a process substitution; a background job started with &; and a coprocess.
From the lesson Subshells and the execution environment
-
Which variables does a subshell see, and which does a new program such as bash -c see?
A subshell starts as a copy of the parent and sees every variable, exported or not. A new program receives only the exported ones.
From the lesson Subshells and the execution environment
-
What does exit do inside ( ... )?
It ends only the subshell. The parent carries on and finds the subshell's exit status in $?.
From the lesson Subshells and the execution environment
-
How do $$, BASHPID and BASH_SUBSHELL tell processes apart?
$$ is the main shell's process ID, the same inside subshells; BASHPID is the ID of the process running now; and BASH_SUBSHELL counts how deeply subshells are nested.
From the lesson Subshells and the execution environment
-
What does the status of cmd & tell you?
Only that the job started: & returns 0 at once. The job's own result comes later, from wait.
From the lesson Background jobs, wait and parallel work
-
What does $! hold, and when must you read it?
The process ID of the most recent background job. Read it straight after the & that started the job you want to track, because the next background job overwrites it.
From the lesson Background jobs, wait and parallel work
-
What do wait PID and a bare wait return?
wait PID waits for that job and returns its own exit status. A bare wait waits for every job and returns 0, so it reports nothing about what failed.
From the lesson Background jobs, wait and parallel work
-
How can a script tell which of several background jobs failed?
Keep each job's PID and call wait on them one at a time; naming several PIDs in one wait returns only the last one's status, and a background job's failure never trips set -e.
From the lesson Background jobs, wait and parallel work
-
How does a script run many jobs but only a few at a time?
With a job pool: start jobs until the cap is reached, then wait -n to reuse a slot as soon as any job ends, and wait -n -p var, from Bash 5.1, to learn which one ended. wait -n needs Bash 4.3, so a stock macOS Bash falls back to batches of plain wait.
From the lesson Background jobs, wait and parallel work
-
What do xargs -P N and nohup add for background work?
xargs -P N is a one-line pool that runs up to N commands at once, with one combined status. nohup cmd > log 2>&1 & keeps a job alive after you log out by ignoring SIGHUP.
From the lesson Background jobs, wait and parallel work
-
What does Ctrl-Z do to the foreground job?
The terminal sends SIGTSTP to the foreground process group, which pauses the job and gives the prompt back. The job is stopped, not finished, and stays in the shell's job table.
From the lesson Job control: Ctrl-Z, fg, bg, jobs, disown
-
What do bg, fg and jobs do?
bg resumes a stopped job in the background, fg resumes it in the foreground, and jobs lists the job table with each job's number, state and command, + marking the current job and - the previous one.
From the lesson Job control: Ctrl-Z, fg, bg, jobs, disown
-
How do you name a job in fg, bg or kill?
With a job spec starting with %: %1 by number, %+ and %- by marker, %name by the start of its command and %?text by text anywhere in it. Bash refuses a prefix that matches two jobs.
From the lesson Job control: Ctrl-Z, fg, bg, jobs, disown
-
Why do fg and bg fail inside a script?
Job control is on in interactive shells and off in scripts, so they answer no job control. set -m turns it on, and then each job gets its own process group.
From the lesson Job control: Ctrl-Z, fg, bg, jobs, disown
-
Why does job control give each job its own process group?
So the terminal can send keys such as Ctrl-C and Ctrl-Z to the foreground job alone, while background jobs keep running untouched.
From the lesson Job control: Ctrl-Z, fg, bg, jobs, disown
-
How can a job survive the hang-up of its terminal?
At a hang-up an interactive shell resends SIGHUP to every job in its table. disown removes a job from the table, disown -h keeps it listed but marked so it gets no SIGHUP, and nohup starts a command that ignores SIGHUP.
From the lesson Job control: Ctrl-Z, fg, bg, jobs, disown
-
How do TERM and KILL differ?
TERM asks: a process can catch it, ignore it or answer with its own cleanup. KILL forces: the kernel handles it, like STOP, and the program never sees it.
From the lesson Signals, kill and timeout
-
Why should a script name signals instead of numbering them?
Several common signal numbers differ between macOS and Linux, while names such as TERM, INT and HUP mean the same on both.
From the lesson Signals, kill and timeout
-
What do kill PID, kill -0 PID and kill -- -PGID do?
kill sends TERM unless told otherwise. kill -0 sends nothing and only tests that the process exists and may be signalled. A minus sign before the number reaches every process of that process group.
From the lesson Signals, kill and timeout
-
Why can killing a job's first process leave its helpers running?
The signal reaches only that process. Its children go on unless the whole group is signalled with kill -TERM -- -PGID, which a script has per job only with set -m, or unless the leader forwards the signal.
From the lesson Signals, kill and timeout
-
What do the statuses 130, 137 and 143 mean?
Each is 128 plus the number of the signal that killed the command: 130 is INT, as from Ctrl-C; 137 is KILL, which cannot be caught; 143 is TERM. kill -l 143 names the signal.
From the lesson Signals, kill and timeout
-
Which statuses does timeout end with?
124 when the time limit was reached and the command obeyed TERM, 137 when KILL was needed, the command's own status when it finished in time, and 125 to 127 for timeout's own failures.
From the lesson Signals, kill and timeout
-
How does a launcher script make sure its program receives TERM?
Either it traps TERM, forwards it to the child and waits twice, since the first wait returns 128 plus the signal at once, or it replaces itself with exec so the program is the process that gets the signal.
From the lesson Signals, kill and timeout
-
What is a FIFO made with mkfifo?
A file-system entry of type p, a named pipe: two processes open the same path, and the bytes pass through the kernel without being stored on disk.
From the lesson Named pipes and coprocesses
-
What happens when a process opens a FIFO?
The open blocks until the other end is opened too: a writer waits for a reader, and a reader for a writer.
From the lesson Named pipes and coprocesses
-
When does the reader of a FIFO see end of file?
When no process has it open for writing any more. A long-lived reader keeps it open with exec 3<>fifo, so one producer closing it does not end the loop.
From the lesson Named pipes and coprocesses
-
Where should a script create its FIFO?
Inside a folder made with mktemp -d, removed by an EXIT trap, because the special file stays until something removes it.
From the lesson Named pipes and coprocesses
-
What does coproc NAME { command; } give the shell?
command runs asynchronously with two pipes to the shell: write to ${NAME[1]}, read from ${NAME[0]}, and wait on $NAME_PID. The Bash 3.2.57 of macOS and dash have no coproc.
From the lesson Named pipes and coprocesses
-
Why must the write end of a coprocess running tr be closed before the first read?
A block-buffered filter flushes only at end of input, so the shell closes ${NAME[1]} with exec {NAME[1]}>&- and only then reads the answer.
From the lesson Named pipes and coprocesses
-
How do you choose between a pipe, a temporary file, a FIFO and a coprocess?
Pick the lightest shape that fits: a plain pipe for one-way flow, a FIFO for several writers and one reader, and a coprocess or two FIFOs for a live back-and-forth; sh scripts keep to pipes, FIFOs and temporary files.
From the lesson Named pipes and coprocesses
-
Which startup files does a Bash login shell read?
/etc/profile first; then it looks for ~/.bash_profile, ~/.bash_login and ~/.profile, in that order, and reads only the first one it finds.
From the lesson Startup files, PATH and login shells
-
Which startup file does an interactive non-login shell read, and which does a script read?
An interactive shell that is not a login shell reads ~/.bashrc. A shell running a script reads neither, unless BASH_ENV names a file for it to read.
From the lesson Startup files, PATH and login shells
-
How do login and non-login interactive shells share one set of settings?
A one-line ~/.bash_profile that sources ~/.bashrc, so the settings live in one file that both kinds of shell end up reading.
From the lesson Startup files, PATH and login shells
-
How does Bash search PATH, and how should a folder be added to it?
From left to right, and the first match wins. Prepend with PATH="$dir:$PATH" so the system folders stay; type -a NAME shows every match in order.
From the lesson Startup files, PATH and login shells
-
Why does a command that works in your terminal fail in cron or CI?
Cron and CI run the job in a non-interactive shell, and such a shell skips your startup files, so your PATH and variables are missing. Set them inside the job, or call programs by their absolute path.
From the lesson Startup files, PATH and login shells
-
How do you find and rerun an older command at the prompt?
Press Ctrl-R and type part of it: Bash shows the most recent matching command as you type, and Enter runs it.
From the lesson History, Ctrl-R and readline editing
-
What do !!, !$ and !n expand to?
!! is the whole previous command, so sudo !! repeats it with sudo; !$ is its last word, as in mkdir reports && cd !$; and !n is command number n of the history list.
From the lesson History, Ctrl-R and readline editing
-
Where does history expansion with ! happen?
Only in interactive shells, before the line runs. An ! in a script is harmless, but at the prompt an ! inside double quotes needs care.
From the lesson History, Ctrl-R and readline editing
-
What do HISTCONTROL=ignoreboth and shopt -s histappend change?
ignoreboth keeps out of the history a command that repeats the one before it and every command that starts with a space, and histappend makes each terminal append to the history file instead of overwriting the others.
From the lesson History, Ctrl-R and readline editing
-
Which keys does readline use to edit the command line, and where is it configured?
Emacs keys by default, or vi keys after set -o vi. ~/.inputrc configures readline for Bash and every other program that uses the library.
From the lesson History, Ctrl-R and readline editing
-
Which escapes build a PS1 prompt?
\u for the user, \h for the host, \w for the current folder and \$, which shows # for root and $ for everyone else.
From the lesson Prompt, aliases, dirstack, completion
-
Why do colour codes in PS1 go inside \[ and \]?
The brackets mark them as non-printing, so Bash counts the prompt's width correctly and long lines wrap in the right place.
From the lesson Prompt, aliases, dirstack, completion
-
What does PROMPT_COMMAND do?
It runs code before each prompt is shown, which can display things such as the last exit status or the current git branch.
From the lesson Prompt, aliases, dirstack, completion
-
When should a shortcut be a function rather than an alias?
When it needs an argument, any logic, or must work in a script: an alias is plain text substitution at the start of an interactive command, and scripts do not expand aliases unless expand_aliases is set.
From the lesson Prompt, aliases, dirstack, completion
-
How do you move between folders and come back?
pushd changes folder and saves the old one on a stack, popd returns to it, and dirs lists the stack; cd - goes back to the previous folder without any stack.
From the lesson Prompt, aliases, dirstack, completion
-
How do you give a command its own Tab completion?
compgen -W 'list of words' filters words by the prefix typed, and complete attaches a word list, or a function that fills COMPREPLY, to the command.
From the lesson Prompt, aliases, dirstack, completion
-
Which program runs a script whose first line is #!/bin/sh?
Whatever shell the system puts at /bin/sh: dash on Ubuntu, Bash 3.2.57 started as sh on the Mac of the lesson's session. sh names a role, so such a script may rely only on POSIX sh.
From the lesson POSIX sh vs Bash: spotting and fixing bashisms
-
What is a bashism?
A Bash feature that the POSIX shell language does not require, such as arrays, [[ ]], {1..3}, source, ${s:1:3} and <<<. It matters only in a script that claims to be plain sh.
From the lesson POSIX sh vs Bash: spotting and fixing bashisms
-
Why is a zero exit status not enough to show that a sh script ran correctly under dash?
Of the twelve Bash features the lesson tried, five failed in dash with an error and status 2, and seven ended with status 0 and a wrong result, so the output must be checked too.
From the lesson POSIX sh vs Bash: spotting and fixing bashisms
-
What are the POSIX forms of [[ ]], source, &> and echo for data?
case or [ ] instead of [[ ]], . instead of source, cmd > file 2>&1 instead of &>, and printf '%s\n' instead of echo for data.
From the lesson POSIX sh vs Bash: spotting and fixing bashisms
-
Where does dash behave differently from Bash on code that is valid in both?
dash's echo reads backslash escapes, its set -e reaches into $( ), and its $(( )) does not look through a variable that holds another variable's name.
From the lesson POSIX sh vs Bash: spotting and fixing bashisms
-
How do you check that a script really is POSIX sh?
Run shellcheck -s sh on it and run it with dash. bash --posix keeps every Bash extra, so a pass there proves nothing about dash.
From the lesson POSIX sh vs Bash: spotting and fixing bashisms
-
What is POSIX.1-2024, and how does it relate to the previous edition?
Issue 8 of The Open Group Base Specifications. It mostly adds to Issue 7, POSIX.1-2017, and its few removals, such as test's -a and -o, had been marked obsolescent first.
From the lesson What POSIX.1-2024 changed for shell scripts
-
Which four shell-language changes of POSIX.1-2024 do scripts meet first?
$'...' quoting with backslash escapes, the pipefail option of set, a Job Control section, and trap -p, which prints the current trap actions.
From the lesson What POSIX.1-2024 changed for shell scripts
-
Which utility changes of POSIX.1-2024 reach shell scripts?
read -d, the test operators -nt, -ot and -ef, numbered printf conversions such as %1$s, and new utilities, among them realpath, readlink and timeout.
From the lesson What POSIX.1-2024 changed for shell scripts
-
How many of the seven probed Issue 8 features did bash 5.3.20 and dash 0.5.13.5 run?
bash ran six and refused %1$s in its printf builtin; dash ran four and refused trap -p, read -d and %1$s.
From the lesson What POSIX.1-2024 changed for shell scripts
-
Does a feature's place in POSIX.1-2024 make it safe in a #!/bin/sh script?
Not yet: Debian policy still lets sh scripts assume only POSIX.1-2017 plus named extras, and ShellCheck still flags $'...' in sh, so check that the running shell has the feature.
From the lesson What POSIX.1-2024 changed for shell scripts
-
How does a script test for an option such as pipefail before it relies on it?
Probe it in a subshell, as in (set -o pipefail) 2>/dev/null, turn it on only when the probe passes, and give the other branch a clear message or a flag.
From the lesson What POSIX.1-2024 changed for shell scripts
-
Why can the same command name behave differently on Linux and macOS?
macOS ships utilities descended from BSD, while most Linux systems ship GNU coreutils or tools that take its options. They agree on most of POSIX and differ on a short list of flags and defaults.
From the lesson GNU vs BSD tools: Linux and macOS differences
-
Which GNU-versus-BSD differences are hardest to catch?
A different default rather than a missing flag. A Linux-only spelling such as grep -P, find -printf or date -d fails with an error on a Mac, but xargs on empty input gives a different result with no error, so only a test of the result finds it.
From the lesson GNU vs BSD tools: Linux and macOS differences
-
What is the shortest sed in-place edit that GNU and BSD sed read the same way?
sed -i.bak 's/x/y/' file, with the backup suffix attached to -i. A wrapper that writes a mktemp file and moves it over the original avoids -i altogether.
From the lesson GNU vs BSD tools: Linux and macOS differences
-
How does a script choose between GNU and BSD spellings?
uname -s is the usual first filter, and a one-off feature probe is the reliable second step, because a Mac may have the GNU tools first on PATH.
From the lesson GNU vs BSD tools: Linux and macOS differences
-
How does Homebrew add the GNU tools to a Mac?
Its coreutils, gnu-sed, grep, gawk and findutils formulae install them with a g prefix, such as gsed; keeping /usr/bin ahead of their gnubin folders keeps the stock tools as the default.
From the lesson GNU vs BSD tools: Linux and macOS differences
-
Which GNU-only features did POSIX.1-2024 make standard?
realpath, xargs -r, test -nt and set -o pipefail, so some portability wrappers can go once every target system has a 2024-standard shell and tools.
From the lesson GNU vs BSD tools: Linux and macOS differences
-
In which Bash versions did associative arrays, wait -n, ${v@Q}, EPOCHSECONDS, SRANDOM and ${ cmd; } arrive?
Associative arrays in 4.0, wait -n in 4.3, ${v@Q} in 4.4, EPOCHSECONDS in 5.0, SRANDOM in 5.1 and ${ cmd; } in 5.3.
From the lesson Bash 3.2 to 5.3: versions and feature checks
-
Which Bash can a script expect on macOS and on current Linux releases?
macOS still ships Bash 3.2.57 at /bin/bash, while the Linux releases in the lesson's table package Bash 5.2 or 5.3, so a script for both pins or probes the version it needs.
From the lesson Bash 3.2 to 5.3: versions and feature checks
-
How does a script refuse to run on a Bash older than 4?
Its first top-level command is (( BASH_VERSINFO[0] >= 4 )) || { echo "needs bash 4+" >&2; exit 1; }, placed before any 4.x syntax.
From the lesson Bash 3.2 to 5.3: versions and feature checks
-
Why must the version guard sit at the top level and not inside a function?
Bash parses one top-level command at a time, but a function's whole body when the function is defined, so 4.x syntax in that body fails at parse time before the guard can run.
From the lesson Bash 3.2 to 5.3: versions and feature checks
-
When is a feature probe better than a version check?
When backports or vendor builds make the version number misleading: the probe runs the feature in a child shell and compares its output with a known string.
From the lesson Bash 3.2 to 5.3: versions and feature checks
-
What are BASH_COMPAT and the compatNN shell options for?
They keep a specific older behaviour in a newer Bash, which keeps legacy scripts running; new code should never be written against them.
From the lesson Bash 3.2 to 5.3: versions and feature checks
-
What is the root bug behind most shell injection?
An unquoted expansion: $x is split at IFS and globbed, while "$x" stays one word. Quote every expansion and put -- before operands, as ShellCheck's SC2086 advises.
From the lesson Injection, eval and untrusted input
-
Why is eval "echo $input" a bug when input comes from a user?
eval reads its argument again as shell code, so a semicolon or && in the value starts a second command and $( ) in it runs a substitution, even though the expansion was quoted.
From the lesson Injection, eval and untrusted input
-
What replaces eval when a script builds a command from user input?
A command array run as "${cmd[@]}" with a case allowlist for the words it may hold, or, when a command string is needed, sh -c with a fixed script and the value passed as a separate argument.
From the lesson Injection, eval and untrusted input
-
Why is [[ $x -eq 0 ]] unsafe on a value from outside the script?
Every arithmetic context, including (( )), $(( )), let and [[ -eq ]], evaluates array subscripts inside the value, and a command substitution inside a subscript runs. The lesson's session showed it on Bash 5.3.20.
From the lesson Injection, eval and untrusted input
-
How should a script check and compare a number it received?
Validate the shape first with [[ $x =~ ^[0-9]+$ ]], then compare with [ "$x" -eq 0 ], which does not evaluate subscripts.
From the lesson Injection, eval and untrusted input
-
Why can a file named like an option turn rm *.log into a different command?
Most utilities take any argument beginning with - for an option, and a glob hands them file names as ordinary arguments. rm -- *.log and rm ./*.log are the two fixes ShellCheck's SC2035 gives.
From the lesson Injection, eval and untrusted input
-
How should a script validate input before trusting it?
Decide the exact shape first, accept only that shape, and turn everything else away with an error message and an exit status other than 0: a case with a short literal list for one of a few words, an anchored regex for a pattern of characters.
From the lesson Injection, eval and untrusted input
-
Which five lines open a script that writes to disk?
A shebang that names Bash, a PATH the script sets itself, umask 077, a working folder from mktemp -d, and a single-quoted trap on EXIT that removes it.
From the lesson Secure scripting: temp files, secrets and PATH
-
Why pair mktemp -d with an EXIT trap instead of using /tmp/myscript.$$?
A guessable name leaves a gap between checking it and using it, in which another local user can place a symlink of their own. mktemp -d creates an unpredictable folder in one step, and the trap removes it however the script ends, except after SIGKILL.
From the lesson Secure scripting: temp files, secrets and PATH
-
Where should a script receive a secret?
Not on the command line, which other users can list. Standard input, a mode-600 file or a one-command environment variable keep it off argv; read -rs keeps a typed token out of the history, and set -x stays off around every line that uses it.
From the lesson Secure scripting: temp files, secrets and PATH
-
How should a CI workflow handle a secret value?
Keep the value in the secret store and let the workflow refer to it by name, mask values derived from it with ::add-mask::, and never type a value into the workflow file.
From the lesson Secure scripting: temp files, secrets and PATH
-
Why set umask 077 before a sensitive file is created rather than chmod it afterwards?
The mask gives the file mode 600 from the moment it exists. A later chmod leaves a window in which the file was readable, and a process that opened it then keeps its access; check the result with stat.
From the lesson Secure scripting: temp files, secrets and PATH
-
Which inherited settings should a script reset on its first lines?
It sets its own PATH and unsets CDPATH and BASH_ENV, so the caller's environment cannot change which programs run or which folder cd picks.
From the lesson Secure scripting: temp files, secrets and PATH
-
How should a script run something it downloads?
Never pipe the download into a shell. Download it to a file, check it with sha256sum -c against a sum taken from a second channel, and only then run it.
From the lesson Secure scripting: temp files, secrets and PATH
-
What does ShellCheck do with a script, and what do its exit statuses mean?
It reads the script without running it and reports each likely bug with a code, the file, the line and a severity. It exits 0 when clean, 1 when it finds something and 2 when a file cannot be checked, which makes it a CI gate.
From the lesson ShellCheck: catch bugs before a script runs
-
What does SC2086 mean, and how do you clear it?
An expansion is not quoted, so its value will be split into words and globbed: echo $file sends two arguments when file holds my report.txt. Quote it, "$file", and the finding goes away because the bug does.
From the lesson ShellCheck: catch bugs before a script runs
-
Why fix the cause of a ShellCheck finding instead of muting it?
Each finding is a claim about the code, such as an expansion that can split or a trap that expands too early. Fixing the cause makes the claim untrue; muting it leaves the risk in place.
From the lesson ShellCheck: catch bugs before a script runs
-
How does ShellCheck decide which shell language to check?
From the shebang, or from -s, which forces a dialect. Checking as sh flags Bash-only features, mostly with SC3xxx codes.
From the lesson ShellCheck: catch bugs before a script runs
-
Why does ShellCheck 0.11.0 still warn on $'...' in a sh script?
Its sh checks follow the 2017 edition of POSIX there and report SC3003, although POSIX.1-2024 added the form.
From the lesson ShellCheck: catch bugs before a script runs
-
How do you tell ShellCheck about a sourced file or a false positive?
-x lets it follow sourced files, and a source= directive names a file it cannot work out. For a false positive, a commented disable directive covers one command, and a .shellcheckrc holds project-wide choices.
From the lesson ShellCheck: catch bugs before a script runs
-
What is a Bats test file, and what does it print?
Bash with @test "name" { ... } blocks. bats --formatter tap FILE prints a plan line and one ok or not ok line per test, in the Test Anything Protocol that CI systems read.
From the lesson Testing Bash scripts with Bats
-
What do $status, $output and ${lines[@]} hold after run CMD?
The command's exit status, its combined standard output and standard error, and that output split into lines with empty ones left out. run --separate-stderr adds $stderr.
From the lesson Testing Bash scripts with Bats
-
Why call a command through run inside a @test instead of directly?
Bats runs each test with set -e, so a command that fails directly ends the test at once. run captures the status and returns 0, so the test can assert on the status it expected.
From the lesson Testing Bash scripts with Bats
-
How does Bats keep tests from affecting each other?
Every @test runs in a fresh process with a new, empty $BATS_TEST_TMPDIR; setup and teardown run around each test, and setup_file and teardown_file once per file.
From the lesson Testing Bash scripts with Bats
-
What must a script have before a Bats file can source it to test its functions?
A main guard, so main does not run when the file is sourced under a different $0; to test the whole command, run the script itself.
From the lesson Testing Bash scripts with Bats
-
How do you test a script that calls curl without touching the network?
Put a fake curl first on PATH, in $BATS_TEST_TMPDIR/bin, so the script calls the stub. Then prove the suite with a mutant that must turn at least one test to not ok.
From the lesson Testing Bash scripts with Bats
-
Which naming convention does the style guide give for a shell script?
lower_case for functions and local variables, UPPER_CASE for constants and exported variables, and readonly on constants so an accidental assignment fails.
From the lesson Style and maintainable scripts
-
Why does the style guide prefer $(...) to backticks and [[ ]] to [ ]?
$(...) nests without escaping inner backquotes. [[ ]] handles && and || inside the test, does not split or glob an unquoted variable, and matches patterns with == and =~.
From the lesson Style and maintainable scripts
-
What belongs in the comment above a function?
Its Globals, Arguments, Outputs and Returns, and comments in general explain why the code does something, not what it does.
From the lesson Style and maintainable scripts
-
Where do functions and main go in a maintainable script?
Functions are grouped near the top, with a usage block first, and a main function at the bottom calls them.
From the lesson Style and maintainable scripts
-
How do you format a script with shfmt and check it in CI?
shfmt -i 2 -ci -w rewrites the file in place, and CI runs the same flags with -d, shfmt -i 2 -ci -d, or the style lives in an .editorconfig; a bare shfmt -d checks against tabs instead.
From the lesson Style and maintainable scripts
-
What should a script reaching about 100 lines prompt you to ask?
Whether it is doing more than one job, whether a structured language would make the logic clearer, and whether the team could still debug it.
From the lesson Style and maintainable scripts
-
Why does calling an external command inside a loop cost so much?
Each call forks a process and execs the program, and a $(...) capture forks too. A builtin or a parameter expansion does the same job inside the shell with no new process.
From the lesson Fast Bash: builtins, forks and measuring
-
What replaces $(basename "$f") and $(dirname "$f") in a tight loop?
${f##*/} and ${f%/*}. On the Mac the lesson measured, 500 basename calls took 3.99 seconds and 500 expansions under 2 milliseconds.
From the lesson Fast Bash: builtins, forks and measuring
-
How should a script's speed be measured before it is changed?
Time the block with time { ...; } and TIMEFORMAT='%6R', repeat it several times and compare the medians, because one noisy run can hide the real cost; hyperfine does this for whole commands.
From the lesson Fast Bash: builtins, forks and measuring
-
How can a loop keep its variables and capture output without a subshell?
Feed the loop with < <(cmd) or load the lines with mapfile instead of a pipe, and capture output with printf -v or Bash 5.3's ${ cmd; }.
From the lesson Fast Bash: builtins, forks and measuring
-
Why hand a whole file to awk instead of running grep once per line?
A per-line external multiplies the fork cost by the number of lines: on the lesson's Mac, a grep per line over 1,000 lines took 10 seconds and one awk pass under a hundredth of a second.
From the lesson Fast Bash: builtins, forks and measuring
-
When does LC_ALL=C make sort and grep faster?
When the input is ASCII or the order does not need the locale's collation rules: comparing bytes made sort about three times as fast on ASCII words in the lesson.
From the lesson Fast Bash: builtins, forks and measuring
-
What do the five time fields of a crontab line mean?
From the left: the minute, the hour, the day of the month, the month and the day of the week, where 1 to 5 is Monday to Friday and both 0 and 7 are Sunday. The command follows them, and the daemon compares every line with the clock once a minute.
From the lesson cron: scheduling scripts that actually run
-
When does the crontab line */15 9-17 * * 1-5 run?
Every 15 minutes, at :00, :15, :30 and :45, during the hours 9 to 17 inclusive, on Monday to Friday.
From the lesson cron: scheduling scripts that actually run
-
Why does */40 in the minute field run at :00 and :40 rather than every 40 minutes?
Lists, ranges and steps count inside their own field, so the step starts again every hour. Steps, names and the @ shortcuts are cronie's extensions, not POSIX's.
From the lesson cron: scheduling scripts that actually run
-
When does a line with both a day of the month and a day of the week run?
When both fields are restricted, on a day that matches either one. When one of them starts with *, */2 included, cronie runs it only on days that match both.
From the lesson cron: scheduling scripts that actually run
-
What does an unescaped % do in a crontab command?
cron cuts the command at the first unescaped % and sends the rest to the command's standard input, so write \% or move the command into a script.
From the lesson cron: scheduling scripts that actually run
-
Which rules does the crontab file itself have?
Settings such as SHELL, PATH, MAILTO and cronie's CRON_TZ apply to the jobs below them, comments go on lines of their own, and the file ends with a newline.
From the lesson cron: scheduling scripts that actually run
-
What does a wrapper on a crontab line give every run?
The same PATH, umask and folder, a lock that turns a second copy away, a log with UTC times, silence on success and one line on trouble, and a status that says what happened.
From the lesson cron: scheduling scripts that actually run
-
Which shell setup do GitHub Actions, Docker, make and Git hooks give your code?
None of your own: each starts a shell with its own command line and reads none of your startup files, and a Git hook runs under its own #! line.
From the lesson Shell in CI, Docker, Makefiles and Git hooks
-
When does a GitHub Actions step run with pipefail?
Only with shell: bash, or defaults.run.shell: bash for the whole workflow, which runs bash --noprofile --norc -eo pipefail. A step without a shell key runs as bash -e, without pipefail, and every step is a new shell.
From the lesson Shell in CI, Docker, Makefiles and Git hooks
-
Which shell runs a shell-form RUN line in a Dockerfile?
/bin/sh -c with no options, which is dash in Debian-based images, so a pipeline passes when its last command does. SHELL ["/bin/bash", "-o", "pipefail", "-c"] gives the later lines pipefail.
From the lesson Shell in CI, Docker, Makefiles and Git hooks
-
What should an entrypoint script do before the application starts?
Check its settings, refuse to run without a command, write the files it needs, and end with exec "$@", so the app keeps the script's process ID, gets CMD as its arguments and receives signals directly.
From the lesson Shell in CI, Docker, Makefiles and Git hooks
-
Why does a cd on one line of a make recipe not reach the next line?
make runs each recipe line in its own shell, which exits with the line. Join the commands on one line, or use .ONESHELL, and write the shell's own $ as $$.
From the lesson Shell in CI, Docker, Makefiles and Git hooks
-
What does .ONESHELL change in GNU make?
Each recipe runs in one shell, so a cd carries over, but only the last line's status reaches make until .SHELLFLAGS adds -e. It needs GNU make 3.82 or later.
From the lesson Shell in CI, Docker, Makefiles and Git hooks
-
What should a pre-commit hook check, and how does it stop a commit?
The staged files: names from git diff --cached --name-only -z, read whole, and the staged copies through :path. It collects every problem, then exits 1 once; git commit --no-verify skips it.
From the lesson Shell in CI, Docker, Makefiles and Git hooks
-
What makes a script's log lines easy to filter?
One line per event with an ISO 8601 time in UTC, a level and the message, written to standard error, so grep and awk can filter by field and the data stream stays clean.
From the lesson Logging, locking and idempotent scripts
-
How does a script send its messages to a file or to the system log?
exec 2>>file sends all of its error output to a log file, and on Linux logger -t tag -p user.level hands each line to syslog or the systemd journal.
From the lesson Logging, locking and idempotent scripts
-
Why does mkdir make a good lock?
It tests and creates in one step, so exactly one of two racing copies succeeds. A PID written inside the folder and checked with kill -0 tells a live holder from a run that SIGKILL ended.
From the lesson Logging, locking and idempotent scripts
-
How does flock avoid stale locks?
Its lock belongs to an open file descriptor, which the kernel closes when the process ends however it ends, so nothing is left to remove after a crash. It comes with util-linux on Linux; macOS has no flock command.
From the lesson Logging, locking and idempotent scripts
-
When is a script step idempotent, and how do you prove a script is?
When running it again leaves the same state as one run did, as mkdir -p, ln -sfn, grep -qxF before an append and cmp -s before a copy do. Run the script twice and compare a listing of the state.
From the lesson Logging, locking and idempotent scripts
-
How do you replace a file so readers never see half of it?
Write the new content to a temporary file in the target's own folder, then rename it over the old one. The rename is a single step, so anyone reading the file meets the earlier content or the finished replacement, never a half-written mix.
From the lesson Logging, locking and idempotent scripts
-
How should a tool that deletes or overwrites present its changes?
Route every change through one function that prints the command, quoted with printf %q, and runs it only when asked, with printing as the default.
From the lesson Logging, locking and idempotent scripts
-
In tar -czf site.tgz site, which program packs and which compresses?
tar packs: it walks the folder and writes one stream holding every file with its path, mode, owner, size and time. gzip compresses that single stream and knows nothing of folders.
From the lesson Archives, checksums and backups
-
Why archive a folder with tar -C folder . rather than by its full path?
The members then have paths relative to that folder, starting with ./, so they restore into any folder you name with -C. Every --exclude goes before the names.
From the lesson Archives, checksums and backups
-
What does gzip store in its header, and which command unpacks a .gz on every system?
A file name and a time, unless -n leaves them out. gzip -dc reads a .gz everywhere, while the zcat of macOS looks for a .Z file.
From the lesson Archives, checksums and backups
-
Why can two archives of the same unchanged files have different bytes?
The gzip time, the owners and the extended attributes go into them, so compare the files' own checksums to see whether anything changed, and fix those details when the bytes must match.
From the lesson Archives, checksums and backups
-
How do you check a whole folder of files in one command?
Write a SHA-256 manifest with find -print0, sort -z and xargs -0 sha256sum, then run sha256sum -c on it. It ends with status 1 for a changed or missing file, and scripts test that status, because the result lines differ between tools.
From the lesson Archives, checksums and backups
-
When does a backup count as a backup?
Once it has been unpacked into an empty folder and every file has passed the manifest check. gzip -t and tar find damage, but only that check finds an archive that is valid but incomplete.
From the lesson Archives, checksums and backups
-
How should a script rotate dated backups, and why is a mirror not a backup?
Zero-padded dated names let a glob list them oldest first, so rotation keeps a count and a missed night never costs an old backup. A mirror made with rsync -a --delete is a copy of today, not a history.
From the lesson Archives, checksums and backups
-
How does ssh pass the words after the host to the remote side?
It joins them into one string, and the remote login shell parses it again. Quote each value with printf %q, or send the script on standard input with bash -s, whose arguments still need quoting.
From the lesson Remote work: ssh, rsync and curl in scripts
-
Which ssh options make a script fail fast instead of hanging?
BatchMode=yes, a ConnectTimeout and server-alive messages, so a missing key or a dead host fails in seconds. The first value set in ssh_config wins.
From the lesson Remote work: ssh, rsync and curl in scripts
-
After ssh host command returns, how does a script tell which side failed?
Status 255 means ssh itself failed, for example no connection or a refused key. Any other status is the remote command's own.
From the lesson Remote work: ssh, rsync and curl in scripts
-
How do scp, sftp and rsync differ for copying files over SSH?
scp copies whole files and uses SFTP since OpenSSH 9.0; sftp -b runs a checked batch and can resume; rsync compares first and sends only what changed, which needs rsync on both machines.
From the lesson Remote work: ssh, rsync and curl in scripts
-
How does rsync decide which files to send, and how much of each?
Its quick check skips files whose size and time match, which -c replaces with full checksums at the cost of reading everything. The delta then sends only the changed parts: in the lesson, 700 of 260,000 bytes for one changed byte.
From the lesson Remote work: ssh, rsync and curl in scripts
-
Why does curl exit 0 on a 404, and how does a script make it fail?
curl counts any HTTP answer as a success unless told otherwise: -f makes an HTTP error exit 22 and -L follows redirects. Exit 28 means the transfer timed out and 7 a refused connection.
From the lesson Remote work: ssh, rsync and curl in scripts
-
How should a script pass an API token to curl and check the JSON answer?
Keep the token off the command line with -H @file, and check the answer with jq -e under pipefail, so a wrong or empty answer fails the step.
From the lesson Remote work: ssh, rsync and curl in scripts
-
How can Bash test a TCP port without any other tool?
Redirect to /dev/tcp/host/port, which Bash opens as a connection, under timeout. dash and POSIX sh have no /dev/tcp.
From the lesson Remote work: ssh, rsync and curl in scripts
-
What kind of work is Bash best at?
Running other programs: commands in order with a stop at the first failure, moving and packing files, short pipelines, CI steps and small wrappers.
From the lesson When to use Bash, and when to switch to Python
-
What did writing the same log summary in Bash and in Python show?
The Bash version took 13 lines and the Python one 48, 14 of them checks, but only Python stopped at the bad line instead of quietly averaging in a 0.
From the lesson When to use Bash, and when to switch to Python
-
Which signs say a script has outgrown Bash?
Nested data, decimals, many error cases, lasting state, Windows users, more than about 100 lines, and speed. One sign can often be handled in place; two or more call for a program.
From the lesson When to use Bash, and when to switch to Python
-
Which four questions decide whether a task belongs in Bash?
What the task mostly does, whether Bash may be missing where it runs, whether it will grow, and whether Windows users will run it.
From the lesson When to use Bash, and when to switch to Python
-
How should a Python program call command-line tools after a script is rewritten?
Pass subprocess.run a list of arguments and read each tool's status yourself. One string with shell=True let a file name run a command in the lesson.
From the lesson When to use Bash, and when to switch to Python
-
Can a script for other people be written in zsh, fish or PowerShell syntax?
They are other languages, not other Bashes, so scripts meant for others are written in Bash or POSIX sh.
From the lesson When to use Bash, and when to switch to Python
-
What does a thin Bash wrapper around a program do?
It checks the tools it needs and ends with exec program "$@", so the command, its arguments and its exit status stay exactly as the program gives them.
From the lesson When to use Bash, and when to switch to Python
-
What makes an interview answer about shell behaviour strong?
It restates the question, proves the behaviour with a two-line test, names the rule and its source, names the trap and gives the fix.
From the lesson Shell scripting interview questions, explained
-
Why does a script hand its arguments on with "$@" and not "$*"?
"$@" keeps each argument a separate word, so an argument a b stays one: printf '[%s]\n' "$@" printed [a b] and [c]. "$*" glues every argument together into a single word, with IFS's first character as the glue.
From the lesson Shell scripting interview questions, explained
-
Why does a counter set in a loop at the end of a pipeline vanish?
The loop runs in a subshell. Feed it with done < <(cmd) instead, or set lastpipe in a script, where job control is off.
From the lesson Shell scripting interview questions, explained
-
Does set -e stop a function that fails when the function was called from the left of ||?
No. set -e does nothing inside a function called from the left of || or &&, however many of its commands fail.
From the lesson Shell scripting interview questions, explained
-
Which one-liners print the tenth line of a file and drop repeated lines in first-seen order?
sed -n '10{p;q;}' prints line 10 and stops reading, and awk '!seen[$0]++' prints each line the first time it appears.
From the lesson Shell scripting interview questions, explained
-
Does an EXIT trap run when an untrapped TERM ends the script?
In Bash 5.3 it does; in dash it does not.
From the lesson Shell scripting interview questions, explained
-
How do you debug a broken pipeline in an interview question?
Find the failing stage before guessing: check the stages one at a time, with a PS4 that adds the line number to each trace line.
From the lesson Shell scripting interview questions, explained