Brace and tilde expansion

mkdir -p proj/{src,test}/{main,util} creates seven folders in one command, and cp config.ini{,.bak} makes a backup without typing the name twice. Both are brace expansion, the first expansion Bash performs, which is also why {1..$n} never works.

You will generate lists, ranges, zero-padded and stepped sequences and cross products, replace variable ranges with loops that work, and use the tilde forms only where Bash really expands them.

Braces turn one word into a list, before anything else is expanded

Brace expansionbrace expansion: The first expansion Bash performs: a comma list {a,b} or a range {1..9..2} in a word becomes several words, with the text around the braces copied into each. turns a word with a comma list, {a,b,c}, or a range, {x..y} or {x..y..step}, into several words. The text before and after the braces is copied into each one, and the words keep their left-to-right order: nothing is sorted, and nothing has to exist as a file. The session tries the common forms:

from session import run

# Lists, sequences, padding, steps, a cross product and a backup name, all by brace expansion; then why a variable
# cannot set a range, and dash, which has no brace expansion. Braces make words whether or not files exist, unlike a
# glob, and a space inside the braces stops the expansion.
run("""
echo {a,b,c}
echo {1..5} {5..1}
echo {01..10}
echo {a..e} {1..10..3}
echo {x,y}{1,2}
echo config{,.bak}
n=4
echo {1..$n}
dash -c 'echo {1..3}'
touch a.txt
!ls {a,b}.txt
ls [ab].txt
echo proj/{build, dist}
""")

$ echo {a,b,c}
a b c
$ echo {1..5} {5..1}
1 2 3 4 5 5 4 3 2 1
$ echo {01..10}
01 02 03 04 05 06 07 08 09 10
$ echo {a..e} {1..10..3}
a b c d e 1 4 7 10
$ echo {x,y}{1,2}
x1 x2 y1 y2
$ echo config{,.bak}
config config.bak
$ n=4
$ echo {1..$n}
{1..4}
$ dash -c 'echo {1..3}'
{1..3}
$ touch a.txt
$ ls {a,b}.txt
ls: b.txt: No such file or directory
a.txt
[exit status 1]
$ ls [ab].txt
a.txt
$ echo proj/{build, dist}
proj/{build, dist}

{01..10} padded every number to the width of the widest end, {5..1} counted down, {1..10..3} stepped by 3 and {x,y}{1,2} gave the four combinations. config{,.bak} has an empty first item, so it gave config and config.bak. The manual allows only two kinds of range end, whole numbers or single letters, so {a..e} counts letters while {a..ee} is left alone, and a leading zero on either number forces every term to the same width. Then the two lines that do not expand: {1..$n}, because the braces are expanded while $n is still unexpanded, and dash, which has no brace expansion at all and printed {1..3} unchanged.

The last three commands show how braces differ from globs. ls {a,b}.txt asked for both names although only a.txt exists, so ls complained about b.txt; the glob [ab].txt matched only the file that is there. Braces generate words from what you wrote, globs select from what is on disk. And a space inside the braces breaks them: proj/{build, dist} is two words, proj/{build, and dist}, neither of them a valid brace expression, so both stayed as they were.

Braces are fine for loops with literal ends, for i in {1..5} or for p in {0..100..25}, but they build every word in memory before the command starts. {1..100000} creates 100,000 words first; for a large range, an arithmetic loop that counts as it goes is lighter.

Nested braces build folder trees and backup names

Braces may hold braces, and two lists in one word multiply. That makes one mkdir -p build a whole project skeleton. Step through how Bash expands the word before mkdir ever sees it:

The session runs that line and the backup trick, and compares {01..12} with the same names made by printf in a loop:

from session import run, write

# A project skeleton from one mkdir with nested braces, a backup copy with {,.bak}, zero-padded names compared with
# printf, and ShellCheck's warning for braces in a script meant for sh (SC3009).
write("make.sh", "#!/bin/sh\nmkdir -p out/{img,css}\n")
run("""
mkdir -p proj/{src,test}/{main,util}
find proj -type d | sort
touch config.ini
cp config.ini{,.bak}
ls config.ini*
echo week-{01..12}
for i in $(seq 1 12); do printf 'week-%02d ' "$i"; done; echo
!shellcheck make.sh
""")

$ mkdir -p proj/{src,test}/{main,util}
$ find proj -type d | sort
proj
proj/src
proj/src/main
proj/src/util
proj/test
proj/test/main
proj/test/util
$ touch config.ini
$ cp config.ini{,.bak}
$ ls config.ini*
config.ini
config.ini.bak
$ echo week-{01..12}
week-01 week-02 week-03 week-04 week-05 week-06 week-07 week-08 week-09 week-10 week-11 week-12
$ for i in $(seq 1 12); do printf 'week-%02d ' "$i"; done; echo
week-01 week-02 week-03 week-04 week-05 week-06 week-07 week-08 week-09 week-10 week-11 week-12
$ shellcheck make.sh

In make.sh line 2:
mkdir -p out/{img,css}
             ^-------^ SC3009 (warning): In POSIX sh, brace expansion is undefined.

For more information:
  https://www.shellcheck.net/wiki/SC3009 -- In POSIX sh, brace expansion is u...
[exit status 1]

find listed the 7 folders, cp config.ini{,.bak} left both files, and the two ways of making week-01 to week-12 gave the same line. The printf loop is the one to use when the end of the range is in a variable. The last command shows the warning for a script meant for sh: ShellCheck reports braces there as SC3009, because brace expansion is a Bash extension that POSIX sh does not have.

Before a brace word goes to a command that deletes or moves files, preview it. echo rm -r proj/{build,dist} prints exactly the words rm would receive, and costs nothing; a missing comma or a stray space then shows on screen instead of in the file system.

Brace expansion works in the arguments of any command, not only mkdir and cp: touch log-{a,b,c}.txt creates three files, and rm -i draft-{1,2}.txt asks before deleting each of the two. The same {,…} idea gives a small set of everyday commands. mv notes.{txt,md} renames a file by changing its extension, diff config.ini{.bak,} compares the backup with the current file, and cp -p config.ini{,.bak} keeps the permissions and times of the original in the copy. Each is one word that Bash expands into two before the command runs, so each can be previewed with echo first.

A tilde expands only at the start of a word, and never in quotes

Tilde expansiontilde expansion: The replacement of ~ by the home folder, ~name by that user's home, and ~+ and ~- by the current and previous folders, at the start of a word or after = or : in an assignment, and never inside quotes. replaces ~ with the home folder and ~name with the home folder of the user called name. Two more forms come from the shell’s folder history: ~+ is the current folder and ~- the previous one. The session sets HOME to /home/asha, so every expansion is visible:

from session import run

# The tilde forms: ~ is $HOME, ~+ the current folder and ~- the previous one. A tilde expands only at the start of a
# word or after = and : in an assignment, never inside quotes. The session sets HOME to /home/asha so the
# expansions are visible.
run("""
HOME=/home/asha
echo ~ ~/notes
echo ~root ~nosuchuser
echo "~/notes" x~y
paths=~/bin:~/tools; echo "$paths"
quoted="~/bin:$HOME/tools"; echo "$quoted"
mkdir first second
cd first; cd ../second
echo ~+ ~-
""")

$ HOME=/home/asha
$ echo ~ ~/notes
/home/asha /home/asha/notes
$ echo ~root ~nosuchuser
/var/root ~nosuchuser
$ echo "~/notes" x~y
~/notes x~y
$ paths=~/bin:~/tools; echo "$paths"
/home/asha/bin:/home/asha/tools
$ quoted="~/bin:$HOME/tools"; echo "$quoted"
~/bin:/home/asha/tools
$ mkdir first second
$ cd first; cd ../second
$ echo ~+ ~-
~/demo/second ~/demo/first

~ and ~/notes expanded at the start of their words. ~root gave the home folder of the user root, /var/root on this Mac (/root on most Linux systems), while ~nosuchuser stayed as written because no such user exists. "~/notes" in double quotes and x~y in the middle of a word stayed as written too. In an assignment, a tilde right after = or after a : expands, which is why paths=~/bin:~/tools gave two full paths. After cd first and cd ../second, ~+ and ~- gave the two folders.

The quoted assignment shows a mistake that is common in shell start-up files. In quoted="~/bin:$HOME/tools" the double quotes keep the tilde as text while $HOME still expands, so the value starts with a literal ~/bin. A PATH written that way, export PATH="~/bin:$PATH", adds a folder called ~ that does not exist, and programs in your bin folder are never found. Write it as PATH="$HOME/bin:$PATH", or leave the tilde unquoted.

Two smaller rules complete the picture. The manual counts everything from the tilde up to the first unquoted slash as the tilde-prefix, so in ~asha/notes the name is asha and the rest is an ordinary path. And ~- holds the same folder as the variable OLDPWD, the one cd - returns to, so cp report.pdf ~- copies a file back to the folder you just left. Keep in mind that the shell does the expanding, not the program: a path that reaches a program from a config file, or inside quotes, still begins with a plain tilde character, and the program sees exactly that. In scripts many people write $HOME instead of ~ for that reason: it works the same inside and outside double quotes, and a reader never has to remember where a tilde expands.

Practise: lists, ranges and home folders

The questions ask what brace words and tildes become, and which line builds a given list. Every one can be checked against the sessions above:

Interview questions

Warm-up (fresher to 2–5 years): why does echo {1..$n} not print a range in Bash?

Brace expansion is the first expansion Bash performs, before parameter expansion. When the braces are expanded, the word still contains the text $n, so {1..$n} is not a valid range and is left as it is; afterwards $n is expanded, and the result is the literal text {1..3} when n is 3. To loop over a range whose end is in a variable, use an arithmetic loop, for ((i = 1; i <= n; i++)), or seq 1 "$n". Brace ranges are for literal numbers, such as {01..12}, and they are a Bash feature: a POSIX sh such as dash prints them unchanged.

Two harder questions follow, about building many folders safely and about a script whose paths break. Choose your answer before you open each first step:

Key takeaways

• Brace expansion turns {a,b} lists and {x..y..step} ranges into words, in order, copying the text around them, and it is the first expansion Bash performs.
• {1..$n} stays literal because the braces are expanded before the variable, so ranges from variables need an arithmetic loop or seq.
• Nested and repeated braces build folder trees and backup names in one command, and ShellCheck flags them in sh scripts as SC3009.
• A tilde expands only at the start of a word or after = and : in an assignment, never inside quotes, and ~+ and ~- give the current and previous folders.