How-to: Shell Expansion

Expansion is performed on the command line after it has been split into tokens.

The order of expansions is: brace expansion, tilde expansion, parameter, variable, and arithmetic expansion and command substitution (done in a left-to-right fashion), word splitting, and filename expansion. Quote removal is performed after all the other expansions.

Brace Expansion { }

Brace expansion is a mechanism by which arbitrary strings can be generated. This mechanism is similar to filename expansion (see section Filename Expansion), but the file names generated need not exist. Patterns to be brace expanded take the form of an optional preamble, followed by a series of comma-separated strings between a pair of braces, followed by an optional postscript. The preamble is prefixed to each string contained within the braces, and the postscript is then appended to each resulting string, expanding left to right.

Brace expansions can be nested. The results of each expanded string are not sorted; left to right order is preserved. For example,

bash$ echo a{d,c,b}e
ade ace abe

A sequence expression takes the form:
{x..y[..incr]}

where x and y are either integers or single characters, and incr, an optional increment, is an integer.

When integers are supplied, the expression expands to each number between x and y, inclusive. Supplied integers may be prefixed with ‘0’ to force each term to have the same width. When either x or y begins with a zero, the shell attempts to force all generated terms to contain the same number of digits, zero-padding where necessary.

When characters are supplied, the expression expands to each character lexicographically between x and y, inclusive, using the default C locale. Note that both x and y must be of the same type. When the increment is supplied, it is used as the difference between each term. The default increment is 1 or -1 as appropriate.

Brace expansion is performed before any other expansions, and any characters special to other expansions are preserved in the result. It is strictly textual. Bash does not apply any syntactic interpretation to the context of the expansion or the text between the braces. To avoid conflicts with parameter expansion, the string ${ is not considered eligible for brace expansion.

A correctly-formed brace expansion must contain unquoted opening and closing braces, and at least one unquoted comma. Any incorrectly formed brace expansion is left unchanged.

This construct is often useful when the strings involved have a long common prefix, such as a long file path.

Examples

$ mv app/ss64/alpha.py app/ss64/alphabeta.py
 can be
$ mv app/ss64/{alpha,alphabeta}.py
 or even
$ mv app/ss64/alpha{,beta}.py

$ mkdir /usr/local/src/bash/{old,new,dist,bugs}

$ chown root /usr/{ucb/{ex,edit},lib/{ex?.?*,how_ex}}

Tilde Expansion ~

If a word begins with an unquoted tilde character ('~'), all of the characters up to the first unquoted slash (or all characters, if there is no unquoted slash) are considered a tilde-prefix. If none of the characters in the tilde-prefix are quoted, the characters in the tilde-prefix following the tilde are treated as a possible login name. If this login name is the null string, the tilde is replaced with the value of the HOME shell variable. If HOME is unset, the home directory of the user executing the shell is substituted instead. Otherwise, the tilde-prefix is replaced with the home directory associated with the specified login name.

If the tilde-prefix is '~+', the value of the shell variable PWD replaces the tilde-prefix. If the tilde-prefix is '~-', the value of the shell variable OLDPWD, if it is set, is substituted.

If the characters following the tilde in the tilde-prefix consist of a number N, optionally prefixed by a '+' or a '-', the tilde-prefix is replaced with the corresponding element from the directory stack, as it would be displayed by the dirs builtin invoked with the characters following tilde in the tilde-prefix as an argument. If the tilde-prefix, sans the tilde, consists of a number without a leading '+' or '-', '+' is assumed.

If the login name is invalid, or the tilde expansion fails, the word is left unchanged.

Each variable assignment is checked for unquoted tilde-prefixes immediately following a ':' or '='. In these cases, tilde expansion is also performed. Consequently, one can use file names with tildes in assignments to PATH, MAILPATH, and CDPATH, and the shell assigns the expanded value.

The following table shows how Bash treats unquoted tilde-prefixes:

~
The value of $HOME

~/foo
$HOME/foo

~fred/foo
The subdirectory foo of the home directory of the user fred

~+/foo
$PWD/foo

~-/foo
${OLDPWD-'~-'}/foo ~N

~+N
The string that would be displayed by dirs +N

~-N
The string that would be displayed by dirs -N

Shell Parameter Expansion ${ }

The $ character introduces parameter expansion, command substitution, or arithmetic expansion. The parameter name or symbol to be expanded can be enclosed in braces, which are optional but serve to protect the variable to be expanded from characters immediately following it which could be interpreted as part of the name.

When braces are used, the matching ending brace is the first } not escaped by a backslash or within a quoted string, and not within an embedded arithmetic expansion, command substitution, or parameter expansion.

The basic form of parameter expansion is:
${parameter}
The value of parameter is substituted. The braces are required when parameter is a positional parameter with more than one digit, or when parameter is followed by a character that is not to be interpreted as part of its name.

If the first character of parameter is an exclamation point,a level of variable indirection is introduced. Bash uses the value of the variable formed from the rest of parameter as the name of the variable ${!variablename}This variable is then expanded and that value is used in the rest of the substitution, rather than the value of parameter itself. This is known as indirect expansion. The exception to this is the expansion of ${!prefix*} described below.

In each of the cases below, word is subject to tilde expansion, parameter expansion, command substitution, and arithmetic expansion.

When not performing substring expansion, Bash tests for a parameter that is unset or null; omitting the colon results in a test only for a parameter that is unset. Put another way, if the colon is included, the operator tests for both existence and that the value is not null; if the colon is omitted, the operator tests only for existence.

${parameter:-word}
If parameter is unset or null, the expansion of word is substituted. Otherwise, the value of parameter is substituted.

Example Make the first parameter passed to this script ('$1') default to the value 500:
#!/bin/bash
FIRST_ARG="${1:-500}"

${parameter:=word}
If parameter is unset or null, the expansion of word is assigned to parameter. The value of parameter is then substituted. Positional parameters and special parameters can not be assigned to in this way.

${parameter:?word}
If parameter is null or unset, the expansion of word (or a message to that effect if word is not present) is written to the standard error and the shell, if it is not interactive, exits. Otherwise, the value of parameter is substituted.

${parameter:+word}
If parameter is null or unset, nothing is substituted, otherwise the expansion of word is substituted.

${parameter:offset}
${parameter:offset:length}

Expands to up to length characters of parameter starting at the character specified by offset. If length is omitted, expands to the substring of parameter starting at the character specified by offset. length and offset are arithmetic expressions.
This is referred to as Substring Expansion.

${!prefix*}
Expands to the names of variables whose names begin with prefix, separated by the first character of the IFS special variable. ${#parameter}

The length in characters of the expanded value of parameter is substituted. If parameter is '*' or '@', the value substituted is the number of positional parameters. If parameter is an array name subscripted by '*' or '@', the value substituted is the number of elements in the array.

${parameter#word}
${parameter##word}
The word is expanded to produce a pattern just as in filename expansion (see section Filename Expansion). If the pattern matches the beginning of the expanded value of parameter, then the result of the expansion is the expanded value of parameter with the shortest matching pattern (the '#' case) or the longest matching pattern (the '##' case) deleted. If parameter is '@' or '*', the pattern removal operation is applied to each positional parameter in turn, and the expansion is the resultant list. If parameter is an array variable subscripted with '@' or '*', the pattern removal operation is applied to each member of the array in turn, and the expansion is the resultant list.

${parameter%word}
${parameter%%word}
The word is expanded to produce a pattern just as in filename expansion. If the pattern matches a trailing portion of the expanded value of parameter, then the result of the expansion is the value of parameter with the shortest matching pattern (the '%' case) or the longest matching pattern (the '%%' case) deleted. If parameter is '@' or '*', the pattern removal operation is applied to each positional parameter in turn, and the expansion is the resultant list. If parameter is an array variable subscripted with '@' or '*', the pattern removal operation is applied to each member of the array in turn, and the expansion is the resultant list.

${parameter/pattern/string}
${parameter//pattern/string}
The pattern is expanded to produce a pattern just as in filename expansion. Parameter is expanded and the longest match of pattern against its value is replaced with string. In the first form, only the first match is replaced. The second form causes all matches of pattern to be replaced with string.

Command Substitution $( )

Command substitution allows the output of a command to replace the command itself. Command substitution occurs when a command is enclosed as follows:

$(command)
or
'command'

The main difference is that multiple nested commands are easier to read when using brackets.

Bash performs the expansion by executing command and replacing the command substitution with the standard output of the command, with any trailing newlines deleted. Embedded newlines are not deleted, but they can be removed during word splitting. The command substitution $(cat file) can be replaced by the equivalent but faster $(< file).

When the old-style backquote form of substitution is used, backslash retains its literal meaning except when followed by $, ' , or \ . The first backquote not preceded by a backslash terminates the command substitution. When using the $(command) form, all characters between the parentheses make up the command; none are treated specially.

Command substitutions can be nested. To nest when using the backquoted form, escape the inner backquotes with backslashes.

If the substitution appears within double quotes, word splitting and filename expansion are not performed on the results.

Word Splitting

The shell scans the results of parameter expansion, command substitution, and arithmetic expansion that did not occur within double quotes for word splitting.

The shell treats each character of $IFS as a delimiter, and splits the results of the other expansions into words on these characters. If IFS is unset, or its value is exactly <space><tab><newline>, the default, then any sequence of IFS characters serves to delimit words. If IFS has a value other than the default, then sequences of the whitespace characters space and tab are ignored at the beginning and end of the word, as long as the whitespace character is in the value of IFS (an IFS whitespace character). Any character in IFS that is not IFS whitespace, along with any adjacent IFS whitespace characters, delimits a field. A sequence of IFS whitespace characters is also treated as a delimiter. If the value of IFS is null, no word splitting occurs.

Explicit null arguments ("" or ") are retained. Unquoted implicit null arguments, resulting from the expansion of parameters that have no values, are removed. If a parameter with no value is expanded within double quotes, a null argument results and is retained.

Note that if no expansion occurs, no splitting is performed.

Filename Expansion

After word splitting, unless the -f option has been set (see the Set Builtin), Bash scans each word for the characters *, ?, and [.
If one of these characters appears, then the word is regarded as a pattern, and replaced with an alphabetically sorted list of file names matching the pattern.

The presence of shell expansion characters in a word causes the shell to perform a transformation on the word. The transformation may replace the word with more than one word. In the following grep example, the *.txt will be replaced by multiple words if there is more than one file with a .txt suffix in the working directory:

grep snark *.txt

If no matching file names are found, and the shell option nullglob is disabled, the word is left unchanged. If the nullglob option is set, and no matches are found, the word is removed. If the shell option nocaseglob is enabled, the match is performed without regard to the case of alphabetic characters.

When a pattern is used for filename generation, the character '.' at the start of a filename or immediately following a slash must be matched explicitly, unless the shell option dotglob is set. When matching a file name, the slash character must always be matched explicitly. In other cases, the . character is not treated specially.

See the description of shopt, for a description of the nocaseglob, nullglob, and dotglob options.

The GLOBIGNORE shell variable can be used to restrict the set of filenames matching a pattern. If GLOBIGNORE is set, each matching filename that also matches one of the patterns in GLOBIGNORE is removed from the list of matches. The filenames . and .. are always ignored, even when GLOBIGNORE is set. However, setting GLOBIGNORE has the effect of enabling the dotglob shell option, so all other filenames beginning with a . will match. To get the old behavior of ignoring filenames beginning with a . , make .* one of the patterns in GLOBIGNORE. The dotglob option is disabled when GLOBIGNORE is unset.

Pattern Matching

See the Wildcards page for a description of this.

Locale-Specific Translation

A double-quoted string preceded by a dollar sign (‘$’) will cause the string to be translated according to the current locale. The gettext infrastructure performs the message catalog lookup and translation, using the LC_MESSAGES and TEXTDOMAIN shell variables, as explained below. See the gettext documentation for additional details. If the current locale is C or POSIX, or if there are no translations available, the dollar sign is ignored. If the string is translated and replaced, the replacement is double-quoted.

Some systems use the message catalog selected by the LC_MESSAGES shell variable. Others create the name of the message catalog from the value of the TEXTDOMAIN shell variable, possibly adding a suffix of ‘.mo’. If you use the TEXTDOMAIN variable, you may need to set the TEXTDOMAINDIR variable to the location of the message catalog files. Still others use both variables in this fashion: TEXTDOMAINDIR/LC_MESSAGES/LC_MESSAGES/TEXTDOMAIN.mo.

Quote Removal

After the preceding expansions, all unquoted occurrences of the characters \, ', and " that did not result from one of the above expansions are removed.

Related linux commands

Hyperpolyglot - variable assignment and expansion (compares bash, ksh,tsh)
Bash shell variables
BASH Syntax


 
Copyright © 1999-2024 SS64.com
Some rights reserved