====== More On Shell Scripting ======
----
* Also see the [[http://en.tldp.org/LDP/abs/html/index.html|Advanced Bash Scripting Guide]] and [[http://mywiki.wooledge.org/BashFAQ|Bash FAQ]].
----
===== Writing Shell Scripts =====
==== Setup ====
* Create the //~/cs370/examples/shellscripting// subdirectory:
mkdir -p ~/cs370/examples/scripting
* Change to the just-created directory:
cd ~/cs370/examples/scripting
==== Shell variables ====
* Setting a variable (in //sh// and //sh//-compatible shells)
var=value, where value is any valid string
day='Sep 27, 2004' # Use quotes if string contains spaces
day="Sep 27, 2004"
day=$(date +%a) # command substitution, day=Mon
* There can't be any spaces around the assignment operator, //=//.
==== Accessing variable values ====
* Access variables with ''$var'' or ''${var}'' syntax
* Example:
#!/bin/bash
# Save as shellvars: Using a variable as part of a string
var=bat
echo $varman # won't work; empty variable; prints blank line
echo '$var'man # single quotes suppress variable expansion
# All the following will print "batman":
echo "$var"man # double quote the variable
echo $var"man" # double quote the constant
echo $var'man' # single quote the constant
echo $var\man # separate the parameters
echo ${var}man # isolate the variable
==== Variable types ====
* Unexpected things may happen with variables, especially when coming from other programming languages.
* "Essentially, Bash variables are character strings, but, depending on context, Bash permits arithmetic operations and comparisons on variables. The determining factor is whether the value of a variable contains only digits."
* See http://www.tldp.org/LDP/abs/html/untyped.html
==== Special built-in script variables ====
* Positional parameters
* ''$#'' : number of arguments passed to a script on the command line
* ''$0'' : the name of the current shell or program
* ''$n'' : argument on the command line, where n starts from 1, reading left to right
* ''$*'' : all arguments on the command line except ''$0'' as a single string
* Often used to pass all the arguments to another program or script
* ''$@'' : all arguments on the command line, each separately quoted ''("$1" "$2" ... "$9" ...)''
* Consider this a //list// of the command line arguments.
* ''$?'' : exit value of the last command executed in the script
* ''$?'' is 0 if successful completion, not 0 if unsuccessful
* Useful for error handling
* ''$$'' : process id of the script itself
* ''$!'' : process id of the last command done in background
* Example:
#!/bin/bash
# Save as autovars: display special shell variables
ps
echo '$?:' $? # exit status from ps
echo '$$:' $$ # PID of this script
echo '$!:' $! # PID of last command run in background
echo '$0:' $0 # name of script
echo '$#:' $# # number of command line args
echo '$*:' $* # all command line args as string
echo '$@:' $@ # all command line args as list
echo '$1:' $1 # 1st command line arg
echo '$2:' $2
echo '$3:' $3
echo '$4:' $4 # 4th command line arg
Make the autovars script executable (chmod +x).
Run the autovars script with four commandline arguments:
./autovars mon tue wed thu
Expected output (approximate):
PID TTY TIME CMD
1316 pts/8 00:00:00 autovars
1317 pts/8 00:00:00 ps
30486 pts/8 00:00:02 bash
30650 pts/8 00:00:00 bash
$?: 0
$$: 1316
$!:
$0: ./autovars
$#: 4
$*: mon tue wed thu
$@: mon tue wed thu
$1: mon
$2: tue
$3: wed
$4: thu
==== Input in shell scripts ====
* The shell can use the built-in command, //read//, to read in a line, e.g.:
read var
* Example:
#!/bin/bash
# Save as read_input
echo "Input a string below:"
read inp
echo "You entered: $inp"
Make the script executable (chmod +x).
Run the script:
./read_input
Expected output:
Input a string below:
That boy sure is a Unix fool.
You entered: That boy sure is a Unix fool.
* The //read// command can also read input non-interactively but can only read the 1st line of multi-line input.
Run above script again with input through a pipe:
echo "That boy sure is a Unix fool." | ./read_input
fortune | ./read_input
==== Shell functions ====
* A shell function, //after it has been defined//, has the form:
fcn ()
{
line 1;
line 2;
...
line n
}
* Running the built-in //set// command in the shell will show you any defined functions.
* Defining a shell function requires this syntax:
function fcn { command 1; command 2; ... command n; }
* The space after { and the semicolons (;) are required.
* The function definition can be written over multiple lines, i.e.,
function fcn
{
command 1;
command 2;
...
command n;
}
* Parentheses are not used to define function input parameters.
* Instead, the same set of [[cs_370_-_shell_scripting#special_built-in_script_variables|positional parameters]] are used, but as local parameters.
* Example:
function lls { /bin/ls -sbF "$@"; }
# Here, the special var "$@" contains the list of arguments to the function lls.
# Using "$@", the function can be called with multiple files and directories as
# command line arguments, e.g.,
# lls /dev /home /var /tmp
* A function defined in an interactive shell or defined in your shell config (~/.bashrc) becomes a command that you can run in the shell.
* (**In lab**) Create a function //mkdircd// that [[cs_370_-_unix_shells_and_shell_scripting#shell_scripts_run_in_sub-shells|makes a directory and changes to the directory]].
* Add the function to your //~/.bashrc//.
* Unlike a shell script, a function does not run in a sub-shell.
* Functions can //return// an integer between 0-256 using the //return// statement.
* That is, //return// is only meant to return an exit status from the function.
* A function's return value is assigned to //$?//, the exit status variable.
* If a function doesn't have a //return// statement in it, it returns the value of //$?// from the last command or statement in the function body.
* Example:
The following calc function is useless as a calculator:
function calc { answer=$(( $1 )); return $answer; }
Try to run the calc() function:
ans=calc "2*9" # Can't do this; it's a syntax error.
calc "2*9" # returns 18, which is assigned to $?
echo $? # outputs 18
calc "2*9*18"
echo $? # expect 324, but outputs 68
A better calculator:
function calc { answer=$(( $1 )); echo $answer; }
Run the new calc() function:
calc "2*9*18" # outputs 324 to stdout
echo $? # exit status from the function is 0
To store stdout from calc() in a var, use command substitution:
ans=$(calc "2*9*18")
echo $ans # outputs 324
----
----
===== Shell Control Structures =====
==== Shell conditional expressions with test ====
* Also see the Advanced Bash-Scripting Guide on [[https://tldp.org/LDP/abs/html/testconstructs.html|Test Constructs]] and the ''test'' manual page (''man test'').
* Conditional expressions
* Shell control structures often branch based on whether an expression evaluates to true or false using the //test// command, or its more common equivalent, the //[ ]// operators.
* Syntax of test:
[ expression ] # the spaces around ''expression'' are significant
or
test expression
* //test// returns a zero [[cs_370_-_shell_scripting#special_built-in_script_variables|exit status]] if expression evaluates to true; else it returns a non-zero exit status.
* Syntax examples:
FOO=bar # Set FOO
test $FOO = bar # Test equality using test command. $ before FOO required.
[ $FOO = bar ] # Test quality using [ ] operator. Spaces are significant.
echo $? # The exit status $? should be 0 since previous test statement was true.
[ $FOO = buzz ]
echo $? # Exit status $? should be non-zero since previous statement was false.
# Conditional chaining also depends on the value of the $? exit status:
[ $FOO = bar ] && echo "That's correct."
[ $FOO = buzz ] || echo "That's incorrect."
# Always use spaces around comparison operators and operands:
[ $FOO = bar ] # Legal
[ $FOO=bar ] # Legal, but bash sees $FOO=bar as a single var name,
so this condition will always be TRUE.
[ $FOO= bar ] # Illegal
[ $FOO =bar ] # Illegal
[$FOO = bar] # Illegal
==== File/directory tests ====
* In shell scripts it is often desirable to //test// for the existence or certain attributes of files. To //test// file attributes, a //test// expression will have the form:
[ -option filename ]
or
test -option filename
* Some options available for the //test// operator for files:
-r filename true if filename exists and is readable
-w filename true if filename exists and is writable
-x filename true if filename exists and is executable
-f filename true if filename exists and is a regular file
(not a directory)
-d filename true if filename exists and is a directory
-h or
-L filename true if filename exists and is a symbolic link
-p filename true if file exists and is a named pipe (fifo)
-s filename true if file exists and is greater than zero in size
==== String tests ====
* Testing for strings:
[ -z string ] true if the string length is zero
[ -n string ] true if the string length is non-zero
[ string1 = string2 ] true if string1 is identical to string2; spaces ARE significant
[ string1 != string2 ] true if string1 is not identical to string2; spaces ARE significant
[ string ] true if string is not NULL
==== Numeric comparison tests ====
* **Integer** comparisons:
[ n1 -eq n2 ] true if integers n1 and n2 are equal
[ n1 -ne n2 ] true if integers n1 and n2 are not equal
[ n1 -gt n2 ] true if integer n1 is greater than integer n2
[ n1 -ge n2 ] true if integer n1 is greater than or equal to integer n2
[ n1 -lt n2 ] true if integer n1 is less than integer n2
[ n1 -le n2 ] true if integer n1 is less than or equal to integer n2
==== Logical tests ====
* Logical operations:
[ ! expression ] true if expression is false
[ expr1 -a expr2 ] true if both expr1 and expr2 are true
[ expr1 -o expr2 ] true if either expr1 or expr2 are true
----
==== The if structure ====
* Need //if// structure for more complex decision making using [].
* Syntax:
if condition1; then
command list if condition1 is true
[elif condition2; then
command list if condition2 is true]
[else
command list if condition1 is false]
fi
* The conditions are evaluated using the [ ] operator (//test//).
* The //if// and //then// must be separated, either with a or a semicolon (;).
* Example:
#!/bin/bash
# Save as ifdemo1: Demonstrate use of if with []
if [ $# -ge 2 ]; then # '$#' is the built-in var that contains the number of command line args
echo $2
elif [ $# -eq 1 ]; then
echo $1
else
echo No input
fi
# which is the same as...
if [ $# -ge 2 ]; then echo $2; elif [ $# -eq 1 ]; then echo $1; else echo No input; fi
* Again, spaces are significant in the format of the conditional test.
* At least one space needed after [ and one before ]
----
==== The case structure ====
* Syntax:
case parameter in
pattern1[|pattern1a]) command list1;;
pattern2) command list2
command list2a;;
pattern3) command list3;;
*) ;;
esac
* The ;; ends each choice and can be on the same line, or following a .
* Additional alternative patterns to be selected for a particular case are separated by the vertical bar (|) as in the first pattern line in the example above.
* The wildcard symbols, "?" to indicate any one character and "*" to match any number of characters, can be used either alone or adjacent to fixed strings.
* Example:
#!/bin/bash
# Save as casedemo1: Demonstrate use of case
case $1 in
aa|ab) echo A
;;
b?) echo B
;;
c*) echo C;;
*) echo D;;
esac
* The following might be inserted in .bashrc to find and run the //fortune// command, if available:
# Using case to try to find path to 'fortune'
case $HOSTNAME in
csse*clus*|aristotle*|plato )
# applies to Linux cluster machines or plato (aristotleii)
FORTUNECMD=/usr/games/fortune
;;
rockhopper )
# applies to rockhopper
FORTUNECMD=/bin/fortune
;;
* )
# applies to everything else
FORTUNECMD=''
esac
$FORTUNECMD # Runs the fortune command defined above.
----
==== The for structure ====
* Syntax:
for variable [in list_of_values]; do
command list
[break]
[continue]
done
* This is a for-each type loop.
* The list_of_values is optional, with $@ (list of command line arguments) assumed if no list is specified.
* Each value in this list is sequentially substituted for the variable until the list is emptied.
* Wildcards can be used and are applied to file names in the current or other specified directory.
* The //break// command exits the for loop.
* The //continue// command jumps to the beginning of the //for// loop.
* Example 1:
#!/bin/bash
#
# Save as old2new:
# Illustrate the for loop in copying all files ending in ".old"
# to similar names ending in ".new".
for file in *.old; do # list contains files ending in ".old" in current dir
newfile=$(basename $file .old) # See basename manpage
cp $file $newfile.new
done
* Example 2:
#!/bin/bash
# Save as forargs:
# Show use of $@ (list of command line args) in for loop
echo
echo 'Looping through items in $@'
for i in $@; do
echo $i
done
* Example 3:
#!/bin/bash
#
# Save as ping-hh305:
# Use ping in a for loop to determine what machines are up in HH 305.
# Download list of hostnames at http://tiny.cc/rhm7vz (Summer 2023)
# and save it as hh305.hosts in the same directory as this script.
#
list_of_hostnums=$(cat hh305.hosts) # Command substitution
for each in $list_of_hostnums; do
ping -q -c 1 -w 2 $each # See Linux ping manpage
done
* Loop through a fixed sequence of numbers using the //seq// command
seq manpage summary:
NAME
seq - print a sequence of numbers
SYNOPSIS
seq [OPTION]... LAST
seq [OPTION]... FIRST LAST
seq [OPTION]... FIRST INCREMENT LAST
Using seq command substitution to set number of for loop reps:
for each in $(seq 1 10); do # repeat 10 times
echo $each
done
* Loop through a fixed sequence of numbers using bash {} ([[http://www.tldp.org/LDP/abs/html/special-chars.html#BRACEEXPREF|brace expansion]])
for each in {1..10}; do # repeat 10 times
echo $each
done
----
==== The while structure ====
* while syntax:
while condition; do
command list
[break]
[continue]
done
* The condition is evaluated using [ ] (//test//).
* The condition is tested at the start of each loop and the loop is terminated when the condition is false.
* The //break// command exits the //while// loop.
* The //continue// command jumps to the beginning of the //while// loop.
* Example (using shift):
#!/bin/bash
#
# Save as whileargs:
# This script takes the list of arguments, echoes the first one,
# then shifts the list to the left. It loops through until it has
# shifted all the arguments off the argument list.
while [ $# -gt 0 ]; do
echo "Number of arguments: $#"
echo "First argument: $1"
echo
echo "shift executed"; shift
done
==== Reading lines from files ====
* Reading one line at a time from a text file requires a loop.
* See [[CS 370 - Reading Lines from Files|Reading Lines From Files]].
----
===== ksh / bash Extensions =====
==== arithmetic operations ====
* The //let// command and the equivalent (( )) notation
* Supports all basic math operators using standard operator precedence rules.
* No spaces or tabs are allowed when using let:
let x = 2 + 2 # expression contains illegal spaces
ksh: =: unexpected `=' # ksh returns an error
(( x = 2 + 2 )) # spaces are allowed using (( ))
* For arithmetic tests, (( )) can be used instead of test expressions:
while (( i <= 32 ))
is the same as
while [ i -le 32 ]
==== Extended test construct, [[ ]] ====
See http://en.tldp.org/LDP/abs/html/testconstructs.html#DBLBRACKETS
----
==== More on shell functions ====
* Function arguments:
* Functions may process input parameters passed to them.
* The function refers to the input parameters by position, that is, $1, $2, and so forth.
* Note that a shell script also uses the positional parameters ($1, $2, etc.) for its command line args.
* If you see $1, $2, etc. in a function body, those are the function's input parameters, not the shell script's command line args.
* Example 1:
#!/bin/bash
# Save as use_function.
# function to demonstrate input parameters
function func2
{
if [ -z "$1" ]; then # Checks if input parameter 1 is zero length.
echo "-Parameter #1 is zero length.-" # Also applies if no parameters are passed.
else
echo "-Parameter #1 is \"$1\".-"
fi
if [ "$2" ]; then
echo "-Parameter #2 is \"$2\".-"
fi
return 0 # Return 0 exit status by default.
}
echo
echo "Nothing passed to function."
func2 # Called with no params
echo
echo "Zero-length parameter passed to function."
func2 "" # Called with zero-length param
echo
echo "Null parameter passed to function."
func2 "$uninitialized_param" # Called with uninitialized param
echo
echo "One parameter passed to function."
func2 first # Called with one param
echo
echo "Two parameters passed to function."
func2 first second # Called with two params
echo
second="2ndParam"
echo "Zero-length and string parameters passed to function."
func2 "" $second # Called with zero-length first parameter
echo # and string as a second parameter.
echo "Show that arguments passed to the shell script are not the same"
echo "as arguments passed to functions in the script:"
echo
echo "The script's 1st argument is $1"
echo "The script's 2nd argument is $2"
exit 0 # Explicitly set a default exit status from this shell script.
* Example 2:
* Functions can return ints between 0-256.
* Returned values are assigned to $? (exit status var).
#!/bin/bash
# Save as return_max: Maximum of two integers using function return.
# Script global variables
E_PARAM_ERR=-198 # If less than 2 params passed to function.
EQUAL=-199 # Return value if both params equal.
function max2 # Returns larger of two numbers.
{ # Note: numbers compared must be between 0-256.
if [ -z "$2" ]; then
return $E_PARAM_ERR
fi
if [ "$1" -eq "$2" ]; then
return $EQUAL
else
if [ "$1" -gt "$2" ]; then
return $1
else
return $2
fi
fi
}
max2 33 34 # Call max2 w/ 2 params
return_val=$?
if [ "$return_val" -eq $E_PARAM_ERR ]; then
echo "Need to pass two parameters to the function."
elif [ "$return_val" -eq $EQUAL ]; then
echo "The two numbers are equal."
else
echo "The larger of the two numbers is $return_val."
fi
exit 0
* Variable scope:
* Before a function is called, all variables declared within the function are invisible outside the body of the function.
* Variables declared local are always invisible outside the body of the function.
* Example 3:
#!/bin/bash
# Save as fnvarscope: Test function var scope
function func
{
global_var=37 # Visible only within the function block
# before the function has been called.
local func_var=38 # Local to func ()
} # END OF FUNCTION
echo "global_var = $global_var" # global_var =
# Function "func" has not yet been called,
# so $global_var is not visible here.
echo "func_var = $func_var" # Local var; expect this to be empty
func
echo "The function has been called."
echo "global_var = $global_var" # global_var = 37
# Has been set by function call.
echo "func_var = $func_var" # Local var; expect this to be empty
# even after function call
----