More Shell Scripts

• Also see the Advanced Bash Scripting Guide and Bash FAQ.

• Create the ~/cs370/examples/scripts subdirectory:

mkdir -p ~/cs370/examples/scripts

• Change to the just-created directory:

cd ~/cs370/examples/scripts

• Setting a variable (in sh and sh-compatible shells)

var=value, where value is any valid string
  
day='Sep 27, 2004'   # Use quotes if string contains spaces
day="Sep 27, 2004"
day=$(date +%a)      # command substitution, day=Mon

• There can't be any spaces around the assignment operator, =.

• Access variables with $var or ${var} syntax
• (Do in class) Example:

#!/bin/bash
# Save as shellvars.sh: Using a variable as part of a string

var=bat

echo $varman         # won't work; empty variable; prints blank line
echo '$var'man       # single quotes suppress variable expansion

# All the following will print "batman":
echo "$var"man       # double quote the variable
echo $var"man"       # double quote the constant
echo $var'man'       # single quote the constant
echo $var\man        # separate the parameters
echo ${var}man       # isolate the variable

Make the shellvars.sh script executable (chmod +x).
Run the shellvars.sh script:

./shellvars.sh

• Unexpected things may happen with variables, especially when coming from other programming languages.
  • “Essentially, Bash variables are character strings, but, depending on context, Bash permits arithmetic operations and comparisons on variables. The determining factor is whether the value of a variable contains only digits.”
  • See http://www.tldp.org/LDP/abs/html/untyped.html

• Positional parameters
  • $# : number of arguments passed to a script on the command line
  • $0 : the name of the current shell or program
  • $n : argument on the command line, where n starts from 1, reading left to right
• $* : all arguments on the command line except $0 as a single string
  • Often used to pass all the arguments to another program or script
• $@ : all arguments on the command line, each separately quoted (“$1” “$2” … “$9” …)
  • Consider this a list of the command line arguments.

• $? : exit value of the last command executed in the script
  • $? is 0 if successful completion, not 0 if unsuccessful
  • Useful for error handling
• $$ : process id of the script itself
• $! : process id of the last command done in background

• (Do in class) Example:

#!/bin/bash
# Save as autovars.sh: display special shell variables
ps
echo '$?:' $? # exit status from ps
echo '$$:' $$ # PID of this script
echo '$!:' $! # PID of last command run in background
echo '$0:' $0 # name of script
echo '$#:' $# # number of command line args
echo '$*:' $* # all command line args as string
echo '$@:' $@ # all command line args as list
echo '$1:' $1 # 1st command line arg
echo '$2:' $2
echo '$3:' $3
echo '$4:' $4 # 4th command line arg

Make the autovars.sh script executable (chmod +x).
Run the autovars.sh script with four commandline arguments:

./autovars.sh mon tue wed thu


Expected output (approximate):

  PID TTY          TIME CMD
 1316 pts/8    00:00:00 autovars.sh
 1317 pts/8    00:00:00 ps
30486 pts/8    00:00:02 bash
30650 pts/8    00:00:00 bash
$?: 0
$$: 1316
$!:
$0: ./autovars.sh
$#: 4
$*: mon tue wed thu
$@: mon tue wed thu
$1: mon
$2: tue
$3: wed
$4: thu

• The shell can use the built-in command, read, to read in a line, e.g.:

read var

• Example:

#!/bin/bash
# Save as read_input.sh

echo "Input a string below:"
read inp
echo "You entered: $inp"


Make the script executable (chmod +x).
Run the script:

./read_input.sh


Expected output:
  
Input a string below:
That boy sure is a Unix fool.
You entered: That boy sure is a Unix fool.

• The read command can also read input non-interactively but can only read the 1st line of multi-line input.

Run above script again with input through a pipe:

echo "That boy sure is a Unix fool." | ./read_input.sh

fortune | ./read_input.sh

• A shell function, after it has been defined, has the form:

fcn () 
{ 
   line 1;
   line 2;
   ...
   line n
}

• Running the built-in set command in the shell will show you any defined functions.

• Defining a shell function requires this syntax:

function fcn { command 1; command 2; ... command n; }

• The space after { and the semicolons (;) are required.
• The function definition can be written over multiple lines, i.e.,

function fcn
{
   command 1;
   command 2;
   ...
   command n;
}

• Parentheses are not used to define function input parameters.
  • Instead, the same set of positional parameters are used, but as local parameters.
• Example:

function lls { /bin/ls -sbF "$@"; }
   # Here, the special var "$@" contains the list of arguments to the function lls.
   # Using "$@", the function can be called with multiple files and directories as
   # command line arguments, e.g.,
   #     lls /dev /home /var /tmp

• A function defined in an interactive shell or defined in your shell config (~/.bashrc) becomes a command that you can run in the shell.
  • (Do in class) Create a function mkdircd that makes a directory and changes to the directory.
    • Add the function to your ~/.bashrc along with a comment block.
  • Unlike a shell script, a function does not run in a sub-shell.

• Functions can return an integer between 0-256 using the return statement.
  • That is, return is only meant to return an exit status from the function.
  • A function's return value is assigned to $?, the exit status variable.
  • If a function doesn't have a return statement in it, it returns the value of $? from the last command or statement in the function body.
  • Example:

The following calc function is useless as a calculator:

function calc { answer=$(( $1 )); return $answer; } 

Try to run the calc() function:

ans=calc "2*9" # Can't do this; it's a syntax error.

calc "2*9" # returns 18, which is assigned to $?
echo $?    # outputs 18

calc "2*9*18"
echo $?       # expect 324, but outputs 68


A better calculator:

function calc { answer=$(( $1 )); echo $answer; } 

Run the new calc() function:

calc "2*9*18" # outputs 324 to stdout
echo $?       # exit status from the function is 0

To store stdout from calc() in a var, use command substitution:

ans=$(calc "2*9*18")
echo $ans     # outputs 324

• Also see the Advanced Bash-Scripting Guide on Test Constructs and the test manual page (man test).

• Conditional expressions
  • Shell control structures often branch based on whether an expression evaluates to true or false using the test command, or its more common equivalent, the [ ] operators.

• Syntax of test:

[ expression ]       # the spaces around ''expression'' are significant
  
or
  
test expression

• test returns a zero exit status if expression evaluates to true; else it returns a non-zero exit status.

• Syntax examples:

FOO=bar           # Set FOO
test $FOO = bar   # Test equality using test command. $ before FOO required.
[ $FOO = bar ]    # Test quality using [ ] operator. Spaces are significant.
echo $?           # The exit status $? should be 0 since previous test statement was true.
  
[ $FOO = buzz ]
echo $?           # Exit status $? should be non-zero since previous statement was false.
  
# Conditional chaining also depends on the value of the $? exit status:
[ $FOO = bar ] && echo "That's correct."
[ $FOO = buzz ] || echo "That's incorrect."
  
# Always use spaces around comparison operators and operands:
[ $FOO = bar ] # Legal
[ $FOO=bar ]   # Legal, but bash sees $FOO=bar as a single var name,
                 so this condition will always be TRUE.
[ $FOO= bar ]  # Illegal
[ $FOO =bar ]  # Illegal
[$FOO = bar]   # Illegal

• In shell scripts it is often desirable to test for the existence or certain attributes of files. To test file attributes, a test expression will have the form:

[ -option filename ]

	   or

test -option filename

• Some options available for the test operator for files:

-r filename    true if filename exists and is readable

-w filename    true if filename exists and is writable

-x filename    true if filename exists and is executable

-f filename    true if filename exists and is a regular file
			 (not a directory)

-d filename    true if filename exists and is a directory

-h or
-L filename    true if filename exists and is a symbolic link

-p filename    true if file exists and is a named pipe (fifo)

-s filename    true if file exists and is greater than zero in size

• Testing for strings:

[ -z string ]            true if the string length is zero

[ -n string ]            true if the string length is non-zero

[ string1 = string2 ]    true if string1 is identical to string2; spaces ARE significant

[ string1 != string2 ]   true if string1 is not identical to string2; spaces ARE significant

[ string ]               true if string is not NULL

• Integer comparisons:

[ n1 -eq n2 ]  true if integers n1 and n2 are equal

[ n1 -ne n2 ]  true if integers n1 and n2 are not equal

[ n1 -gt n2 ]  true if integer n1 is greater than integer n2

[ n1 -ge n2 ]  true if integer n1 is greater than or equal to integer n2

[ n1 -lt n2 ]  true if integer n1 is less than integer n2

[ n1 -le n2 ]  true if integer n1 is less than or equal to integer n2

• Logical operations:

[ ! expression ]       true if expression is false

[ expr1 -a expr2 ]     true if both expr1 and expr2 are true

[ expr1 -o expr2 ]     true if either expr1 or expr2 are true

• Need if structure for more complex decision making using the [ ] test construct.

• Syntax:

if condition1; then

	command list if condition1 is true

[elif condition2; then

	command list if condition2 is true]

[else

	command list if condition1 is false]

fi

• The conditions are evaluated using the [ ] operator (test).
• The if and then must be separated, either with a <newline> or a semicolon (;).

• Example:

#!/bin/bash
# Save as ifdemo1.sh: Demonstrate use of if with []

if [ $# -ge 2 ]; then   # '$#' is the built-in var that contains the number of command line args
	echo $2
elif [ $# -eq 1 ]; then
	echo $1
else
	echo No input
fi


# which is the same as...


if [ $# -ge 2 ]; then echo $2; elif [ $# -eq 1 ]; then echo $1; else echo No input; fi

• Again, spaces are significant in the format of the conditional test.
  • At least one space is needed after [ and one before ].

• Syntax:

case parameter in
  
    pattern1[|pattern1a]) command list1;;
  
    pattern2) command list2
  
              command list2a;;
  
    pattern3) command list3;;
  
    *) ;;

esac

• The ;; ends each choice and can be on the same line, or following a <newline>.

• Additional alternative patterns to be selected for a particular case are separated by the vertical bar (|) as in the first pattern line in the example above.

• The wildcard symbols, “?” to indicate any one character and “*” to match any number of characters, can be used either alone or adjacent to fixed strings.

• Example:

#!/bin/bash
# Save as casedemo1.sh: Demonstrate use of case 
  
case $1 in
    aa|ab) echo A
    ;;
    b?)    echo B
    ;;
    c*)    echo C;;
    *)     echo D;;
esac

• The following might be inserted in ~/.bashrc to find and run the fortune command, if available:

# Using case to try to find path to 'fortune'
case $HOSTNAME in
     csse*clus*|aristotle*|plato )
         # applies to Linux cluster machines or plato (aristotleii)
         FORTUNECMD=/usr/games/fortune
         ;;
     rockhopper )
         # applies to rockhopper
         FORTUNECMD=/bin/fortune
         ;;
     * )
         # applies to everything else
         FORTUNECMD=''
esac

$FORTUNECMD    # Runs the fortune command defined above.

• (Do in class) Revise the ~/bin/mycal.sh script.
  • Revise mycal.sh to run either the cal or ncal command, depending on which host it is being run.
    • If it's running on rockhopper, run cal, else run ncal.

# Change to your ~/bin directory
cd ~/bin

# Edit and save a new text file, mycal.sh
# See https://cssegit.monmouth.edu/jchung/csse370repo/-/blob/main/scripts/mycal 
# for the contents of mycal.sh.
nano mycal.sh

# Follow instructions given in class for revising mycal.sh.

• Syntax:

for variable [in list_of_values]; do
  
         command list
  
         [break]
  
         [continue]
  
done

• This is a for-each type loop.

• The list_of_values is optional, with $@ (list of command line arguments) assumed if no list is specified.

• Each value in this list is sequentially substituted for the variable until the list is emptied.

• Wildcards can be used and are applied to file names in the current or other specified directory.

• The break command exits the for loop.

• The continue command jumps to the beginning of the for loop.

• Example 1:

#!/bin/bash
#
# Save as old2new.sh:
# Illustrate the for loop in copying all files ending in ".old"
# to similar names ending in ".new".
  
for file in *.old; do    # list contains files ending in ".old" in current dir
         newfile=$(basename $file .old)        # See basename manpage
         cp $file $newfile.new
done

• Example 2:

#!/bin/bash
# Save as forargs.sh:
# Show use of $@ (list of command line args) in for loop
  
echo
echo 'Looping through items in $@'
  
for i in $@; do
         echo $i
done

• Example 3: (Do in class)

#!/bin/bash
#
# Save as ping-hh305.sh:
# Use ping in a for loop to determine what machines are up in HH 305.
# Download list of hostnames and save it as hh305.hosts in the same 
# directory as this script by running:
# wget http://tiny.cc/b7zayz -O hh305.hosts

list_of_hostnames=$(cat hh305.hosts)          # Command substitution
  
for each in $list_of_hostnames; do
         ping -q -c 1 -w 2 $each             # See Linux ping manpage
done

• Loop through a fixed sequence of numbers using the seq command

seq manpage summary:

NAME
       seq - print a sequence of numbers

SYNOPSIS
       seq [OPTION]... LAST
       seq [OPTION]... FIRST LAST
       seq [OPTION]... FIRST INCREMENT LAST

Using seq command substitution to set number of for loop reps:

for each in $(seq 1 10); do # repeat 10 times
   echo $each
done

• Loop through a fixed sequence of numbers using bash {} (brace expansion)

for each in {1..10}; do # repeat 10 times
   echo $each
done

• while syntax:

while condition; do
  
         command list
  
         [break]
  
         [continue]
  
done

• The condition is evaluated using [ ] (test).

• The condition is tested at the start of each loop and the loop is terminated when the condition is false.

• The break command exits the while loop.

• The continue command jumps to the beginning of the while loop.

• Example (using shift):

#!/bin/bash
#
# Save as whileargs.sh:
# This script takes the list of arguments, echoes the first one,
# then shifts the list to the left. It loops through until it has
# shifted all the arguments off the argument list.
  
while [ $# -gt 0 ]; do
         echo "Number of arguments: $#"
         echo "First argument: $1"
         echo
         echo "shift executed"; shift
done

• Reading one line at a time from a text file requires a loop.
• See Reading Lines From Files.

• The let command and the equivalent (( )) notation

• Supports all basic math operators using standard operator precedence rules.

• No spaces or tabs are allowed when using let:

 let x = 2 + 2      # expression contains illegal spaces
 ksh: =: unexpected `='       # ksh returns an error
 
 (( x = 2 + 2 ))    # spaces are allowed using (( ))

• For arithmetic tests, (( )) can be used instead of test expressions:

 while (( i <= 32 ))
 
 is the same as
 
 while [ i -le 32 ]

See http://en.tldp.org/LDP/abs/html/testconstructs.html#DBLBRACKETS

• Function arguments:
  • Functions may process input parameters passed to them.
  • The function refers to the input parameters by position, that is, $1, $2, and so forth.
    • Note that a shell script also uses the positional parameters ($1, $2, etc.) for its command line args.
    • If you see $1, $2, etc. in a function body, those are the function's input parameters, not the shell script's command line args.

• Example 1:

#!/bin/bash
# Save as use_function.sh.

# function to demonstrate input parameters
function func2
{
     if [ -z "$1" ]; then     # Checks if input parameter 1 is zero length.
       echo "-Parameter #1 is zero length.-"  # Also applies if no parameters are passed.
     else
       echo "-Parameter #1 is \"$1\".-"
     fi

     if [ "$2" ]; then
       echo "-Parameter #2 is \"$2\".-"
     fi

     return 0     # Return 0 exit status by default.
}

echo

echo "Nothing passed to function."   
func2                          # Called with no params
echo

echo "Zero-length parameter passed to function."
func2 ""                       # Called with zero-length param
echo

echo "Null parameter passed to function."
func2 "$uninitialized_param"   # Called with uninitialized param
echo

echo "One parameter passed to function."   
func2 first           # Called with one param
echo

echo "Two parameters passed to function."
func2 first second    # Called with two params
echo

second="2ndParam"
echo "Zero-length and string parameters passed to function."
func2 "" $second      # Called with zero-length first parameter
echo                  # and string as a second parameter.

echo "Show that arguments passed to the shell script are not the same"
echo "as arguments passed to functions in the script:"
echo
echo "The script's 1st argument is $1"
echo "The script's 2nd argument is $2"

exit 0 # Explicitly set a default exit status from this shell script.

• Example 2:
  • Functions can return ints between 0-256.
  • Returned values are assigned to $? (exit status var).

#!/bin/bash
# Save as return_max.sh: Maximum of two integers using function return.

# Script global variables
E_PARAM_ERR=-198    # If less than 2 params passed to function.
EQUAL=-199          # Return value if both params equal.

function max2       # Returns larger of two numbers.
{                   # Note: numbers compared must be between 0-256.
   if [ -z "$2" ]; then
      return $E_PARAM_ERR
   fi

   if [ "$1" -eq "$2" ]; then
      return $EQUAL
   else
      if [ "$1" -gt "$2" ]; then
         return $1
      else
         return $2
      fi
   fi
}

max2 33 34           # Call max2 w/ 2 params
return_val=$?

if [ "$return_val" -eq $E_PARAM_ERR ]; then
   echo "Need to pass two parameters to the function."
elif [ "$return_val" -eq $EQUAL ]; then
   echo "The two numbers are equal."
else
   echo "The larger of the two numbers is $return_val."
fi

exit 0

• Variable scope:
  • Before a function is called, all variables declared within the function are invisible outside the body of the function.
  • Variables declared local are always invisible outside the body of the function.

• Example 3:

#!/bin/bash
# Save as fnvarscope.sh: Test function var scope

function func
{
   global_var=37        #  Visible only within the function block
                        #  before the function has been called.
   local func_var=38    #  Local to func ()
}                       #  END OF FUNCTION

echo "global_var = $global_var"  # global_var =
                                 # Function "func" has not yet been called,
                                 # so $global_var is not visible here.     

echo "func_var = $func_var"      # Local var; expect this to be empty

func
echo "The function has been called."

echo "global_var = $global_var"  # global_var = 37
                                 # Has been set by function call.

echo "func_var = $func_var"      # Local var; expect this to be empty
                                 # even after function call