====== More Shell Scripts ====== ---- * Also see the [[http://en.tldp.org/LDP/abs/html/index.html|Advanced Bash Scripting Guide]] and [[http://mywiki.wooledge.org/BashFAQ|Bash FAQ]]. ---- ===== Writing Shell Scripts ===== ==== Setup ==== * Create the ''~/cs370/examples/scripts'' subdirectory: mkdir -p ~/cs370/examples/scripts * Change to the just-created directory: cd ~/cs370/examples/scripts ==== Shell variables ==== * Setting a variable (in ''sh'' and ''sh''-compatible shells) var=value, where value is any valid string day='Sep 27, 2004' # Use quotes if string contains spaces day="Sep 27, 2004" day=$(date +%a) # command substitution, day=Mon * There can't be any spaces around the assignment operator, ''=''. ==== Accessing variable values ==== * Access variables with ''$var'' or ''${var}'' syntax * **(Do in class)** Example: #!/bin/bash # Save as shellvars.sh: Using a variable as part of a string var=bat echo $varman # won't work; empty variable; prints blank line echo '$var'man # single quotes suppress variable expansion # All the following will print "batman": echo "$var"man # double quote the variable echo $var"man" # double quote the constant echo $var'man' # single quote the constant echo $var\man # separate the parameters echo ${var}man # isolate the variable Make the shellvars.sh script executable (chmod +x). Run the shellvars.sh script: ./shellvars.sh ==== Variable types ==== * Unexpected things may happen with variables, especially when coming from other programming languages. * "Essentially, Bash variables are character strings, but, depending on context, Bash permits arithmetic operations and comparisons on variables. The determining factor is whether the value of a variable contains only digits." * See http://www.tldp.org/LDP/abs/html/untyped.html ==== Special built-in script variables ==== * Positional parameters * ''$#'' : number of arguments passed to a script on the command line * ''$0'' : the name of the current shell or program * ''$n'' : argument on the command line, where n starts from 1, reading left to right * ''$*'' : all arguments on the command line except ''$0'' as a single string * Often used to pass all the arguments to another program or script * ''$@'' : all arguments on the command line, each separately quoted ''("$1" "$2" ... "$9" ...)'' * Consider this a ''list'' of the command line arguments. * ''$?'' : exit value of the last command executed in the script * ''$?'' is 0 if successful completion, not 0 if unsuccessful * Useful for error handling * ''$$'' : process id of the script itself * ''$!'' : process id of the last command done in background * **(Do in class)** Example: #!/bin/bash # Save as autovars.sh: display special shell variables ps echo '$?:' $? # exit status from ps echo '$$:' $$ # PID of this script echo '$!:' $! # PID of last command run in background echo '$0:' $0 # name of script echo '$#:' $# # number of command line args echo '$*:' $* # all command line args as string echo '$@:' $@ # all command line args as list echo '$1:' $1 # 1st command line arg echo '$2:' $2 echo '$3:' $3 echo '$4:' $4 # 4th command line arg Make the autovars.sh script executable (chmod +x). Run the autovars.sh script with four commandline arguments: ./autovars.sh mon tue wed thu Expected output (approximate): PID TTY TIME CMD 1316 pts/8 00:00:00 autovars.sh 1317 pts/8 00:00:00 ps 30486 pts/8 00:00:02 bash 30650 pts/8 00:00:00 bash $?: 0 $$: 1316 $!: $0: ./autovars.sh $#: 4 $*: mon tue wed thu $@: mon tue wed thu $1: mon $2: tue $3: wed $4: thu ==== Input in shell scripts ==== * The shell can use the built-in command, ''read'', to read in a line, e.g.: read var * Example: #!/bin/bash # Save as read_input.sh echo "Input a string below:" read inp echo "You entered: $inp" Make the script executable (chmod +x). Run the script: ./read_input.sh Expected output: Input a string below: That boy sure is a Unix fool. You entered: That boy sure is a Unix fool. * The ''read'' command can also read input non-interactively but can only read the 1st line of multi-line input. Run above script again with input through a pipe: echo "That boy sure is a Unix fool." | ./read_input.sh fortune | ./read_input.sh ==== Shell functions ==== * A shell function, //after it has been defined//, has the form: fcn () { line 1; line 2; ... line n } * Running the built-in ''set'' command in the shell will show you any defined functions. * Defining a shell function requires this syntax: function fcn { command 1; command 2; ... command n; } * The space after { and the semicolons (;) are required. * The function definition can be written over multiple lines, i.e., function fcn { command 1; command 2; ... command n; } * Parentheses are not used to define function input parameters. * Instead, the same set of [[cs_370_-_shell_scripting#special_built-in_script_variables|positional parameters]] are used, but as local parameters. * Example: function lls { /bin/ls -sbF "$@"; } # Here, the special var "$@" contains the list of arguments to the function lls. # Using "$@", the function can be called with multiple files and directories as # command line arguments, e.g., # lls /dev /home /var /tmp * A function defined in an interactive shell or defined in your shell config (''~/.bashrc'') becomes a command that you can run in the shell. * (**Do in class**) Create a function ''mkdircd'' that [[cs_370_-_unix_shells_and_shell_scripting#shell_scripts_run_in_sub-shells|makes a directory and changes to the directory]]. * Add the function to your ''~/.bashrc'' along with a comment block. * Unlike a shell script, a function does not run in a sub-shell. * Functions can ''return'' an integer between 0-256 using the ''return'' statement. * That is, ''return'' is only meant to return an exit status from the function. * A function's return value is assigned to ''$?'', the exit status variable. * If a function doesn't have a ''return'' statement in it, it returns the value of ''$?'' from the last command or statement in the function body. * Example: The following calc function is useless as a calculator: function calc { answer=$(( $1 )); return $answer; } Try to run the calc() function: ans=calc "2*9" # Can't do this; it's a syntax error. calc "2*9" # returns 18, which is assigned to $? echo $? # outputs 18 calc "2*9*18" echo $? # expect 324, but outputs 68 A better calculator: function calc { answer=$(( $1 )); echo $answer; } Run the new calc() function: calc "2*9*18" # outputs 324 to stdout echo $? # exit status from the function is 0 To store stdout from calc() in a var, use command substitution: ans=$(calc "2*9*18") echo $ans # outputs 324 ---- ---- ===== Shell Control Structures ===== ==== Shell conditional expressions with test ==== * Also see the Advanced Bash-Scripting Guide on [[https://tldp.org/LDP/abs/html/testconstructs.html|Test Constructs]] and the ''test'' manual page (''man test''). * Conditional expressions * Shell control structures often branch based on whether an expression evaluates to true or false using the ''test'' command, or its more common equivalent, the ''[ ]'' operators. * Syntax of test: [ expression ] # the spaces around ''expression'' are significant or test expression * ''test'' returns a zero [[cs_370_-_shell_scripting#special_built-in_script_variables|exit status]] if expression evaluates to true; else it returns a non-zero exit status. * Syntax examples: FOO=bar # Set FOO test $FOO = bar # Test equality using test command. $ before FOO required. [ $FOO = bar ] # Test quality using [ ] operator. Spaces are significant. echo $? # The exit status $? should be 0 since previous test statement was true. [ $FOO = buzz ] echo $? # Exit status $? should be non-zero since previous statement was false. # Conditional chaining also depends on the value of the $? exit status: [ $FOO = bar ] && echo "That's correct." [ $FOO = buzz ] || echo "That's incorrect." # Always use spaces around comparison operators and operands: [ $FOO = bar ] # Legal [ $FOO=bar ] # Legal, but bash sees $FOO=bar as a single var name, so this condition will always be TRUE. [ $FOO= bar ] # Illegal [ $FOO =bar ] # Illegal [$FOO = bar] # Illegal ==== File/directory tests ==== * In shell scripts it is often desirable to ''test'' for the existence or certain attributes of files. To ''test'' file attributes, a ''test'' expression will have the form: [ -option filename ] or test -option filename * Some options available for the ''test'' operator for files: -r filename true if filename exists and is readable -w filename true if filename exists and is writable -x filename true if filename exists and is executable -f filename true if filename exists and is a regular file (not a directory) -d filename true if filename exists and is a directory -h or -L filename true if filename exists and is a symbolic link -p filename true if file exists and is a named pipe (fifo) -s filename true if file exists and is greater than zero in size ==== String tests ==== * Testing for strings: [ -z string ] true if the string length is zero [ -n string ] true if the string length is non-zero [ string1 = string2 ] true if string1 is identical to string2; spaces ARE significant [ string1 != string2 ] true if string1 is not identical to string2; spaces ARE significant [ string ] true if string is not NULL ==== Numeric comparison tests ==== * **Integer** comparisons: [ n1 -eq n2 ] true if integers n1 and n2 are equal [ n1 -ne n2 ] true if integers n1 and n2 are not equal [ n1 -gt n2 ] true if integer n1 is greater than integer n2 [ n1 -ge n2 ] true if integer n1 is greater than or equal to integer n2 [ n1 -lt n2 ] true if integer n1 is less than integer n2 [ n1 -le n2 ] true if integer n1 is less than or equal to integer n2 ==== Logical tests ==== * Logical operations: [ ! expression ] true if expression is false [ expr1 -a expr2 ] true if both expr1 and expr2 are true [ expr1 -o expr2 ] true if either expr1 or expr2 are true ---- ==== The if structure ==== * Need ''if'' structure for more complex decision making using the ''[ ]'' test construct. * Syntax: if condition1; then command list if condition1 is true [elif condition2; then command list if condition2 is true] [else command list if condition1 is false] fi * The conditions are evaluated using the [ ] operator (''test''). * The ''if'' and ''then'' must be separated, either with a or a semicolon (;). * Example: #!/bin/bash # Save as ifdemo1.sh: Demonstrate use of if with [] if [ $# -ge 2 ]; then # '$#' is the built-in var that contains the number of command line args echo $2 elif [ $# -eq 1 ]; then echo $1 else echo No input fi # which is the same as... if [ $# -ge 2 ]; then echo $2; elif [ $# -eq 1 ]; then echo $1; else echo No input; fi * Again, spaces are significant in the format of the conditional test. * At least one space is needed after ''['' and one before '']''. ---- ==== The case structure ==== * Syntax: case parameter in pattern1[|pattern1a]) command list1;; pattern2) command list2 command list2a;; pattern3) command list3;; *) ;; esac * The ;; ends each choice and can be on the same line, or following a . * Additional alternative patterns to be selected for a particular case are separated by the vertical bar (|) as in the first pattern line in the example above. * The wildcard symbols, "?" to indicate any one character and "*" to match any number of characters, can be used either alone or adjacent to fixed strings. * Example: #!/bin/bash # Save as casedemo1.sh: Demonstrate use of case case $1 in aa|ab) echo A ;; b?) echo B ;; c*) echo C;; *) echo D;; esac * The following might be inserted in ''~/.bashrc'' to find and run the ''fortune'' command, if available: # Using case to try to find path to 'fortune' case $HOSTNAME in csse*clus*|aristotle*|plato ) # applies to Linux cluster machines or plato (aristotleii) FORTUNECMD=/usr/games/fortune ;; rockhopper ) # applies to rockhopper FORTUNECMD=/bin/fortune ;; * ) # applies to everything else FORTUNECMD='' esac $FORTUNECMD # Runs the fortune command defined above. * **(Do in class)** Revise the ''[[cs370/cs_370_-_unix_shells_and_shell_scripting#simplest_scripts|~/bin/mycal.sh]]'' script. * Revise ''mycal.sh'' to run either the ''cal'' or ''ncal'' command, depending on which host it is being run. * If it's running on rockhopper, run ''cal'', else run ''ncal''. # Change to your ~/bin directory cd ~/bin # Edit and save a new text file, mycal.sh # See https://cssegit.monmouth.edu/jchung/csse370repo/-/blob/main/scripts/mycal # for the contents of mycal.sh. nano mycal.sh # Follow instructions given in class for revising mycal.sh. ---- ==== The for structure ==== * Syntax: for variable [in list_of_values]; do command list [break] [continue] done * This is a for-each type loop. * The list_of_values is optional, with $@ (list of command line arguments) assumed if no list is specified. * Each value in this list is sequentially substituted for the variable until the list is emptied. * Wildcards can be used and are applied to file names in the current or other specified directory. * The ''break'' command exits the for loop. * The ''continue'' command jumps to the beginning of the ''for'' loop. * Example 1: #!/bin/bash # # Save as old2new.sh: # Illustrate the for loop in copying all files ending in ".old" # to similar names ending in ".new". for file in *.old; do # list contains files ending in ".old" in current dir newfile=$(basename $file .old) # See basename manpage cp $file $newfile.new done * Example 2: #!/bin/bash # Save as forargs.sh: # Show use of $@ (list of command line args) in for loop echo echo 'Looping through items in $@' for i in $@; do echo $i done * Example 3: **(Do in class)** #!/bin/bash # # Save as ping-hh305.sh: # Use ping in a for loop to determine what machines are up in HH 305. # Download list of hostnames and save it as hh305.hosts in the same # directory as this script by running: # wget http://tiny.cc/b7zayz -O hh305.hosts list_of_hostnames=$(cat hh305.hosts) # Command substitution for each in $list_of_hostnames; do ping -q -c 1 -w 2 $each # See Linux ping manpage done * Loop through a fixed sequence of numbers using the ''seq'' command seq manpage summary: NAME seq - print a sequence of numbers SYNOPSIS seq [OPTION]... LAST seq [OPTION]... FIRST LAST seq [OPTION]... FIRST INCREMENT LAST Using seq command substitution to set number of for loop reps: for each in $(seq 1 10); do # repeat 10 times echo $each done * Loop through a fixed sequence of numbers using bash ''{}'' ([[http://www.tldp.org/LDP/abs/html/special-chars.html#BRACEEXPREF|brace expansion]]) for each in {1..10}; do # repeat 10 times echo $each done ---- ==== The while structure ==== * while syntax: while condition; do command list [break] [continue] done * The condition is evaluated using [ ] (''test''). * The condition is tested at the start of each loop and the loop is terminated when the condition is false. * The ''break'' command exits the ''while'' loop. * The ''continue'' command jumps to the beginning of the ''while'' loop. * Example (using shift): #!/bin/bash # # Save as whileargs.sh: # This script takes the list of arguments, echoes the first one, # then shifts the list to the left. It loops through until it has # shifted all the arguments off the argument list. while [ $# -gt 0 ]; do echo "Number of arguments: $#" echo "First argument: $1" echo echo "shift executed"; shift done ==== Reading lines from files ==== * Reading one line at a time from a text file requires a loop. * See [[CS 370 - Reading Lines from Files|Reading Lines From Files]]. ---- ===== ksh / bash Extensions ===== ==== arithmetic operations ==== * The ''let'' command and the equivalent (( )) notation * Supports all basic math operators using standard operator precedence rules. * No spaces or tabs are allowed when using let: let x = 2 + 2 # expression contains illegal spaces ksh: =: unexpected `=' # ksh returns an error (( x = 2 + 2 )) # spaces are allowed using (( )) * For arithmetic tests, (( )) can be used instead of test expressions: while (( i <= 32 )) is the same as while [ i -le 32 ] ==== Extended test construct, [[ ]] ==== See http://en.tldp.org/LDP/abs/html/testconstructs.html#DBLBRACKETS ---- ==== More on shell functions ==== * Function arguments: * Functions may process input parameters passed to them. * The function refers to the input parameters by position, that is, ''$1'', ''$''2, and so forth. * Note that a shell script also uses the positional parameters (''$1'', ''$2'', etc.) for its command line args. * If you see ''$1'', ''$2'', etc. in a function body, those are the function's input parameters, not the shell script's command line args. * Example 1: #!/bin/bash # Save as use_function.sh. # function to demonstrate input parameters function func2 { if [ -z "$1" ]; then # Checks if input parameter 1 is zero length. echo "-Parameter #1 is zero length.-" # Also applies if no parameters are passed. else echo "-Parameter #1 is \"$1\".-" fi if [ "$2" ]; then echo "-Parameter #2 is \"$2\".-" fi return 0 # Return 0 exit status by default. } echo echo "Nothing passed to function." func2 # Called with no params echo echo "Zero-length parameter passed to function." func2 "" # Called with zero-length param echo echo "Null parameter passed to function." func2 "$uninitialized_param" # Called with uninitialized param echo echo "One parameter passed to function." func2 first # Called with one param echo echo "Two parameters passed to function." func2 first second # Called with two params echo second="2ndParam" echo "Zero-length and string parameters passed to function." func2 "" $second # Called with zero-length first parameter echo # and string as a second parameter. echo "Show that arguments passed to the shell script are not the same" echo "as arguments passed to functions in the script:" echo echo "The script's 1st argument is $1" echo "The script's 2nd argument is $2" exit 0 # Explicitly set a default exit status from this shell script. * Example 2: * Functions can return ints between 0-256. * Returned values are assigned to $? (exit status var). #!/bin/bash # Save as return_max.sh: Maximum of two integers using function return. # Script global variables E_PARAM_ERR=-198 # If less than 2 params passed to function. EQUAL=-199 # Return value if both params equal. function max2 # Returns larger of two numbers. { # Note: numbers compared must be between 0-256. if [ -z "$2" ]; then return $E_PARAM_ERR fi if [ "$1" -eq "$2" ]; then return $EQUAL else if [ "$1" -gt "$2" ]; then return $1 else return $2 fi fi } max2 33 34 # Call max2 w/ 2 params return_val=$? if [ "$return_val" -eq $E_PARAM_ERR ]; then echo "Need to pass two parameters to the function." elif [ "$return_val" -eq $EQUAL ]; then echo "The two numbers are equal." else echo "The larger of the two numbers is $return_val." fi exit 0 * Variable scope: * Before a function is called, all variables declared within the function are invisible outside the body of the function. * Variables declared local are always invisible outside the body of the function. * Example 3: #!/bin/bash # Save as fnvarscope.sh: Test function var scope function func { global_var=37 # Visible only within the function block # before the function has been called. local func_var=38 # Local to func () } # END OF FUNCTION echo "global_var = $global_var" # global_var = # Function "func" has not yet been called, # so $global_var is not visible here. echo "func_var = $func_var" # Local var; expect this to be empty func echo "The function has been called." echo "global_var = $global_var" # global_var = 37 # Has been set by function call. echo "func_var = $func_var" # Local var; expect this to be empty # even after function call ----