The "shell" sits between you and the operating system

• a user interface
  • reads input
  • translates commands into actions to be taken by the system
• acts as a command interpreter
  • If commands are in a shell script, interprets the shell script.
    • The "Shell" language can still be considered an important language in some domains, despite being about as old as C.

See http://www.faqs.org/faqs/unix-faq/shell/shell-differences/ for a history of Unix shells and a shell feature comparison.

Also see https://wikiless.tiekoetter.com/wiki/Unix_shell, the Wikipedia article.

• I/O control features - often used for scripts
• Not well suited for interactive use
  • Lacks modern bells & whistles, e.g. command history, completion, aliases, shell prompt setting
• Other shells based on Bourne are better suited for interactive users
  • ksh, bash, zsh are sh-compatible and far easier to work with interactively
• Default shell prompt is $ (can't be customized)

• Nicer for interactive use than sh
• Uses C-like syntax for scripting
  • Not sh-compatible
• I/O more awkward than Bourne shell
• Introduced concept of job control
• Introduced command history
• Introduced use of ~ symbol to indicate a home directory (user's or others')
• Default prompt is %
• tcsh is a more modern implementation that introduces more bells & whistles such as completion and scrollable command history

• Developed by David Korn at AT&T
• sh-compatible
• Retained many csh features that were good for interactive work
• Became part of System V UNIX standard
  • Licensed by AT&T;
    • used to have to pay to use it
    • public domain ksh (pdksh) is free and used in Linux and other free versions of UNIX

• Developed as a sh-compatible, free (open source ) shell for the GNU project
• The currently dominant Linux/UNIX shell
• Default shell in macOS (was)
• Features from csh and ksh

• Z shell (zsh)
• friendly interactive shell (fish)
• Almquist shell (ash/dash)

Based on the Bourne Shell (sh):

• Korn (ksh)
• Bourne-Again Shell (bash)
• Z Shell (zsh)

Based on the C Shell (csh):

• T-C shell (tcsh)
• ksh, bash, zsh also incorporate useful features of csh

• The shells have a number of built-in commands:
  • Executed directly by the shell
  • Don't have to call another program to be run
  • Different shells have different built-ins
  • There is no executable file in a bin directory for built-in commands.
  • Examples: alias, echo, jobs, fg, bg, cd, history, pwd

• The which command returns the full path of a command, if the command is an external program.

$ which echo
/bin/echo                # (output of which)

• The type built-in command tells you whether a command is a built-in command or a different type.

$ type -a echo
echo is a shell builtin
echo is /bin/echo        # (Note that echo is both a shell-builtin as well as an external command.)

(See previous discussion)

Additional commands:

• kill command kills the specified background process
  • kill -9 kills the specified background process quickly and messily; should use as last resort
• pkill and pkill -9 commands kill jobs matching a text pattern
  • pkill -u jchung kills all jobs belonging to user jchung and will probably log jchung off.
    • Useful only for the root user and jchung himself.
• nice and renice
  • affect the priority of processes
    • nice allows you to start jobs with lower system priority.
    • renice allows you to change a job's “niceness” (priority) while it's running.
      • However, only root can make a running process “not nice”, that is, give it a higher than normal priority.

• Note previous discussion of the cat command.

• I/O redirection and piping allow…
  • Output redirection to a file
  • Input redirection from a file
  • Piping
    • Output of one command becomes the input of a subsequent command
    • Enables the “UNIX philosophy”

• See Standard streams.

• stdin - Standard input to the program, file descriptor 0
  • Normally from the keyboard, but can redirect from a file or command
  • Java uses stdin through the java.util.Scanner class.
  • C++ uses stdin through the cin standard input stream statement.

• stdout - Standard output from the program, file descriptor 1
  • Normal output from a program
  • Java uses stdout through the System.out.print (and println) method.
  • C++ uses stdout through the cout standard output stream statement.

• stderr - Standard error output, file descriptor 2
  • Error messages or warnings of unusual situations
  • Java uses stderr through the System.err.print (and println) method.
  • C++ uses stderr through the cerr standard output stream statement.

• Stdin can be manual keyboard input or can be redirected from a file or other command.
• Both stdout & stderr normally sent to the terminal screen, but can redirect either or both to a file or pipe to another command

• To redirect stdout to outfile

          command > outfile

• To append stdout to outfile

          command >> outfile

• To redirect input from infile

          command < infile

• To pipe standard output to another command

          command1 | command2

• To direct a command's stderr to file

          command 2> file

• To direct both stdout and stderr to file

          command > file 2>&1
          
          or just
          
          command &> file

• To append both stdout and stderr to file

          command >> file 2>&1
          
          or just
          
          command &>> file

• To pipe stdout and stderr to command2

          command1 2>&1 | command2
          
          or just
          
          command |& command2

• To redirect stdout and stderr to two separate files
  • (the “1” in “1>” is optional)

          command 1> stdoutfile 2> stderrfile

• To discard stderr
  • (The /dev/null special file is a system “black hole” for output or errors.)

          command 2> /dev/null

• To discard both stdout and stderr

          command > /dev/null 2>&1
          
          or just
          
          command &> /dev/null

# In this example, we will redirect the standard output from a command
# to stdoutfile and the standard error from the same command to stderrfile.

# First, run the command normally to send stderr and stdout from ls to the screen:
ls -l /dev/sda1 /dev/foo

(This is stderr->) ls: cannot access /dev/foo: No such file or directory
(This is stdout->) brw-rw---- 1 root disk 8, 1 2010-08-21 16:32 /dev/sda1


# Next, run the command to redirect both stdout (1>) and stderr (2>) to files:
ls -l /dev/sda1 /dev/foo 1> stdoutfile 2> stderrfile

cat stdoutfile
brw-rw---- 1 root disk 8, 1 2010-08-21 16:32 /dev/sda1

cat stderrfile
ls: cannot access /dev/foo: No such file or directory

# Run first 'ls -l' command, but now discarding stderr:
ls -l /dev/sda1 /dev/foo 2> /dev/null

; - command separator

# enter ~/cs370; display disk usage; return to previous dir
cd ~/cs370; du -h; cd -

# print 2 blank lines around output of fortune  
echo; echo; fortune; echo; echo

&& - conditional chaining; run the following command only if previous command completes successfully

# if string "jchung" is found in ~/.bashrc, cat ~/.bashrc  
grep jchung ~/.bashrc && cat ~/.bashrc

|| - conditional chaining; run the following command only if previous command did not complete successfully

# if string "JCHUNG" not found in ~/.bashrc, print error message
grep JCHUNG ~/.bashrc || echo "String not found."

( ) - grouping - commands within parentheses are executed in a sub-shell (or child-shell) of the current shell

(cd /; pwd)  # Execute commands in a sub-shell.
/            # This is output from ''pwd'' in the sub-shell.

pwd          # Execute ''pwd'' command in the current shell.
/home/jchung # This is output from the current shell ''pwd'' command.

\ - escape the following character (take it literally)

' ' - don't allow any special meaning to characters within single quotes

“ ” - allow variable and command substitution inside double quotes (does not disable $ and \ within the string)

   
Command:   echo The \$SHELL env. var. contains \"$SHELL\".
Output:    The $SHELL env. var. contains "/bin/bash".

Command:   echo The '$SHELL' env. var. contains \"$SHELL\".
Output:    The $SHELL env. var. contains "/bin/bash".

Command:   touch \'  # Creates a file called '
Command:   rm \'     # Deletes the file called '

• Command substitution can be a useful technique, used for the “in-line” usage of a command's output.

• In command substitution, the output of a command or pipeline of commands can be used to initialize variables
  • The output can also be used anywhere a command argument or variable might be used.

# Take the output of command and substitute it into the command line
`command` # Note the backward quotes (back-ticks).

# Run the uname command, which outputs Linux
uname
Linux

# Run the `uname` command in the shell; the shell
# attempts to run a nonexistent command called Linux.
`uname`
-bash: Linux: command not found

# Employing command substitution correctly, the output of 
# `uname` gets included in the output of echo.
echo "Hey, I'm using `uname`!"
Hey, I'm using Linux!

• In ksh, bash and zsh, command substitution can also be done with $(command)
  • This makes command substitutions more natural to use as pseudo-variables.

# Use $() notation for command substitution.
echo "Hey, I'm using $(uname)!"
Hey, I'm using Linux!
  
# Edit a file that contains the date (MMDDYY format) embedded in its name;
# uses the 'date' command output; See the 'date' man page.
nano /tmp/activitylog-$(date +%m%d%y)

# Copy every .wav audio file on the system to /tmp, with (v)erbose cp output;
# uses output of the 'locate' command.
cp -v $(locate .wav) /tmp

# Play every .wav audio file on the system. (Control-C to stop)
aplay $(locate .wav)

• (Do in class)

• At the command line, create your own bin directory (~/cs370/bin or ~/se370/bin).

mkdir ~/cs370/bin   # or mkdir ~/se370/bin

• Create a symbolic link (a kind of shortcut) called ~/bin that points to your class directory's ~/bin directory.

ln -s  ~/cs370/bin  ~/bin

• We want our shell scripts to go in ~/bin, our personal bin directory.

• Verify that ~/bin is part of system's $PATH environment variable

# See if ~/bin is in the $PATH environment variable (It should be).
echo $PATH

• At their simplest, text files that contain lists of commands
  • The Bourne shell (sh) supported shell scripts and helped reduce the burden of repetitive typing of long commands for human operators.
• Text files that contain the shell scripting programming language
  • Can vary in sophistication
  • Advances in csh, ksh and bash brought about a dynamic, interpreted, procedural programming language that is especially important in system programming, OS development, system administration, DevOps and cloud ops.
  • Difference between shell scripting and other interpreted scripting languages like Perl, Python, Ruby
    • In shell scripting, your library of functions is the large collection of UNIX commands that you use together to perform tasks.
• Shell scripts are interpreted by the shell.
  • As Perl scripts are interpreted by the Perl interpreter, Python scripts are interpreted by the Python interpreter, etc.
  • The specific shell interpreter can be specified or just left to the default system shell to interpret.
    • The default system shell (/bin/sh) is typically bash or other shell that is Bourne shell compatible.
• Shell scripts often are given a .sh file extension to distinguish them from other files.

• Our first shell script: ~/.bashrc
  • Being a config file, it's not a true shell script, but everything in ~/.bashrc is also found in shell scripts.

• Contain commands that you would normally type at the command line
• Not much different from aliases.
• (Do in class) Create the ~/bin/mycal.sh script.

# Change to your ~/bin directory
cd ~/bin

# Edit and save a new text file, mycal.sh
# See https://cssegit.monmouth.edu/jchung/csse370repo/-/blob/main/scripts/mycal 
# for the contents of mycal.sh.
nano mycal.sh

# Make mycal.sh executable (a runnable program)
chmod +x mycal.sh

# Run mycal.sh in the current directory (./)
./mycal.sh

# Run mycal.sh from your home directory. Since your ~/bin directory is in $PATH,
# you should be able to run your scripts from any directory.
cd
mycal.sh

#!/bin/bash
#
# The above line means:
# /bin/bash will be used to interpret this shell script, if /bin/bash exists.
#
# Shell scripts should specify a shell interpreter on the first line.
#
# If the #!/bin/bash line is omitted from the script, either
#
#    1 - the script will run using the current interactive shell (bash)
# or
#    2 - the system shell (/bin/sh) will be used as the shell
#        command interpreter by default.

• Any non-built-in command that you run in a shell will run in its own sub-shell.
  • See the note on “() - grouping” from command chaining.
• (Do in class) This can present problems for some shell scripts, like the following:

#!/bin/bash

# mkcd.sh: Create a dir and change to it immediately.

# Usage: mkcd.sh <dirname>

mkdir -p $1 # $1 is the script's first command line argument 
cd $1

• The mkcd script takes a directory name as a command line argument ($1), creates the directory and changes (cd) to the directory.
  • That's fine, except the cd part of the script will take place in a sub-shell. So, after running the mkcd.sh script, we are still in the same directory we started in.
  • The solution is to source the mkcd.sh script.
    • The source shell built-in command runs scripts inside the current shell, not in a sub-shell.

# Create ~/cs370/projects/1 directory and cd to it.
source mkcd.sh ~/cs370/projects/1

• An alias added to ~/.bashrc makes this command more natural to use.
  • Make sure you've created the mkcd.sh script in your ~/bin directory.

alias mkcd='source ~/bin/mkcd.sh'