User Tools

Site Tools


cs370:cs_370_-_introduction_unix_fundamentals

Introduction, UNIX Fundamentals


Accessing Your UNIX Account

  • HH 310 (Linux/Mac Lab), HH 305 (Mac Lab)
  • Remote access via Internet or LAN
    • UNIX servers (Linux)
      • rockhopper.monmouth.edu
      • plato.monmouth.edu
    • UNIX lab machines (Linux)
    • Terminal access via SSH
      • PuTTY telnet/ssh client for Windows
    • Graphical desktop access

A Bit of UNIX History

UNIX "standards": SysV and BSD

  • AT&T's Bell Labs creates UNIX
    • Evolves into “System V” UNIX
  • Univ. of California-Berkeley takes AT&T's UNIX and develops BSD (Berkeley Standard Distribution)
  • Today, proprietary versions of UNIX and Linux contain some of both UNIX “standards”
  • Some differences in utilities
    • lpr (BSD) and lp (SysV) for printing
    • different options for utilities like ps
    • Some UNIX variants include both kinds of utilities

Some UNIX Features

  • Multi-user
    • Allows multiple users to access a system at the same time
  • Allows processes and peripherals to talk to each other, even on different machines
  • pipes and sockets
  • Large number of standard command-line utilities
    • for text editing and text processing, compiling, file processing
    • the focus of much of this course because using these utilities together is the core of the UNIX philosophy
      • even in these days of advanced or dumbed-down GUIs and GUI applications
  • An “open” system
    • allows programmers to easily access OS features via system calls
  • A portable OS
    • Relatively easy to compile and run on a wide variety of platforms because it's mostly written in C
      • A contributing reason for so many proprietary variants of UNIX that were developed: Solaris, Irix, HP-UX, AIX, Tru64, Mac OS X…
      • A contributing reason for free UNIX variants that have been available on so many different platforms: Intel, PowerPC, Alpha, MIPS, ARM (Android), Playstation …
  • Everything is (or should be) a file, for example …
    • System processes are listed as pseudo files in /proc
    • System devices are accessed as special files in /dev
  • Everything is (or should be) text.
    • The core UNIX utilities were written to handle text streams allowing them to easily pass data between them.

UNIX File System Hierachy

  • No drive letters, e.g. C:, D:, etc.
  • File systems on hard drive partitions are mounted and appear as directories under the root “/” directory
    • Also goes for remote file systems…
    • … and removable media drives, e.g. floppy drives, CD-ROM, USB pen drives
            $ ls /

            bin/        df.txt   initrd/        media/  root/     sys/       var/
            boot/       etc/     lib/           mnt/    sbin/     tftpboot/
            denyhosts/  export/  linux-images/  opt/    selinux/  tmp/
            dev/        home/    lost+found/    proc/   srv/      usr/



            $ df
            Filesystem           1K-blocks      Used Available Use% Mounted on
            /dev/sda1               522028    404048    117980  78% /
            none                   1557628        48   1557580   1% /dev/shm
            /dev/sda2             10241088   8681048   1560040  85% /usr
            /dev/sda3               819232    639592    179640  79% /var
            /dev/sda5             21823584  20355644   1467940  94% /linux-images
            ...
            ...
            netapp:/home/cslab   286703264 129809216 156894048  46% /export/home

File System Organization

  • varies between different variants of UNIX, but usually…
  • Files and directories associated with applications and programs are usually under usr
    • bin directories contain executable programs and scripts
    • lib directories contain libraries that programs link with
    • include directories contain program header files that correspond with the libraries in lib
    • man directories contain manual pages used with the man command
    • doc directories contain package documentation
      • /usr/share/doc is the usual location for documentation
    • share directories contain other application data such as fonts, pixmaps
    • var directories contain application generated data such as database files, log files
    • tmp directories contain temporary, transient directories and files
      • /tmp (tmp in the root directory) is usually readable/writeable for every user and process
    • etc directories contain application and system configuration files
      • /etc (etc in the root directory) contains most system and application configuration files.
        • the configuration files, in the UNIX tradition, are almost always plain text files
          • …and many have names that end with .conf
                   $ find /etc -name '*.conf' | less
  • Another set of bin, lib, include, man, doc, share, var, etc … directories is under /usr/local

Program organization

  • A large enough program is split into the various directories mentioned above
    • whereis will usually tell you how the program is split up
         $ whereis geany
         geany: /usr/bin/geany /usr/include/geany /usr/share/geany /usr/share/man/man1/geany.1.gz

The UNIX Philosophy or Style

Avoid large, monolithic programs

  • A program should do one thing and do it well
    • Small programs should be combined to accomplish more complex tasks
    • Use “pipes,” which allow output of one process to be used as input (“piped”) to another process
      • …without the need for intermediate temporary files
  • Examples
# upside down cal
cal | tac
  
# number of processes that are associated with me
ps aux | grep jchung | wc -l
  
# ??????
wget -q -O - https://www.gutenberg.org/cache/epub/11231/pg11231.txt | sed 's/ /\n/g' | sed 's/[^A-Za-z0-9]*$//g' | sed 's/^[^A-Za-z0-9]*//g' | tr A-Z a-z | sort | uniq -c | sort -rn

Don't reinvent the wheel

  • The UNIX approach to problem solving
    • If you can solve the problem by combining multiple existing utilities using pipes, do it; otherwise
    • …ask people on the network if they know how to solve it. If they do, great; otherwise
    • …if you could solve the problem with the aid of some other hand-written utilities, write the utilities yourself and add them to the UNIX repertoire.
      • Each utility you write should do one thing and do it well so that it can be reused to solve other problems.
    • If more utilities won't do the trick, then write your own program to solve the problem.

Existing utilities

UNIX Shell Introduction

  • The “command line” or “command line interpreter”
  • Relationship between the shell and “Terminals”
    • The Terminal is not the shell.
      • The shell runs in the Terminal to interpret and execute commands and display output.
      • The shell can run invisibly outside of any Terminal.

Environment variables

  • Environment variables are everywhere in UNIX.
    • Used by running processes and in shells
    • Run env | less # (or set | less)
      • Look for value of the SHELL environment variable
        • env | grep SHELL
        • echo $SHELL
          • In the shell, all variables are accessed by prepending the $.

Shell config files

  • system-wide shell config: /etc/profile
  • user-specific shell config
    • ~/.profile (for ksh and bash)
    • ~/.bashrc, ~/.bash_profile (for bash)
      • ~ (the tilde character) is shorthand for $HOME
        • See output of env | grep HOME or echo $HOME
          • … so ~/.bash_profile for me is the same as /export/home/hawkdom2/jchung/.bash_profile
          • ls ~ or ls $HOME are equivalent
  • When you start an interactive shell, your shell reads its shell config file.
  • Shell configs set env vars, define functions, aliases, etc. and run programs.
  • The Shell prompt
    • PS1 shell variable

Shell command history

  • scrolling
    • Use up and down arrow keys to scroll the command history.
  • searching
    • history command lists previously issued commands
    • history | grep cat searches for cat in the command history
  • shell history files
    • (Bash) Command history written to ~/.bash_history when user exits
      • Number of commands kept in command history and command history file set w/ HISTSIZE and HISTFILESIZE environment variables, respectively.
        • Default value of HISTFILESIZE is 500
        • HISTSIZE and HISTFILESIZE are set in shell configuration file
        • (Do in class) Change HISTSIZE and HISTFILESIZE to 5000 to increase command line history.
          • This requires editing your ~/.bashrc bash config file.
            • Always include a comment with your config file edits.
          • Note: This is a participation exercise.
  • Reissuing previous commands
    • previous commands can be issued again with “!” directive
# Display your command history
history
   
...
   
548  cat .bash_history
549  man bash
550  nano .bashrc
551  source .bashrc
   
!549       # issues 'man bash' command again
!man       # reissues last command starting w/ 'man'
!nan       # reissues the 'nano .bashrc' command

Shell command and filename completion

  • Type first few letters of a command or file name and hit Tab for completion.

Shell command aliases

  • typically set in shell profile so they are available for every interactive shell session
    • type alias to list aliases
      • setting an alias
        • alias commandalias='command(s) plus options'
  • (Do in class) Add at least 3 useful aliases to your bash shell profile (~/.bashrc).
    • Include a comment.
    • Note: This is a participation exercise.

Intro to Basic UNIX Utilities

  • Before proceeding, see the collection setup notes.
    • Create an examples/cat directory under your course directory (cs370 or se370).

Getting help: man pages

  • Searching for apropriate man pages
    • Use man -k (or the equivalent apropos command)
man -k vim   # Search for all man pages containing keyword 'vim'
            
eview (1)            - easy Vim, edit a file with Vim and setup for modeless editing
evim (1)             - easy Vim, edit a file with Vim and setup for modeless editing
gvim (1)             - Vi IMproved, a programmers text editor
gvimdiff (1)         - edit two, three or four versions of a file with Vim and show differences
gvimtutor (1)        - the Vim tutor
rgvim (1)            - Vi IMproved, a programmers text editor
rvim (1)             - Vi IMproved, a programmers text editor
vim (1)              - Vi IMproved, a programmers text editor
vimdiff (1)          - edit two, three or four versions of a file with Vim and show differences
vimtutor (1)         - the Vim tutor
  • Navigating in man pages
    • 1G - move to top of man page, G - move to bottom of man page
    • u, b or <Page Up> to page up
    • d, space or <Page Down> to page down
    • q to quit
  • Searching in man pages
    • Type '/' then the word or phrase you are searching for.
      • 'n' and 'p' to search for next and previous instances of search string, respectively

The cat utility

  • cat (short for catenate) takes input from standard input or from a list of files and sends them to standard output
    • Access contents of a file w/ cat
cat filename
  • Creating a simple text file w/ cat
cat > fileFromCat.txt # The '>' means that keyboard input is 'redirected' to fileFromCat.txt
The first line
second line
third line
EOF          # EOF is usually entered using Control-D
   
cat fileFromCat.txt
   
The first line
second line
third line
  • Concatenating a series of files w/ cat
# output contents of four files to screen
$ cat file1 file2 file3 file4

# output to a file, overwriting contents of allFilesTogether   
$ cat file1 file2 file3 file4 > allFilesTogether

# output to a file, appending to contents of allFilesTogether
$ cat file1 file2 file3 file4 >> allFilesTogether

# using a shell wildcard (*),
# cat the contents of all files whose names begin w/ 'file'; files are
# concatenated in sorted order, i.e. file1 file2 file3 file4
$ cat file*

Processes and jobs

  • Shell facilities for job control
    • suspend, bring job to foreground, put job in background
    • suspend with Control-Z
      • Start the nano editor and then Control-T, Control-Z
      • Control-T, Control-Z suspends nano
        • The nano process is suspended
      • Type the fg command (fg = foreground).
        • nano is brought to the foreground
      • Control-Z again
      • Type the bg command (bg = background).
        • The nano process resumes running in the background.
      • Start man nano and Control-Z to suspend it.
      • Type the jobs command to list the two jobs currently running.
      • Resume nano with fg %1
  • ps command and jobs
    • The ps command displays process IDs (PIDs)
      • The PID and jobs number are not the same
    • ps aux or ps fax show all processes (Linux)
      • ps options depend on SysV or BSD implementation of ps
        • On SysV-type systems, ps -ef displays all processes
        • On BSD-type systems, ps -aux displays all processes
  • The top command shows top processes on system in “real time”

File management

  • The ./.. shortcuts
    • . means current directory
    • .. means parent directory
  • mkdir -p dirname
    • “-p” option of mkdir creates entire subdirectory tree
    • Example: mkdir -p cs370/examples/cat
  • rmdir vs. rm -r dirname
    • rmdir is rarely used because it cannot remove non-empty directories
    • The more commonly used rm -r dirname must be used with care.
    • Example: rm -rf a_directory_name
  • simple shell file name globbing
    • * - wildcard
      • ls abc*: list files and directories that begin with “abc”
      • ls *abc*: list files and directories that contain “abc”
    • [ ] - subset
      • ls [a-z]*: list files and directories that begin with a lower case letter
  • file managers
    • mc: a 2-pane console file manager for power users
    • various GUI file managers

Permissions and security

  • ls -l to view permissions
  • chmod to change permissions of files or directories
    • directories must be executable to be accessible
  • chown to change ownership
    • Only useful to root (system admin)
  • groups to see what groups you are in
  • chgrp to change group ownership of files or directories
  • setfacl, getfacl to use access control lists
    • much more flexibility than standard chmod/chown/chgrp commands
  • Certain GUI file managers provide frontends to chmod/chown/chgrp/setfacl/getfacl

Printing

  • lpr and lp to print from commandline
  • lpq and lpstat to view queued print jobs
  • lprm to remove your own print jobs from print queues
  • enscript to convert text to Postscript for prettier output

Using the mouse

  • If using a GUI desktop environment in UNIX, copying and pasting of text can be done between Terminals and other applications using common clipboard-like functionality.
    • Right-clicking in a GUI Terminal may bring up a context menu which includes “copy” and “paste” options.
    • Keyboard shortcuts to copy and paste in Terminals are usually Control-Shift-C and Control-Shift-V, respectively.
  • UNIX desktops have an older copy-and-paste functionality that is commonly used with the mouse.
    • Select text by swiping with the left mouse button …
      • or double click a word to select it …
      • or triple click a line to select the entire line.
      • The selected text is automatically copied to a different clipboard area in memory.
    • Paste text by placing the mouse cursor at the insertion point and clicking the middle mouse button.

Secure Shell (ssh)

  • Ssh is the de facto method and protocol for logging into a console (terminal) session on a remote UNIX system.
  • The most common way to use ssh is to run
ssh some_host_name  # Example: ssh plato, ssh plato.monmouth.edu
  • Your user account must exist on the remote host or you must specify another existing account when ssh'ing, e.g.,
ssh s1100841@plato  # or ssh s1100841@plato.monmouth.edu (if off-campus)
  • (Do in class) Set up no-password, key authentication for ssh
    • This method uses public key encryption instead of passwords
    • The set up of ssh public key authentication requires the following:
      • Generate a public/private key pair if you don't already have one.
        • command: ssh-keygen
        • local files: ~/.ssh/id_rsa, ~/.ssh/id_rsa.pub
      • Copy the public key contents to authorized_keys on the remote UNIX system that you want to ssh to.
        • command: ssh-copy-id
        • remote file: ~/.ssh/authorized_keys
      • Note: This is a participation exercise.
  • For maximum convenience, most users generate ssh keys without a passphrase.
    • But the best practice for ssh key authentication is to generate the key pair using a strong passphrase, and then use ssh-agent to type in the passphrase for us when ssh'ing to remote systems.
  • Remote file transfer programs that also use the ssh protocol are scp (secure copy), sftp (secure ftp) and rsync.

cs370/cs_370_-_introduction_unix_fundamentals.txt · Last modified: 2024/05/22 21:26 by jchung

Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki