• See General SysAdmin Principles on Startup/Shutdown

• The system administrator (SA) is often required to maintain multiple systems and may have different login IDs on those systems.
  • The systems themselves may be different in terms of operating system, architecture, etc.
  • The SA has to be aware of Who, What and Where.

• Common mistake: Rebooting the wrong machine (after starting a remote root login).

• Figuring out Who

        $ whoami

        $ w

        $ who

        $ finger (if system configured for it)

• Figuring out What

        $ uname -a          # What OS is this? For what architecture? What kernel version?

        $ cat /proc/cpuinfo # What CPU(s)?

        $ dmesg             # Hardware & driver information

        $ cat /etc/fedora-release
        $ cat /etc/debian_version
        # What version of a Linux distribution is this? redhat, fedora, debian, etc?

• Figuring out Where

         $ uname -n         # Which host am I logged into?

         $ cat /etc/hosts   # Local map of what IP addresses are mapped to what hostnames

         $ cat /etc/hostname*   # local hostname; may differ from hostname on domain name server
         $ hostname

         $ cat /etc/dhcp/dhclient*   # may contain hostname request to domain name server

         $ /sbin/ifconfig       # What's my IP address?
         C:\> ipconfig          # Windows
         
         $ ip addr              # ifconfig considered deprecated; use ip instead

         $ host <ip address from ifconfig output> # What's my hostname on the name server?
         $ nslookup <ip address from ifconfig output>

         $ pwd                  # present working directory

• Shell prompts on Linux:

         jchung@rockhopper:/usr/local$        # Tells me who, where & what

• Typically, on modern UNIX and Linux systems, if you're root, your shell prompt becomes #.
  • For a normal user, the shell prompt is $.

• See Analyzing the Linux boot process.
• Also, see some boot charts.

• ROM monitor (BIOS on PCs, OpenBoot PROM on SUNs, etc.) runs some simple hardware diagnostics and finds the boot device
  • the boot device is set in the ROM monitor and can also be manually chosen
    • may be a hard drive, floppy, cd-rom, network, usb, etc.
• ROM monitor loads/runs bootloader (LILO/GRUB on Linux PCs)
  • bootstrap loader resides on the Master Boot Record (MBR) of a physical boot device
    • MBR is the first 512 byte segment on the disk (on PCs)
• Bootloader loads/runs the kernel (PID 0 on UNIX)
  • Kernel may perform more elaborate diagnostics, then checks file systems
  • Kernel mounts file systems, then starts the system (init, PID 1 on UNIX)
    • /etc/inittab: config file that defines important init behavior
  • Initialization scripts may be run (rc scripts on UNIX) to start system services
  • A login process produces login prompts on terminals (or GUI system starts up), and system is ready to go.
    • getty, xdm, gdm, kdm, dtlogin (UNIX)
    • Some of these processes can be identified using ps. Try ps fax on your machines.

• security issues
• by default, not secured from physical access
• BIOS/Bootloader passwords add another layer of security
  • BIOS passwords prevent booting from other media or network
  • Bootloader passwords prevent easy booting to a root shell

• BSD single-user mode vs. multi-user mode.
  • Berkeley UNIX has just two operating states: single-user mode and multi-user mode.
    • Interesting, Debian currently also has effectively only two modes.

• System V run levels.
  • System V has several states, or “run levels”
  • Use 'who -r' or 'runlevel' to see current run level

   cslab00: /export/home/uf/jchung $
   who -r
      .       run-level 3  Jan 21 14:50     3      0  S

   jchung@rockhopper:~$ who -r
      run-level 5  2009-01-07 00:10                   last=3



   Default run level defined in /etc/inittab on Sys V / Linux:

   Table - System V Run Levels
   -------------------------------------------------------------------------------------------
   Run Level  Names and Uses
   -------------------------------------------------------------------------------------------
   0          Powerdown state; conditions where it is safe to turn the power off.
   1          Administrative state.
   s or S     Single-user mode.
   2          Multi-user mode; Normal operating state for isolated,
              non-networked systems.
   3          Remote File Sharing (RFS) state. Alternative state for networking use. (Normal
              multi-user state for OSF/1; other Sys V may use state 2).
   4          User-definable state. (Also not normally used; state 2 used instead).
   5          Firmware state; Used for some types of maintenance on some systems.
   6          Shutdown and reboot state; used to reboot the system from some other running
              state (s, 2, 3 or 4) to state 0 then immediately back to normal operating state.
   -------------------------------------------------------------------------------------------

• The initialization scripts, usually written in Borne Shell (BSD, System V) or Bourne Shell derivative like bash
  • Sets computer's hostname (BSD and some System V)
  • Sets time zone (System V only)
  • Checks unmounted discs with fsck (only in multi-user mode)
    • After “unclean” shutdown due to power outtage, crash/panic
  • Mounts the system's disc partitions
  • Removes files from the /tmp directory
  • Starts up daemons and network services
  • Turns on accounting and quotas (if used)
  • Configures network interface(s)
  • Starts windowing system and graphical login

• BSD rc files
  • /etc/rc (multi-user initialization)
  • /etc/rc.local (site-specific initialization)
    • also present in Linux
  • /etc/rc.boot (NeXTStep single-user initialization and primary SunOS initialization file)
    • /etc/rc.single (under SunOS single-user mode starts shell; under SunOS multi-user mode runs /etc/rc)
    • /etc/rc.net, /etc/rc.tcpip, /etc/rc.pci,/etc/rc.ncs (AIX)

• System V initialization files
  • init reads /etc/inittab to find what to execute
  • Initialization files (executable scripts) stored in /etc/init.d
  • Symbolic links from /etc/init.d are made into /etc/rc0.d, /etc/rc2.d, etc, where the X in rcX.d corresponds to a runlevel number.
    • Symlinked scripts that begin with 'S' are started or killed if they begin with 'K'
      • Example: See /etc/init.d/ssh, /etc/rc2.d .

• System V initialization
  • The following diagram shows the process of initialization under System V style Unix:

• Should avoid just turning the computer off.
  • Often results in file system damage as most modern file systems cache their writes in volatile memory.
    • Raspberry pi and other embedded systems running Linux/UNIX are often susceptible because users tend to just pull the plug without properly shutting down.
  • The file system buffers should be flushed before cutting the power, if at all possible.
    • The sync command does this in UNIX.
• Shutdown of production servers cannot be done frivolously.
  • Systems on which system uptime is important
    • ssh opus uptime
  • Inconveniences users
  • Rebooting should not be seen as a cure-all for system problems
  • May be overkill if all you wanted to do was restart a system service after changing a service configuration
    • After a change to a service configuration, such as SSH, run '/etc/init.d/ssh restart' instead of rebooting.
  • If system seems to be “wedged”, “stuck”, or “crashed”, sometimes the cause is a blocking or runaway processes that can be killed by remotely logging in, if remote login is possible.

• Need to add new hardware that cannot be “hot-plugged”
• System is really “wedged”, “stuck” or “crashed”
  • UNIX: output of dmesg shows kernel related errors
• Kernel panic
  • System will auto-halt to avoid further damage from corrupt data being written to disk
  • blue screen of death (Windows)
• Upgraded to new version of kernel
• Before shutting down or rebooting multi-user systems…
  • Notify logged in users (may use wall command in UNIX)
  • Disable logins (may need to touch /etc/nologin in UNIX)
  • Enable a shutdown message

             # shutdown -t 300 "Shutting down for weekly backups in 5 minutes..."
             # shutdown now "There's water coming through the ceiling. Shutting down now."

• Hardware failures
• Power failures/surges
• Environmental problems
  • thermal failure
• I/O problems (including disk or partition full)
• Software problems (kernel faults, bugs, programs)
• Malicious users/intruders

• Same causes as given above for system crashes
• Errors in initialization scripts
  • One mistake in /etc can sink a system (UNIX)
• Improperly configured kernel (device driver missing or bad)
• Improperly partitioned hard drive
• Improperly configured bootloader

• Interactive
• Batch (scheduled processes, e.g., cron, at, batch on UNIX)
• Daemons
  • background processes
• fork()/exec()
  • fork'ed processes are child processes spawned by a running process
  • exec'ed processes replace the processes that spawn them

• daemons are processes that run continuously in the background
  • suitable for providing various networked and non-networked OS services
    • listen for incoming requests on network “ports”
    • see /etc/services file (UNIX)
  • typically have names ending in “d”
    • sshd, telnetd, ftpd, httpd, lpd, cupsd, talkd, inetd, crond
    • Daemons are sometimes just called “servers”, e.g. “the ssh server”.
  • can be child processes of init (fork'ed by init)
    • see output of 'ps fax' for process tree and identify daemons (Linux)

1. Install package upgrades on your virtual Linux system(s).
   • Create a root cron job to download package upgrades nightly.
2. Install ntp and keep your system time in sync with clock.monmouth.edu.
   • ntp may already have been installed.
   • Run date on your Linux lab host (csselin02-csselin13) and also on your virtual Linux systems. Are the dates/times significantly different?
   • Install the package ntp using apt-get install ntp
   • ntp is now a new service on your Linux systems.
   • What init script has been added to /etc/init.d for ntp?
   • How can you tell that the ntp package has been installed?
   • How can you tell that the ntp service is running?
   • How can you tell that the ntp service is set to start at system boot up and set to stop at system shutdown/reboot?