cs471:cs_471_-_email_networked_file_systems_system_file_sharing
Table of Contents
Email, Networked File Systems, System File Sharing
Electronic Mail
- An essential service
- Email jargon: mail user agents (MUAs), mail transport agents (MTAs), retrieval agents, delivery agents
Mail user agents (MUA)
- MUAs used by users to read and send email
- When run, either bring the user's mail from the system mail spool (e.g. /var/mail/username on UNIX systems) and put it into a mailbox in the user's home directory or ~/Mail directory ..
- .. or read the user's mail in place (/var/mail/username in UNIX).
- Different MUAs can store mail in ~/Mail in different mailbox formats.
/bin/mail
- The original mail agent.
- Often used to send messages noninteractively.
/usr/ucb/mail, mailx
- The Berkeley mail agent.
- Also often used to send messages noninteractively, but with the addition of a “Subject:” header.
Other UNIX mailers
- SUN mailtool
- elm
- pine
- mutt
- MH/xmh
- VM Emacs mode
- X Window system mailers: kmail (KDE), evolution (GNOME), mozilla-thunderbird, et al.
Non-UNIX mailers
- Outlook, mozilla-thunderbird, Web-based mail and many more
- Rarely do users use MUAs on the same server that their mail is spooled on these days.
- In that case, the MUAs that rely on local mail spool files need a retrieval agent to get the mail from the mail server.
- mail, mailx, elm, pine (?), mailtool, mh, VM rely on a retrieval agent for remote mail.
- See “Protocols for managing mail on remote servers” below.
Mail transport agents (MTA)
- The MTA is responsible for actually sending, receiving and storing messages. It should:
- Accept messages from the user agent (MUA)
- Understand destination addresses
- Deliver mail originating on the local machine to the proper mailbox[es] if local, or to a remote MTA if remote
- Receive incoming mail from other MTAs and store for each known user account
Process of sending email from a MTA's standpoint
- When a MUA such as mail tries to send a message it issues a request to the MTA, which processes the mail with the specified options.
- The MTA creates a list of recipients from the information and expands any aliases, including mailing lists.
- At this step syntax is checked and local addresses are verified.
- Duplicate recipients are removed.
- If no addresses are valid the message is returned with an error message.
- The MTA then tries to deliver the message.
- If it can't deliver the message immediately it stores the header and body of the message in temporary files in a queue directory (e.g. /var/spool/mqueue) and tries to send it again later.
- MTA protocol: SMTP, stands for Simple Mail Transfer Protocol, and is the protocol used for Internet mail
- requires an entry in /etc/services, i.e.:
smtp 25/tcp mail
- The most widely-used MTA on UNIX systems is sendmail.
- Configuration of sendmail (/etc/mail/sendmail.cf on rockhopper)
- Other MTAs on UNIX systems include: Exim (default on Debian), Qmail, Postfix
- Microsoft Exchange is the de-facto choice on Windows networks.
Filters and delivery agents
- can work with MTAs
- often called by an MTA before it delivers mail to user mailboxes
- may filter mail for spam, viruses
- example: spamassassin (http://spamassassin.apache.org/)
- example: clamav (http://www.clamav.net/)
- can also work with MUAs and retrieval agents
- i.e., can be called by MUAs or retrieval agents
- used to filter or categorize the daily incoming flood of mail
- example: procmail (http://www.procmail.org/)
Protocols for managing mail on remote servers
- (Repeat) Rarely do users use MUAs on the same server that their mail is spooled on these days.
- In that case, the MUAs need a way to communicate with the remote mail server
- 1) to download and store the spooled mail or
- 2) to manage the remote mail spool.
- MUAs can have built-in capability to connect to the remote mail server or depend on a retrieval agent (a remote mail downloader) such as fetchmail (http://freshmeat.net/projects/fetchmail)
- In either case, the protocol used to connect to the remote mail server is likely to be either POP or IMAP (or in Microsoft-Exchange-Outlook, ISAPI)
POP
- Post Office Protocol
- Most used version is POP, version 3 (POP3, http://www.faqs.org/rfcs/rfc1939.html).
- Both mail headers and mail contents are downloaded and stored locally, and the mail deleted from the server (usually).
IMAP
- Internet Message Access Protocol
- Usually only the mail headers are transferred and the mail stays stored on the server.
- POP and IMAP run as services (daemons) on the mail server
- See /etc/services.
- You can have both at the same time.
- examples: qpopper (http://www.eudora.com/qpopper), uw-imapd (http://www.washington.edu/imap)
Ethical and other administration issues involving email
- root on an MTA system can see everyone's email
- Email–alongside printing–can be the most difficult and frustrating part of administration (personal experience).
- Configuring and troubleshooting MTAs is complex and difficult.
- Windows: Everyone uses the same MUA (Outlook) and MTA (Exchange), which is both good and bad.
- UNIX: No dominant MUA, user preferences vary widely.
- End up configuring and troubleshooting not only the MUA but maybe fetchmail and procmail.
Networked file systems (NFS and others)
- Also known as “distributed file systems”
- Allows using remote file systems over the network as if they were locally attached file systems.
- Client-server model
- The client machines do not have direct access to the underlying block storage devices on the server but interact over the network using a distributed file system protocol.
- Uses
- Central file server
- Roaming remote access to files
- Easier to backup
- Application server
- Make software centrally available
Network File System (NFS)
- Widely used in UNIX
- Windows NFS clients exist but are rare
- NFS “exporting” (server side)
- Server host must first be configured to allow your host to mount directories
- /etc/exports defines the directories that are exported and which hosts are allowed to mount them
- /var/lib/nfs/etab lists information about currently exported directories
- The 'rpc.mountd' processes mount requests
- SunOS 5.x (Solaris) uses /etc/dfs/dfstab in slightly different way
- 'exportfs' is used to make directories available for mounting
- SunOS 5.x (Solaris) systems use the share and unshare commands
- NFS mounting (client side)
- Your host (client) must then mount the remote directories
- showmount shows mountable file systems on remote machines
- The mount command is used to mount directories, e.g.
mount kinison:/var /mnt # kinison:/var is the remote NFS export on the host kinison # /mnt is the local mount point mount host.dept.washington.edu:/usr/users/home /export/home
- /etc/fstab contains the list of all file systems that are mounted at boot-time
- Permanent NFS mounts can be included in fstab
- Table of commonly used NFS mount flags
----------------------------------------------------------------------- Flag Description ----------------------------------------------------------------------- ro Mount the file system read-only rw Mount the file system read/write (only if server exports as writable) bg * If the server doesn't respond, keep trying the mount in the background - may also hang startup scripts, or cause strange failures, if they rely on files in these file systems hard * Blocks processes accessing files until server can respond (may look like your shell has "hung") soft Processes trying to access files get an error message if server is not responding intr * Allows user to abort a blocked access (returns error also) timeo= Timeout period (in tenths of a second) for requests retrans= Specify number of times to retry requests before return ing an error when soft mount used. -----------------------------------------------------------------------
Samba (http://www.samba.org)
- Allows UNIX filesystems and printers to be shared with Windows, UNIX and other systems
- Enables support for the Server Message Block (SMB) protocol, aka the Common Internet File System (CIFS), the native resource sharing protocol for Microsoft networks.
- over port 139 (see /etc/services)
- Example: mapping my UNIX home directory as a Windows network drive
- Relevant file/dirs on rockhopper:
/etc/samba/smb.conf (Samba config file) /usr/sbin/smbd (Samba daemon)
Other Networked File Systems
- AFS (OpenAFS, http://openafs.org)
- ssh-based remote fs tools
- fuse/sshfs project (http://fuse.sourceforge.net/sshfs.html)
- shfs (http://shfs.sourceforge.net/)
Sharing System Files
Why do it?
- From administrator's perspective:
- Only have to maintain one set of config files
UNIX: /etc: passwd, shadow, group, hosts, printcap, etc.
- From user's perspective:
- Same login on every type of computer on campus.
Several general methods of sharing system files
- Centralized (UNIX)
- “push” copies (server to multiple clients)
- use rdist or similar tool
- Write your own scripts using rcp/scp
- “pull” copies (multiple clients from server)
- Write your own scripts using rcp/scp
- Distributed
- Access a master server or backup server for each lookup
- Examples:
- NIS: Developed by Sun, UNIX-based
- Server has central copies of /etc/passwd, /etc/group, /etc/aliases, more
- NIS clients pull down copies from the NIS server(s) regularly.
- NIS server can also push updated copies to NIS clients.
- Hard to make secure
- Directory services
- software system that stores, organizes and provides access to information in a directory
- in software, directory = an organized map between names and values
- Lightweight Directory Access Protocol (LDAP)
- application protocol for reading and editing directories over a TCP/IP network
- LDAP-based directory services:
- Microsoft Active Directory
- OpenLDAP
- Provide basis for sharing user and group account information on the network, allowing for things like centralized logins
- User account information is no longer local (See rockhopper|cslin*:/etc/passwd)
- Name Service Switch facility on UNIX-like systems determines the source of system files/system info/user accounts
- See /etc/nsswitch.conf on Linux
Lab Activities (Do not do until instructed.)
1. The Postfix MTA
- Install, configure and test the Postfix MTA on Debian
- Install Postfix
- Debian will only allow one active MTA on the system, so when you install Postfix, the default Debian MTA, Exim, will be removed.
- As root, run apt-get install postfix
- Choose 'Internet with smarthost' for 'General type of configuration'.
- When asked 'Where should mail for root go?', enter root's normal user id.
- Use your fully qualified domain name for 'Mail name'. Example: belushi.monmouth.edu
- Accept the defaults for all remaining questions.
- Make note of the location of the Postfix config file, and the init.d script used to reload Postfix.
- Configure Postfix
- Edit the Postfix config file, /etc/postfix/main.cf.
- It should look similar to the following (your hostname will be different):
- (If you have to make a change in main.cf, reload Postfix.)
# See /usr/share/postfix/main.cf.dist for a commented, more complete # version smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h myhostname = localhost.localdomain alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases myorigin = /etc/mailname mydestination = belushi.monmouth.edu, localhost.localdomain, localhost relayhost = mail.monmouth.edu mynetworks = 127.0.0.0/8 mailbox_command = procmail -a "$EXTENSION" mailbox_size_limit = 0 recipient_delimiter = + inet_interfaces = all
- Test Postfix
- Telnet to port 25 on your machine: telnet localhost 25
- Note the information that is provided. Do not try to login. You won't be able to. Control-] and enter 'quit' to quit.
- Send a test email to using the mail command to jchung@monmouth.edu with main.cf as the message body, or as an attachment.
- Login as non-root. Use the mail command to send a test email to <yourself>@monmouth.edu
- See the mail aliases file, /etc/aliases.
2. Network File System (NFS)
- Install and use NFS.
- Use your VM as a NFS server and your container as a NFS client.
On the nfs server:
- Make sure that the nfs-common and nfs-kernel-server packages are installed.
- See examples in man exports, and edit the /etc/exports file so that the /usr/local/download directory can be mounted read-only from all machines in the *.monmouth.edu sub-domain, i.e.,
# CS-471 export (jchung, 3/2018) /usr/local/download *.monmouth.edu(ro)
- Once /etc/exports is edited, run exportfs -a
- Verify that the directory that you specify in /etc/exports is in the file /var/lib/nfs/etab
- Running 'exportfs' without ' -a' shows the directories that the nfs server is currently exporting.
- If exportfs does not work, just restart the nfs-kernel-server using the appropriate script in /etc/init.d.
On the nfs client:
- Make sure that the nfs-common package is installed.
- See what nfs exports are available on your nfs server by running:
sudo showmount -e <nfs_server_hostname>//
- Create the /mnt/nfs directory
- Mount the /usr/local/download directory of your nfs server on the /mnt/nfs mount point.
sudo mount -t nfs <server hostname>:/usr/local/download /mnt/nfs
- Verify that the nfs server's /usr/local/download directory has been mounted and that it is read-only.
- Unmount the remote nfs export
sudo umount /mnt/nfs
- Using the lab machine's /etc/fstab as an example, create a fstab entry that will mount the /usr/local/download directory of your nfs server on the /mnt/nfs mount point of your client automatically.
- Test and verify that the new entry in /etc/fstab works by running
sudo mount /mnt/nfs
3. The Samba Network File System
- Install the samba package on your Linux VM only, if you have not already done so.
- See /etc/samba/smb.conf using an editor.
- In /etc/samba/smb.conf, make sure that you have something like the following lines to define user home directory shares (they should already be there by default):
[homes] comment = Home Directories browseable = No
- Set the Samba password for a known, non-root user on your Linux systems (e.g., jchung):
(as root) # smbpasswd -a jchung
- Samba passwords are stored separately from system passwords in /var/lib/samba/private/passdb.tdb.
- You should now be able to map a user's home directory share on your Linux systems as a Windows network drive.
- With user jchung on the Linux host belushi.monmouth.edu, I would enter the following share when attempting to map Windows drive Y: to the Samba [homes] share on belushi:
\\belushi.monmouth.edu\jchung (when prompted, username would be 'belushi\jchung', and password would be the password specified using 'smbpasswd -a jchung')
- This can also be done from the Windows command line (Run cmd.exe), for example:
net use y: \\belushi\jchung /user:belushi\jchung
cs471/cs_471_-_email_networked_file_systems_system_file_sharing.txt · Last modified: 2018/04/20 20:21 by jchung