Serving with sendmail

[Ref: OpenBSD 3.6]

OpenBSD base install provides a functioning Mail Server from the http://www.sendmail.org project.

In this brief we will configure sendmail to process smtp requests. We will also install a daemon to process pop requests.

  1. Setup services to process the SMTP as well as forwarding Mail Queues
  2. Setup services for Processing POP

Process Queued Mail and Recieve incoming Mail

File: /etc/rc.conf enable/disable sendmail in this configuration file
File: /etc/rc launches sendmail using the switches specified in /etc/rc.conf

Sendmail is installed as standard part of the base installation of OpenBSD 3.6 and is configured for security, such that it will only process outgoing email and not accept incoming email. To enable processing of incoming email, you merely need to enable the features in the configuration file: /etc/rc.conf.local.

File: /etc/rc.conf.local

Add the below lines mentioned in the /etc/rc.conf file to your rc.conf.local file.

# For normal use: "-L sm-mta -bd -q30m", and note there is a cron job
sendmail_flags="-L sm-mta -C/etc/mail/localhost.cf -q30m"

And make the change as shown below (i.e. add the "-bd")

# For normal use: "-L sm-mta -bd -q30m", and note there is a cron job
sendmail_flags="-L sm-mta -C/etc/mail/localhost.cf -bd -q30m"

The line specifies switches (command line options) that /etc/rc will pass to sendmail during system startup. –bd tells sendmail to run as a daemon, running in the background, listening for and handling incoming SMTP connections. As a daemon, sendmail does a listen(2) on TCP port 25 for incoming SMTP messages. When another site connects to the listening daemon, the daemon performs a fork(2) and the child handles receipt of the incoming mail message. This is the preferred method for high use (smtp) servers.

The /etc/rc file is started at system boot and checks for the value of the above 'options' and if options exist the /etc/rc will execute sendmail with the given options.

Further Documentation ? The Sendmail Operations Manual has a number of suggested recommendations, scripts clearly described for starting sendmail as an smtp service.

In a LAN only mail setup the queuing may not be necessary. The Sendmail bat book suggests 1h is a good setting for most sites, and for sites with few users –q15m may be appropriate to ensure immediate delivery of mail.

Note: To complete the installation process, we need to take heed of the warning: and note there is a cron job from the /etc/rc.conf file.

Run the crontab utility as root to edit the cron jobs for the system and edit the following line referencing sendmail.

#minute hour mday month wday command
#
# sendmail clientmqueue runner
*/30 * * * * /usr/sbin/sendmail -L sm-msp-queue -Ac -q

We need to comment the sendmail referenced line, as exampled below.

#       */30 * * * * /usr/sbin/sendmail -L sm-msp-queue -Ac -q

We place a "#" hash at the beginning of the line to comment the line out, prevent its execution.

Testing smtp

[Ref: The Network People, Inc. Mail Server Testing ]

To simplify testing, we will perform the tests from on the SMTP server itself. You may need to test externally to verify behaviour with an active firewall or other systems between your SMTP (Sendmail) Server and the client.

The test procedure will be to test a few basic commands, writing myself a message.

$ telnet localhost smtp 
Trying ::1...
Connected to localhost.
Escape character is '^]'.
220 hostname.example.com ESMTP Sendmail 8.13.2/8.13.2; Sun, 20 Mar 2005 10:45:20 +1300 (TOT)
helo localhost
250 obsdgcc3.ants.to Hello samt@localhost.example.com [IPv6:::1], pleased to meet you
mail from: samt@localhost
250 2.1.0 samt@localhost... Sender ok
rcpt to: samt@localhost
250 2.1.5 samt@localhost... Recipient ok
data
354 Enter mail, end with "." on a line by itself
Subject: This is my subject line

I continue writing until I'm out of interesting things to say
which is not that far away

.
250 2.0.0 j2JLjKg3024237 Message accepted for delivery
quit
221 2.0.0 hostname.example.com closing connection
Connection closed by foreign host.
You have new mail in /var/mail/samt

Reading Mail through pop3 requests

[Openwall.com's popa3d]

The popa3d Post Office Protocol (POP3) server is included in OpenBSD 3.6 and according to the man pages, has been there since 3.0. So the basic software is included and all we need to do is enable it.

Now we need to configure the super server inetd to route pop requests to be handled by our installed pop daemon. To do this we make the following changes.

File: /etc/services - Make sure there exists a line specifying port address 110 as a pop3 service

pop3    110/tcp     # Post Office Protocol 3
pop3    110/udp

File: /etc/inetd.conf - Make sure you have a line specifying the pop services (pop3) and the responsibility for handling it is popa3d as shown in this example.

Change the line that is 'commented out'

#pop3           stream  tcp     nowait  root    /usr/sbin/popa3d        popa3d
#pop3           stream  tcp6    nowait  root    /usr/sbin/popa3d        popa3d

To read:

pop3            stream  tcp     nowait  root    /usr/sbin/popa3d        popa3d
pop3            stream  tcp6    nowait  root    /usr/sbin/popa3d        popa3d

After making the changes, force inetd to re-read its configuration file by sending it the hang up signal.

# kill –HUP `cat /var/run/inetd.pid` 

From the man page:

CAVEATS
POP3 authenticates using cleartext passwords.

Note the 'caveat' above, and you may want to review a more 'secure' pop3 server once you get things up and running and know the system works.

Testing the Pop3 Server

[Ref: The Network People, Inc. Mail Server Testing ]

To simplify testing, we will perform the tests on the POP server itself. You may need to test externally to verify behaviour with an active firewall or other systems between your POP Server and the client.

The test procedure will be to test a few basic commands to ensure POP is up and able to perform basic functions (like listing mail, deleting mail in your mailbox.)

$ telnet localhost pop3 
Trying ::1...
Connected to localhost.
Escape character is '^]'.
+OK
user samt
+OK
pass mypassword
+OK
list
+OK
1 553
dele 1 
+OK
list
+OK
.
quit
+OK
Connection closed by foreign host.

We now have a functioning smtp server (sendmail) and a functioning pop server (popa3d) Mail should be received and delivered on your mail-server.

pops and handling IMAP requests (alternative)

[package: courier-imap-3.0.5p1 or imap-uw-2004.352-plaintext]

Install the package courier-imap and follow the installation instructions. If you missed the instructions, then you can view them again by going to the directory /var/db/[package-name], e.g. /var/db/pkg/courier-imap-3.0.5p1 and looking at the file +DISPLAY

You could alternatively use the imap-uw release, and in this context we/I am using the plaintext flavor because it will not be made available externally (i.e. will only be used by a web mail client running on the host.)

Testing the IMAP Server

[Ref: How I test an imap server; The Network People, Inc. Mail Server Testing]

IF you are having problems, remember to review the log file /var/log/maillog for any error messages that might assist your diagnostics and if that isn't enough dig in with tcpdump.

Courier-Imap

$ telnet localhost imap
Trying ::1...
Connected to localhost.
Escape character is '^]'.
* OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT 
THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION] Courier-IMAP ready. 
Copyright 1998-2004 Double Precision, Inc. 
See COPYING for distribution information.
abc1 login samt mypassword 
abc1 OK LOGIN Ok.
abc2 select inbox
abc2 NO Unable to open this mailbox.
abc3 fetch2 body[text]
abc3 NO Error in IMAP command received by server.

uw-imap

$ telnet localhost imap
Trying ::1...
telnet: connect to address ::1: Connection refused
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
* OK [CAPABILITY IMAP4REV1 LITERAL+ SASL-IR LOGIN-REFERRALS STARTTLS AUTH=LOGIN] 
localhost.example.com IMAP4rev1 2004.352 at Mon, 21 Mar 2005 00:18:25 +1300 (TOT)
abc1 login samt mypassword 
abc1 OK [CAPABILITY IMAP4REV1 LITERAL+ IDLE NAMESPACE MAILBOX-REFERRALS BINARY 
UNSELECT SCAN SORT THREAD=REFERENCES THREAD=ORDEREDSUBJECT MULTIAPPEND] User samt authenticated
abc2 select inbox
* 1 EXISTS
* 2 EXISTS
* NO Mailbox vulnerable - directory /var/mail must have 1777 protection
* 0 RECENT
* OK [UIDVALIDITY 1111317531] UID validity status
* OK [UIDNEXT 3] Predicted next UID
* FLAGS (\Answered \Flagged \Deleted \Draft \Seen)
* OK [PERMANENTFLAGS (\* \Answered \Flagged \Deleted \Draft \Seen)] Permanent flags
* OK [UNSEEN 2] first unseen message in /home/samt/mbox
abc2 OK [READ-WRITE] SELECT completed
abc3 fetch 2 body[text]
* 2 FETCH (BODY[TEXT] {93}
I continue writing until I'm out of interesting things to say
which is not that far away  )
* 2 FETCH (FLAGS (\Seen))
abc3 OK FETCH completed
abc4 close
abc4 OK CLOSE completed
abc5 logout
* BYE hostname.example.com IMAP4rev1 server terminating connection
abc5 OK LOGOUT completed
Connection closed by foreign host.

If you get a 'connection refused' when attempting a connection, the configuration for inetd.conf is either wrong or the server not restarted.

Sendmail Configuration

The following are tweaks to the configuration files to get the system working smoothly on my configuration and may be helpful.

File:/etc/mail/sendmail.cf
File: /etc/mail/relay-domains List of domains for which you allow relaying through your server
File: /etc/hosts known hostnames
File: /usr/share/sendmail/cf/ Sample Configuration Files for Sendmail

Who Am I ?

Sometimes things may not be happening because you just can't tell who you are (as the mail server.) To find out who sendmail things it is serving as, use the following commands.

# sendmail -d0 < /dev/null
[ . . . stuff cut-out. . . ]
============ SYSTEM IDENTITY (after readcf) ============ 
(short domain name) $w = myhostname 
(canonical domain name) $j = myhostname.example.com 
(subdomain name) $m = example.com 
(node name) $k = myhostname 
======================================================== 
[ . . . stuff cut-out . . . ] 

The screen displays a number of useful information that may assist you in reviewing your assumptions about what settings have been given to sendmail.

slow start-up gethostbyname() blocks

When Sendmail starts it will try to determine the name of the server by using the gethostname and getbyhostname system call. You may also need to edit the /etc/hosts file for the reason specified below from the IMAP Toolkit FAQ

From the IMAP Toolkit FAQ

 Q: Why isn't it
showing the local host name as a fully-qualified domain name? 
Q: Why is the local host name in the From/Sender/Message-ID headers of
outgoing mail not coming out as a fully-qualified domain name? 

A: Your UNIX system is misconfigured. The entry for your system in
/etc/hosts must have the fully-qualified domain name first,
e.g. 
 105.69.1.234 bombastic.blurdybloop.com
bombastic

 A common mistake of novice system
administrators is to have the short name first, e.g. 
 105.69.1.234 bombastic
bombastic.blurdybloop.com 

or to omit the fully qualified domain name entirely, e.g. 
 105.69.1.234 bombastic 

If your system pauses for a length of time when sendmail is started, the above situation may be the solution for your problem.

Make sure we have a fully qualified domain associated with each ip address used by the server. With two network cards, OpenBSD install will only specify the domain name for one of the network ip addresses. [Discovered the same problem with Linux Mandrake 7.1 install]

File: /etc/hosts

 # The NIC for the external
connection is: aaa.bbb.ccc.ddd (a legitimate ip address from my ISP) 
# The NIC for the internal connection is: eee.fff.ggg.hhh (using
private ip address on local lan) 
aaa.bbb.ccc.ddd myhost.example.com myhost 
eee.fff.ggg.hhh myhost.example.com myhost 

Relaying access denied

[Ref: OpenBSD FAQ - 10.0 System Management -> 10.4 ... Relay Access Denied ...]

I want the server to service all smtp messages sent from users in my private-ip network, regardless of where on the network they are sending it from. To do this, we create the file /etc/mail/relay-domains and include a list of the host domains we will service. For example, if the server domain is example.com then including example.com in the /etc/mail/relay-hosts file lets me smtp messages from a remote client, so long as I specify my user information is @example.com

Create or Edit: /etc/mail/relay-domains

 example.com 

This, of course, is a serious security hole. Another approach is to allow by IP address (ie. local lan only) and use your firewall rules to block access by people saying their using your IP addresses coming in on your external network cards. But that's another story.

What we are doing above assumes the standard /etc/mail/sendmail.cf supplied with the OpenBSD 2.7 install. To verify the name of the file where we can place relay-domains use:

# cat /etc/mail/sendmail.cf | grep relay-domains

and the output would be something like this (if the the sendmail.cf supports relaying.)

 FR-o /etc/mail/relay-domains 

If you do not get something similar to the above, then you will have to create another sendmail.cf (cf. /usr/share/sendmail) Sendmail 8.9 changes the previous behaviour of Sendmail, where it now defaults to not forwarding e-mail to combat bulk-mailers/spammers. So, the sendmail.cf needs to include some support for relaying mail from other domains. Look at using

 define(confCR_FILE,`/etc/mail/relay-domains') 

General format for the /etc/mail/relay-domains file is:

.example.com #Allow relaying for/to any host in example.com
sub.example.com #Allow relaying for/to sub.example.com and any host in that domain
10.2 #Allow relaying from all hosts in the IP net 10.2.*.*

Simple Diagnostics

Although sendmail is hard to grasp, there is an abundance of documentation on how to use, configure, and validate your sendmail configuration. This abundant information is available online and in print. The following tidbits are some of the commands I've used to check my sendmail configurations, they are in no special order and in many cases not cleverly written. For a list of reference resoureces visit the Sendmail site http//www.sendmail.org

What's in the Queue

To get a listing of mail stuck in the queue (not yet delivered and for what reason) then you can use the "-bp" option of sendmail. For example:

# sendmail -bp
/var/spool/mqueue
(1 request)
----Q-ID---- --Size-- -----Q-Time-----------------Sender/Recipient------------
e6UBU7B0        2385    24 Mon Jul 31 00:30    anat@myserver.com (host map: lookup (externalserver.com): deferred)
                                                samt@externalserver.com 

Debugging & Verbose Mode

-v (verbose) -d (debug) are two modes which print extra diagnostic information on the screen while processing.

sendmail -v user-id < info-to-send 

The -v (verbose) option will display sendmail's activities in trying to send mail to user-id. I find this especially useful when trying to understand why sendmail fails trying to send to an external host. You can use a user-id on another machine to watch what sendmail's doing in trying to send your mail to that external host. For example:

sendmail -v user-id@remotehost.com < info-to-send

The above will display the sendmail session trying to connect to the SMTP port at remotehost.com. With the details of the connection you can see any rejection messages from the external host.

sendmail -d## user-id < info-to-send

The -d (debug) option (as it should) can generate a lot of information and it is up to you to select the level of information that is useful for isolating the cause of your problem. Further documentation should be available at http://www.sendmail.org otherwise I have seen the use of -d40 (limit output to information about the queue) -d0 (produce general debugging information, as shown in earlier example above.)

To get a comprehensive listing from sendmail of the current settings, use the -d option without qualifiers.

# sendmail -d < /dev/null

The screen output will display the compiled in options and some of the settings configured for sendmail. This is not a comprehensive list of settings.

Looking Up MX records

[Ref: dig(1)]

sendmail prioritises mail using the MX records given by your DNS server. It may be useful sometimes to verify your assumptions of what the MX records are saying by interrogating the MX records with dig, and to follow a mail message using sendmail -v.

 # dig example.com mx  
; <<>> DiG
2.2 <<>> samnet.com mx
;; res options: init recurs defnam dnsrch
;; got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 40547
;; flags: qr aa rd ra; Ques: 1, Ans: 2, Auth: 2, Addit: 2
;; QUESTIONS:
;;example.com, type = MX, class =
IN

;; ANSWERS:
example.com. 86000
MX50 mail2.example.com.
example.com. 86000
MX20 mail.example.com.

;; AUTHORITY RECORDS:
example.com. 86000
NSdns.example.com.
example.com. 86000
NSdns2.example.com.

;; ADDITIONAL RECORDS:
mail2.example.com.
86000 A 192.168.101.130
mail.example.com.86000
A 192.168.101.1

;; Total query time: 3 msec
;; FROM: iwill to SERVER: default -- 192.168.101.1
;; WHEN: Tue Aug1 09:06:47 2000
;; MSG SIZEsent: 28rcvd: 140

Dial-Up Mail Serving?

A few of the places we're working in do not have a live Internet connection, or prefers to process the world mail queue on a scheduled basis. On a dial-up class machine, we do not want the mail server attempting to send mail on every message and the below is one configuration that may be useful.

Essentially, we want the mail server to deliver local mail immediately, but hold all external mail until we process the queue. The concept is to define mail as expensive. For more details on implementation, consult the HOWTO Sendmail and dial-up modem internet by Wouter Hanegraaff

define(SMTP_MAILER_FLAGS, e)
define(`confTO_QUEUEWARN', `16h')
define(`SMART_HOST', `myISP')
MAILER(local)dnl
MAILER(smtp)dnl

I think from the above, smtp of local mail will get sent straight away. The queue will only need to be processed when external mail needs to be delivered (ie. when online)

From the docs:

Mailer Flag
e This mailer is expensive to connect to, so try to avoid
connecting normally; any necessary connection will occur during a queue
run.

Options:
There are a number of global options that can be set from a
configuration file. Options are represented by full words; some are
also representable as single characters for back compatibility.

The syntax of this line is:
O option=value

This sets option option to be value. Note that
there must be a space between 
the letter 'O' and the name of the option. An older version is:

Oo value 

where the option o is a single character.
Depending on the option, value may be a string, an integer, a boolean
(with legal values "t", "T", "f", or "F"; the default is TRUE), or a
time interval.

The options supported (with the old, one
character names in brackets) are:

HoldExpensive [c]
If an outgoing mailer is marked as being expensive, don't connect
immediately. This requires that queueing be compiled in, since it will
depend on a queue run process to actually send the mail.

Other Security Items

ESMTP

Authenticated Mail Relay.