Good configuration style for Exim

Tony Finch <dot@dotat.at>

"Style is a simple way of
saying complicated things"
- Jean Cocteau

Version: $Cambridge: hermes/doc/talks/2006-07-exim-course/talk.html,v 1.8 2006/07/20 14:23:45 fanf2 Exp $

About me

My CV expressed as a list of email addresses.

I went to University at Cambridge, where I started using Exim in 1997. After University I worked at Demon Internet, who use Exim - although I wasn't a postmaster there.

I didn't escape Cambridge for long: I soon returned, and I've now been working for the University of Cambridge Computing Service for four years. I help to run the central email systems and provide technical support to others running email servers in the University.

I have in the past contributed to the Apache HTTP server and FreeBSD projects.

This talk is about some of what I've learnt about working with Exim.

Style in the large

Learning how Exim's parts fit together

Exim is a big program with several significant components. It's important to understand how they fit together, and how to configure them so that they work with each other cleanly.

So I'm going to start off by talking about style in the large - the high-level structure of your configuration. Later on I will talk a little about small-scale style, including some layout suggestions.

Old code

It's usually best to use newer features and avoid the legacies from the early days

One way of making it easier to learn a program is to find out which parts you can safely ignore. If you are a long-time Exim user, or if you are coming to Exim from other MTAs, you might be led astray by differences in terminology etc.

Rewriting

You only need to use the rewriting features
in unusual circumstances

Some MTAs do everything via rewriting. In general you should look at Exim's routing functionality (especially the redirect router) when you think of rewriting.

System filter

You only need to use the system filter
in unusual circumstances

The system filter is a legacy from the early days of Exim. Nowadays ACLs are generally more powerful for anti-spam checks, and they run at SMTP time which is better. Routers are much more flexible now, so most routing hacks no longer need the system filter.

ACLs and Routers


The key to Exim is to understand
how ACLs and routers work together

The interestnig parts of your configuration are implemented in the ACLs and routers. Both are very flexible, so you want a rule of thumb about what parts of your configuration should be implemented using ACLs and which using routers.

Front end - ACLs

Exim has a front end (ACLs) which determines
what email it will take in ...

The rule of thumb for ACLs is that they make decisions entirely based on who the client is, identified by hostname or IP address or TLS client certificate or SASL authentication.

Trusted clients


Example: accepting email from trusted clients



  accept
    hosts = +relay_from_hosts      

  accept
    authenticated = *

A couple of simple examples where we decide to accept a message based on who the client is.

Blacklisted clients


Example: rejecting email from spammers



deny
  message = You are blacklisted:\  
            see ${dnslist_text}
  dnslists = sbl-xbl.spamhaus.org  

Another simple example where we decide to reject a message based on who the client is.

A wrong example


Do not put address logic in ACLs!


  VDOMS = /etc/mail/domains
  
  require
    domains = dsearch;VDOMS
    local_parts = lsearch;VDOMS/$domain

This is an example of what not to do. We'll see the right way to do it in a moment.

Back end - routers


... and a back end (routers) which determines
where the email goes out

The rule of thumb for routers is that they deal only with email addresses. They are blind to how Exim accepted the message.

Virtual domains


Keep all address logic in one place,
in the routers ...


domainlist virtual_domains = \
                dsearch;VDOMS

virtual_domains:
 driver = redirect
 domains = +virtual_domains
 local_parts = lsearch;VDOMS/$domain 
 data = $local_part_data

This is the router for the previous wrong example. The bad example ACL duplicates the logic and must do so accurately - an opportunity for making mistakes when you change something.

Manualroute


... even if the addresses aren't totally local



local_part_list internal_users = \
  lsearch;/etc/mail/internal_users

internal_server:
 driver = manualroute
 domains = +internal_domain
 local_parts = +internal_users
 route_data = internal.mail.example  
 transport = smtp

A common reason for people to put address logic into ACLs is when the addresses are not local to the Exim server - for example when Exim is being a gateway to an internal mail server. The wrong train of thought is "I want to reject email to invalid addresses, and I reject things using ACLs"; the right train of thought is "address verification is done with the routers, so the routers need to know which addresses are valid". We'll come back to this again in a moment.

In many cases the local part list will be (for example) an LDAP lookup rather than a file on disk.

Loosely coupled


ACLs and routers are loosely coupled
via address verification requests

Address verification


In most cases, you will only need the simplest
address verification check in your ACLs

  require
    verify = recipient          
  

With this verification check, and the correct address checking logic in the routers, there's no need for any address logic in the ACLs.

Call-forward verification


If you don't have a list of valid addresses,
use call-forward verification

  require
    verify = recipient/callout=\
             use_sender,defer_ok

This is useful in the absence of something like the LDAP lookup I mentioned earlier, for example when you don't have co-administration of the internal mail server.

User-dependent ACLs


You can pass per-user info from the routers
to the ACLs with the address_data feature

 SENDER = sender_address_data

 require
   authenticated = *
 require
   verify = sender
 deny
   message = Attempted forgery
   condition = ${extract {user}{$SENDER} \
     {${if !eq{$value}{$authenticated_id} }} }

This is probably the most complicated example I'll show. It comes from the ACLs on our message submission server. The routers chase through all the various aliases and redirections, probably ending up with a user's real account. (For example, this maps friendly-name addresses to usernames.) If the account name does not match the authenticated username, then someone is messing around. (If no username is found by the routers - e.g. sending with a shared address or an external vanity address - the extract will yield nothing so the message will be accepted.)

Note the ACL has no knowledge about addresses here, such as how they map on to user names: it's just using the result of the router's logic.

You can do similar things with recipient verification and per-user anti-spam options.

Open relay check


About the only ACL check that must examine an address directly is the no-relay check

 require
   domains = +local_domains

The anti-relay check should be a simple check against a domain list. We'll have a closer look at lists in a moment.

Sender blacklists


One possible exception to the rule is sender address blacklists


SPAMMERS = /etc/mail/spammers  

deny
  senders = lsearch;SPAMMERS  

Although you can implement a sender address blacklist using the routers, it's more conventional to do so using an ACL clause like this. (The router version has the side-effect of preventing you from sending email to the blacklisted address as well as receiving email from it.)

Named lists


Use named lists to avoid repeating lookups

So far in the examples I have used a lot of named lists, and most of them have been fairly trivial. You might think this is pointless, but it isn't.

Repeating lists


Lists usually appear at least twice in a configuration

domainlist virtual_domains = \
        dsearch;VDOMS
domainlist local_domains = \
        +virtual_domains : •••
# •••
require domains = +local_domains  
# •••
virtual_domains:
  driver = redirect
  domains = +virtual_domains
  # •••

The more complicated your configuration the more likely you are to have repeats, and the more complicated your configuration the more you need to use techniques to keep it simple. The local_domains list can get complicated if you have several different kinds of domains. You might have different variants of a router for verification and delivery, or both aliases and local user routers, which all increase the number of times a list is checked.

Side effects


List matches in routers have side-effects that let you avoid nasty string expansions


domainlist domain_routes = \
  lsearch;/etc/mail/domain_routes 

domain_routes:
  driver = manualroute
  domains = +domain_routes
  route_data = $domain_data
  transport = smtp

The lookup syntax in string expansions is cluttered, so it's nice to be able to avoid it. This example shows that the result of the lookup in a domain list is available in $domain_data, which allows you to avoid both repeating the lookup and an ugly string expansion.

This also shows how you can re-use names to tie parts of the configuration together. Give the list the same name as the table it is based on, and give the router the same name as the list it matches.

Routers

Routers can be subtle so we should take care when configuring them.

I'm now going to talk a bit about router configuration.

Order of routers


Minimize the order dependencies of your routers

  • Always put preconditions on your routers, instead of relying on previous routers to handle addresses you don't want this one to handle.

  • Define lists in the same order as the routers they control.

In my configuration I have lots of types of domains: virtual domains, a mapping from long-form to short-form domains (like quns.cam.ac.uk -> queens.cam.ac.uk), obsolete domains which get a special error message (for when a department changes name), manually routed domains, and special domains like Hermes (with the LMTP router above). These mostly do not overlap, so the routers could appear in any order. The order only really matters when a domain changes from one setup to the other, and we need to manage the transition cleanly.

Wrong ordering

Router options can be written in almost any order, but have a fixed order of evaluation

  domain_routes:
    transport = smtp
    driver = manualroute
    route_data = $domain_data
    domains = +domain_routes

This version of the previous example has the same meaning, but it is written in a misleading order which obscures the way it works.

Order of evaluation (1)

Write router preconditions in the order they are evaluated ...

  • driver
  • local_part_prefix/suffix(_optional)  
  • verify(_recipient/_sender)
  • address_test
  • verify_only
  • expn
  • domains
  • local_parts
  • check_local_user
  • debug_print
  • router_home_directory
  • senders
  • require_files
  • condition
  • address_data
  • fail_verify(_recipient/_sender)
  • ignore_target_hosts

The order is not very obvious, so you'll have to refer to section 3.12 and chapter 15 of the manual. It matters partly because some options have side-effects like setting expansion variables, e.g. $domain_data.

Order of evaluation (2)

... then options that can make the router decline, then the miscellaneous options ...



  • check_secondary_mx (dnslookup)  
  • mx_domains (dnslookup)
  • route_data (manualroute)
  • route_list (manualroute)
  • command (queryprogram)
  • data (redirect)
  • file (redirect)


These are most of the other options that can make a router decline.

The miscellaneous options are generally un-ordered.

Order of evaluation (3)

... and last of all the transport-related options

  • errors_to
  • headers_add
  • headers_remove
  • transport
  • directory_transport (redirect)  
  • file_transport (redirect)
  • pipe_transport (redirect)
  • reply_transport (redirect)
  • group
  • initgroups
  • transport_current_directory
  • transport_home_directory
  • user

These last few options are also ordered.

Example router


A real-life example with lots of preconditions


      hermes_lmtp:
        driver		     = manualroute
        local_part_suffix    = +*
        local_part_suffix_optional
        no_verify
        domains		     = hermes.cam.ac.uk
        local_parts	     = +hermes_active
        route_data           = $local_part_data byname
        host_find_failed     = defer
        retry_use_local_part
        transport            = ${if ={0}{$body_zerocount} \
      			            {hermes_lmtp} {hermes_lmtp_filter} }

These last few options are also ordered.

String expansions


String expansions need careful layout
to compensate for cluttered braces

String expansions can easily become completely illegible. Many people get into a muddle as a result, judging by questions on the exim-users list. So here are a couple of tips.

Spaces in braces


Use spaces inside the curly brackets
of ${if, ${lookup, ${extract


${extract {user}{$SENDER} \
    {${if !eq{$value} \
             {$authenticated_id} }} }

The extra white space in this example is not included in the result of the expansion, so you can use it to make the expression more readable. See how the pair of closing braces match the pair of opening braces before if.

You can also omit the {yes} {no} in many cases.

Spaces in conditons


Use spaces inside the curly brackets
of compound conditions

      ${if or{{ eq{$sender_helo_name}{$local_part} } \
              { match{$sender_helo_name}{\N^[0-9.-]+$\N} } \
              { match_domain{$sender_helo_name}{+local_domains} }} }

Again see how the paired braces match. Also see how the spaces around the inner conditions keep them from being mixed up with the or's braces.

The condition condition

Never use the condition option if you can solve the problem more elegantly using the other router or ACL conditions

The condition option is a general-purpose escape route for use when the other conditions can't do what you want. However it relies on string expansions which are ugly, so it should be avoided where possible.

Macros


Use macros to avoid repeating things like filenames or parts of string expansions


DB = /opt/exim/etc/db

SERVER = ${lookup {$interface_address} \
            cdb {DB/server_params.cdb} }

domainlist special_routes = \
           cdb;DB/special_routes.cdb

Named ACL variables


Use macros to give meaningful names to numbered ACL variables


SENDER = acl_m0

require
  verify = sender
  set SENDER = $sender_address_data  

Exim does macro expansion before doing anything else with the configuration file, so you can use them to make ACL variable names more meaningful. In this example we're saving the sender address data for later use. (ACL variables are preserved longer than the $sender_address_data variable.

Main configuration


Group the main configuration options
according to chapter 14 of the spec

There are a lot of configuration options that go in the first part of the configuration file. Chapter 14 of the specification has a series of summary sections which divide the options into groups, which you can use to structure your configuration file.

Order of sections


My preferred order of sections

  • ACLs
  • authenticators
  • local_scan
  • rewrite rules
  • routers
  • transports
  • retry rules

Different from the default configuration, but it has the advantage that it is the order of evaluation. Note that this is for server authentication: if Exim is authenticating as a client it's probably better to put the authenticators after the transports.

Summary


Final words

Remember that these are suggestions,
not strict rules.