@cam change-of-address autoreplies departed users

                 Tony Finch ⟨fanf2@cam.ac.uk⟩

Mail Support University of Cambridge Computing Service

January 2011 ^{∗}

1 Overview

This memo describes our plan to improve the @cam email forwarding service to provide automatic changeof-address notices in respose to messages sent to departed Cambridge users.

• Core functions - initial phase

– Current users will be able to set an address which will later be used to generate friendly CoA autoreplies after they have left.

– The user interface will be provided by the Ibis identity management service. The old interface

  at https://jackdaw.cam.ac.uk/cammail/ will be decommissioned.

– We will abolish Jackdaw’s @cam address search web page.

• Later improvements - subsequent phase(s)

– Hermes users will be able to view and configure both their Hermes and @cam settings from the Hermes webmail redirect configuration page.

– We will make it easier to set up @cam addresses for new users who do not have Hermes accounts.

– We will allow departed users to change their “tombstone” addresses once the Corvid authentication service is available.

• Notable features ruled out of this project

– The current https://jackdaw.cam.ac.uk/cammail/ forwarding setup process requires the user to verify that the destination address works, whereas the equivalent facility in Hermes webmail does not. The lack of safety has not been a problem in practice so @cam will not include this extra verification step.

– This will not be a forwarding service for departed users.

2 Abolishing the @cam email directory

There is a public directory at https://jackdaw.cam.ac.uk/mailsearch/ which allows anyone to find out the affiliation and @cam email address of most people at Cambridge. At the moment, users are encouraged to opt in to this directory when they register for their Computing Service accounts.

∗ $Cambridge:  hermes/doc/misc/atcam.tex,v 1.5 2012/02/06 15:37:59 fanf2 Exp $

We plan to abolish this directory because it has mostly been made redundant by the Lookup and Ibis services, and the convenience of searching the contact pages on institution web sites. Removing the mailsearch “soundex” fuzzy matching facility will make it harder for spammers to get lists of valid @cam email addresses.

3 User interface and data model

Current users will be able to log in to Ibis using Raven authentication. Ibis will allow them to configure their live @cam “forwarding” address, and a “tombstone” address which will be used for CoA autoreplies after their Raven account is cancelled. (The tombstone goes up when the user goes down.)

Mail to a current user’s @cam address will be delivered to their forwarding address if they have set one.

When a user leaves or unsets their forwarding address, mail to their @cam address will trigger a CoA autoreply if they have have configured an @cam “tombstone” address.

Administrative and helpdesk access to these settings will be determined by Ibis.

The above addresses are private. A user of Ibis can also set their “email” attribute to a list of addresses that will appear in the directory, visible to others. This list should include the user’s @cam address if it exists.

The @cam system will periodically pull data about current users from Ibis. It will update a local database according to the lifecycle logic described in the next section. This database will include:

• username (from Ibis)

• full name (from Ibis)

• affiliation (from Ibis)

• forwarding address (from Ibis)

• tombstone address (from Ibis)

• cancellation date (from lifecycle logic)

• new account flag (from lifecycle logic)

Note: In most cases a new user’s @cam forwarding will be set up automatically. This is the case for Hermes users, and we plan to extend this to users of other mail systems in a later phase of this project. Automatic setup can only happen to “new” users.

Note: The two separate addresses are required to cope with the common situation where a Hermes user forwards their email to an external account. Their @cam address should forward to Hermes so that the user’s other filtering settings take effect (such as spam filtering and keeping a copy on Hermes), but the @cam system also needs to know needs to know where Hermes is forwarding the user’s mail in order to generate useful CoA autoreplies after the user has left.

Note: Some categories of people who have Raven accounts are not entitled to an @cam address, such as Continuing Education students. This has not been a problem so far, so we have decided not to spend effot closing this loophole.

4 @cam address lifecycle

The @cam system will run on the same server that handles Hermes administration tasks, so as well as its own data feed from Ibis it will have access to the Hermes user database. This will be used for automatic set up of @cam addresses for Hermes users.

In the following, when a user’s forwarding address is autmomatically set, this change is made to both Ibis and the @cam database. When we talk about a user’s Raven/Ibis account, we mean that we infer the existence of the user’s Raven account from the fact that they appear in the Ibis data feed.

There is a general Computing Service policy that we expect users to remember details of their account if it is restored within 215 days (seven months). When accounts are restored after a short absence (less than 215 days) they are brought back as they were when the user left. We reset easily forgotten settings (such as the user’s password) when restoring accounts after a longer absence.

4.1 @cam address creation

When a user first appears in the data feed from Ibis, they will be added to the @cam database with their “new” flag set. Their @cam address will continue to reject all mail until they set a forwarding or tombstone address, at which point their “new” flag will be cleared.

Question: Should we automatically add their @cam address to their @cam attribute in Ibis at this time?

4.2 Raven/Ibis account cancellation

When a user’s account is cancelled the date is recorded in the @cam database. Mail to their @cam address will no longer be forwarded. If they have enabled a tombstone address then mail to their @cam address will trigger a CoA autoreply, otherwise it will be rejected.

4.3 Raven/Ibis account restoration - short absence

When a user’s account is restored a short time after their cancellation date, forwarding for their @cam address will be re-activated as it was before they left. (But see section 4.6.)

4.4 Raven/Ibis account restoration - long absence

When a user’s account is restored a long time after their cancellation date, their @cam forwarding address will be cleared and their “new” flag set. This means that CoA autorplies will continue until the user sets a forwarding address using Ibis, or one is automatically set up for them.

4.5 Hermes automatic setup

If a “new” user has a Hermes account their @cam forwarding address will be automatically set to the user’s @hermes address and their “new” flag will be cleared. (This will happen whether their Hermes account is created before or after they become “new”.) At the same time their @cam address will be added to their “email” attribute in Ibis.

4.6 Hermes cancellation

When a user’s @cam forwarding address is their @hermes address but their Hermes account does not exist, the @cam system will behave as if their forwarding address were unset.

5 Later improvements

The following subsections describe some improvements that we plan to implement after the basic phase one functionality has been put into service. The details will become clearer as we get closer to implementing these improvements.

5.1 Hermes webmail integration

We plan to modify Prayer, the Hermes webmail software, so that most users can configure their @cam and @hermes forwarding settings in one place.

Users reach the forwarding settings page by logging in to webmail then go to manage then redirect. Webmail will display their current settings:

• @hermes forwarding address

• @cam forwarding address

• @cam tombstone address

Input controls:

• New forwarding address recipient@example.com

• ✓ □ Save copy on Hermes

• ✓ □ Change @cam forwarding address to Hermes (if that is not the current state)

• ✓ □ Change @cam tombsone address match the Hermes forwarding address above.

This can be simplified in the normal case that the user’s @cam forwarding address is their @hermes address, and their @cam tombstone and @hermes forwarding addresses are the same.

Prayer will obtain the user’s @cam settings from Ibis and copy any changes to Ibis. The @cam system will in turn pick them up from Ibis.

Users will not be able to view their Hermes settings from Ibis, though in normal cases Ibis will have a correct copy of the principal settings.

Note: Prayer allows users to write their own Sieve scripts if they need more advanced filtering features. Users with manually maintained Sieve scripts are not able to change their settings using Prayer’s friendly filtering user interface.

Users with manually maintained Sieve scripts that visit webmail’s redirect page will be informed they can use Ibis to control their @cam settings.

5.2 Other Cambridge mail systems

At the moment, new users who do not have Hermes accounts (e.g. staff at the Medical School or Unified Administrative Service) must manually set up their own @cam forwarding when they arrive. We would like to provision their addresses automatically when possible, as we do for Hermes accounts. When a “new” user arrives, if their affiliation is an institution with an email system that we know about, the @cam system will automatically set their forwarding address to that system.

5.3 Authenticating departed users

When we are able to authenticate departed users, we intend to allow them to update the tombstone address in their CoA autoreplies.

6 Outline plan of action

6.1 Phase one

• Feb-Mar: implementation and testing of major system components.

– Autoreply mechanism ∗ Technical notes in following section

∗ Can be tested with a dummy tombstone table

– Administration software ∗ Data feed from Ibis

∗ Modification of data on Ibis

∗ Lifecycle logic

∗ Forwarding and tombstone table generation

– Documentation

• Apr: deployment

– Ensure that all current users with @cam addresses have Raven accounts.

– Sync new @cam database with old Jackdaw cammail database

– Switch live forwarding table from old to new version

– Redirect https://jackdaw.cam.ac.uk/cammail/ to Ibis
– Redirect https://jackdaw.cam.ac.uk/mailsearch/ to http://search.cam.ac.uk

6.2 Phase two

• May-Jun: updates to webmail

– Prayer forwarding UI changes

– Hooks into @cam database via accountd

• Jul onwards

– Better support for other University email systems

– Authentication of departed users

7 Technical details of autoreply implementation

The mechanism should conform to RFC 3834 “reccomendations for automatic responses to electronic mail” to avoid problems with mail loops and sending replies to inappropriate places such as mailing lists. There are further recommendations in RFC 5436 “Sieve notification mechanism: mailto”.

We may be able to arrange that messages are rejected outright if they don’t meet the criteria for a friendly reply. This will quell unwanted mailing list subscriptions, and reduce the extent to which we attract spam to invalid addresses - esc.cam.ac.uk still gets a disproportionately large volume of spam owing to a period when they accepted mail to any address.

The following subsections are a brief survey of the facilities in Exim that might be used in the implementation.

7.1 System filter vs routers

There are a couple of options for determining when redirection autoreplies should be sent. The informal model is that the autoreplies should be similar to vacation messages, which suggests using Exim’s filter mechanism. However it isn’t possible to implement per-recipient logic using a system filter, and we need to handle messages sent to both current and departed users. We can get all the filter functionality just using routers in Exim’s main configuration file, which also avoids the need for another configuration file

Exim supports some of the RFC 3834 requirements via the “personal” test in its filter language. We can make use of this by embedding a filter script fragment in a “redirect” router in the main configuration file.

7.2 Autoreply transport

• Includes “once” mechanism to avoid suppress repeated friendly bounces in a given time interval

• Requires multiple “once” databases, one for each departed user on each ppswitch machine

• Separate replies for each departed recipient if a message has multiple recipients

7.3 Pipe transport

• Output of piped command is returned to sender

• No built-in duplicate suppression

• One reply covering all departed recipients

7.4 Choice of transport

The autoreply transport provides most of what we need, except that the “once” databases will not scale up gracefully. We can either extend Exim to provide the right functionality in the autoreply transport, or use the pipe transport and implement our custom autoreply logic in an external process.