% $Cambridge: hermes/doc/misc/atcam.tex,v 1.5 2012/02/06 15:37:59 fanf2 Exp $

\documentclass[a4paper]{article}

\usepackage{amssymb}
\usepackage{fullpage}
\usepackage{url}
\usepackage{wasysym}

\newenvironment{note}{
  \begin{description}
    \item[Note:]
}{
  \end{description}
}

\newenvironment{query}{
  \begin{description}
    \item[Question:]
}{
  \end{description}
}

\DeclareUrlCommand\email{%
  \def\UrlLeft{<}%
  \def\UrlRight{>}%
  \urlstyle{tt}%
}

\DeclareUrlCommand\domain{%
  \def\UrlLeft{@}%
  \urlstyle{tt}%
}

\newcommand\atcam{\texttt{@cam}}
\newcommand\athermes{\texttt{@hermes}}

\newcommand\redir{\texttt{https://atcam.csx.cam.ac.uk/}}

\newcommand\jackdaw{\texttt{https://jackdaw.cam.ac.uk/cammail/}}

\newcommand\tickybox{$\rlap{\raisebox{.2em}{\hspace{0.1em}\checkmark}}\square$}

\setlength{\parindent}{0in}
\setlength{\parskip}{0.75em}

\setlength{\skip\footins}{2em}

% rubric

\author{
  Tony~Finch \email{fanf2@cam.ac.uk}\\
  Mail Support\\
  University of Cambridge Computing Service\\
}

\title{
  \atcam\ change-of-address autoreplies departed users
}

\date{
  January 2011
  \footnote{
    \texttt{
      $\$$Cambridge: hermes/doc/misc/atcam.tex,v 1.5 2012/02/06 15:37:59 fanf2 Exp $\$$
    }
  }
}

\begin{document}

\maketitle

%

\section{Overview}

This memo describes our plan to improve the \atcam\ email forwarding
service to provide automatic change-of-address notices in respose to
messages sent to departed Cambridge users.

\begin{itemize}

\item Core functions - initial phase

  \begin{itemize}

  \item Current users will be able to set an address which will later
    be used to generate friendly CoA autoreplies after they have left.

  \item The user interface will be provided by the Ibis identity
    management service. The old interface at \jackdaw\ will be
    decommissioned.

  \item We will abolish Jackdaw's \atcam\ address search web page.

  \end{itemize}

\item Later improvements - subsequent phase(s)

  \begin{itemize}

  \item Hermes users will be able to view and configure both their
    Hermes and \atcam\ settings from the Hermes webmail redirect
    configuration page.

  \item We will make it easier to set up \atcam\ addresses for new
    users who do not have Hermes accounts.

  \item We will allow departed users to change their ``tombstone''
    addresses once the Corvid authentication service is available.

  \end{itemize}

\item Notable features ruled out of this project

  \begin{itemize}

  \item The current \jackdaw\ forwarding setup process requires the
    user to verify that the destination address works, whereas the
    equivalent facility in Hermes webmail does not. The lack of safety
    has not been a problem in practice so \atcam\ will not include
    this extra verification step.

  \item This will not be a forwarding service for departed users.

  \end{itemize}

\end{itemize}

%

\section{Abolishing the \atcam\ email directory}

There is a public directory at
\url{https://jackdaw.cam.ac.uk/mailsearch/} which allows anyone to
find out the affiliation and \atcam\ email address of most people at
Cambridge. At the moment, users are encouraged to opt in to this
directory when they register for their Computing Service accounts.

We plan to abolish this directory because it has mostly been made
redundant by the Lookup and Ibis services, and the convenience of
searching the contact pages on institution web sites. Removing the
mailsearch ``soundex'' fuzzy matching facility will make it harder for
spammers to get lists of valid \atcam\ email addresses.

%

\section{User interface and data model}

Current users will be able to log in to Ibis using Raven
authentication. Ibis will allow them to configure their live
\atcam\ ``forwarding'' address, and a ``tombstone'' address which will
be used for CoA autoreplies after their Raven account is cancelled.
(The tombstone goes up when the user goes down.)

Mail to a current user's \atcam\ address will be delivered to their
forwarding address if they have set one.

When a user leaves or unsets their forwarding address, mail to their
\atcam\ address will trigger a CoA autoreply if they have have
configured an \atcam\ ``tombstone'' address.

Administrative and helpdesk access to these settings will be
determined by Ibis.

The above addresses are private. A user of Ibis can also set their
``email'' attribute to a list of addresses that will appear in the
directory, visible to others. This list should include the user's
\atcam\ address if it exists.

The \atcam\ system will periodically pull data about current users
from Ibis. It will update a local database according to the lifecycle
logic described in the next section. This database will include:

\begin{itemize}
\item username (from Ibis)
\item full name (from Ibis)
\item affiliation (from Ibis)
\item forwarding address (from Ibis)
\item tombstone address (from Ibis)
\item cancellation date (from lifecycle logic)
\item new account flag (from lifecycle logic)
\end{itemize}

\begin{note}
  In most cases a new user's \atcam\ forwarding will be set up
  automatically. This is the case for Hermes users, and we plan to
  extend this to users of other mail systems in a later phase of this
  project. Automatic setup can only happen to ``new'' users.
\end{note}

\begin{note}
  The two separate addresses are required to cope with the common
  situation where a Hermes user forwards their email to an external
  account. Their \atcam\ address should forward to Hermes so that the
  user's other filtering settings take effect (such as spam filtering
  and keeping a copy on Hermes), but the \atcam\ system also needs to
  know needs to know where Hermes is forwarding the user's mail in
  order to generate useful CoA autoreplies after the user has left.
\end{note}

\begin{note}
  \label{loophole}
  Some categories of people who have Raven accounts are not entitled
  to an \atcam\ address, such as Continuing Education students. This
  has not been a problem so far, so we have decided not to spend effot
  closing this loophole.
\end{note}

%

\section{\atcam\ address lifecycle}

The \atcam\ system will run on the same server that handles Hermes
administration tasks, so as well as its own data feed from Ibis it
will have access to the Hermes user database. This will be used for
automatic set up of \atcam\ addresses for Hermes users.

In the following, when a user's forwarding address is autmomatically
set, this change is made to both Ibis and the \atcam\ database. When
we talk about a user's Raven/Ibis account, we mean that we infer the
existence of the user's Raven account from the fact that they appear
in the Ibis data feed.

There is a general Computing Service policy that we expect users to
remember details of their account if it is restored within 215 days
(seven months). When accounts are restored after a short absence (less
than 215 days) they are brought back as they were when the user left.
We reset easily forgotten settings (such as the user's password) when
restoring accounts after a longer absence.

\subsection{\atcam\ address creation}

\label{create}
When a user first appears in the data feed from Ibis, they will be
added to the \atcam\ database with their ``new'' flag set. Their
\atcam\ address will continue to reject all mail until they set a
forwarding or tombstone address, at which point their ``new'' flag
will be cleared.

\begin{query}
  Should we automatically add their \atcam\ address to their
  \atcam\ attribute in Ibis at this time?
\end{query}

\subsection{Raven/Ibis account cancellation}

When a user's account is cancelled the date is recorded in the
\atcam\ database. Mail to their \atcam\ address will no longer be
forwarded. If they have enabled a tombstone address then mail to their
\atcam\ address will trigger a CoA autoreply, otherwise it will be
rejected.

\subsection{Raven/Ibis account restoration - short absence}

When a user's account is restored a short time after their
cancellation date, forwarding for their \atcam\ address will be
re-activated as it was before they left. (But see section
\ref{hermes-can}.)

\subsection{Raven/Ibis account restoration - long absence}

\label{restore}
When a user's account is restored a long time after their cancellation
date, their \atcam\ forwarding address will be cleared and their
``new'' flag set. This means that CoA autorplies will continue until
the user sets a forwarding address using Ibis, or one is automatically
set up for them.

\subsection{Hermes automatic setup}

If a ``new'' user has a Hermes account their \atcam\ forwarding
address will be automatically set to the user's \athermes\ address and
their ``new'' flag will be cleared. (This will happen whether their
Hermes account is created before or after they become ``new''.) At the
same time their \atcam\ address will be added to their ``email''
attribute in Ibis.

\subsection{Hermes cancellation}

\label{hermes-can}
When a user's \atcam\ forwarding address is their \athermes\ address
but their Hermes account does not exist, the \atcam\ system will
behave as if their forwarding address were unset.

%

\section{Later improvements}

The following subsections describe some improvements that we plan to
implement after the basic phase one functionality has been put into
service. The details will become clearer as we get closer to
implementing these improvements.

%

\subsection{Hermes webmail integration}

We plan to modify Prayer, the Hermes webmail software, so that most
users can configure their \atcam\ and \athermes\ forwarding settings
in one place.

Users reach the forwarding settings page by logging in to webmail then
go to manage then redirect. Webmail will display their current
settings:

\begin{itemize}
\item \athermes\ forwarding address
\item \atcam\ forwarding address
\item \atcam\ tombstone address
\end{itemize}

Input controls:

\begin{itemize}
\item New forwarding address \fbox{recipient@example.com}
\item \tickybox\ Save copy on Hermes
\item \tickybox\ Change \atcam\ forwarding address to Hermes
  (if that is not the current state)
\item \tickybox\ Change \atcam\ tombsone address match the Hermes
  forwarding address above.
\end{itemize}

This can be simplified in the normal case that the user's
\atcam\ forwarding address is their \athermes\ address, and their
\atcam\ tombstone and \athermes\ forwarding addresses are the same.

Prayer will obtain the user's \atcam\ settings from Ibis and copy any
changes to Ibis. The \atcam\ system will in turn pick them up from
Ibis.

Users will not be able to view their Hermes settings from Ibis, though
in normal cases Ibis will have a correct copy of the principal settings.

\begin{note}
  Prayer allows users to write their own Sieve scripts if they need
  more advanced filtering features. Users with manually maintained
  Sieve scripts are not able to change their settings using Prayer's
  friendly filtering user interface.

  Users with manually maintained Sieve scripts that visit webmail's
  redirect page will be informed they can use Ibis to control their
  \atcam\ settings.
\end{note}

%

\subsection{Other Cambridge mail systems}

At the moment, new users who do not have Hermes accounts (e.g. staff
at the Medical School or Unified Administrative Service) must manually
set up their own \atcam\ forwarding when they arrive. We would like to
provision their addresses automatically when possible, as we do for
Hermes accounts.

When a ``new'' user arrives, if their affiliation is an institution
with an email system that we know about, the \atcam\ system will
automatically set their forwarding address to that system.

%

\subsection{Authenticating departed users}

When we are able to authenticate departed users, we intend to allow
them to update the tombstone address in their CoA autoreplies.

%

\section{Outline plan of action}

%

\subsection{Phase one}

\begin{itemize}
\item Feb-Mar: implementation and testing of major system components.

  \begin{itemize}

  \item Autoreply mechanism
    \begin{itemize}
    \item Technical notes in following section
    \item Can be tested with a dummy tombstone table
    \end{itemize}

  \item Administration software
    \begin{itemize}
    \item Data feed from Ibis
    \item Modification of data on Ibis
    \item Lifecycle logic
    \item Forwarding and tombstone table generation
    \end{itemize}

  \item Documentation

  \end{itemize}

\item Apr: deployment

  \begin{itemize}
  \item Ensure that all current users with \atcam\ addresses have Raven accounts.
  \item Sync new \atcam\ database with old Jackdaw cammail database
  \item Switch live forwarding table from old to new version
  \item Redirect \jackdaw\ to Ibis
  \item Redirect \url{https://jackdaw.cam.ac.uk/mailsearch/}
    to \url{http://search.cam.ac.uk}
  \end{itemize}

\end{itemize}

\subsection{Phase two}

\begin{itemize}
\item May-Jun: updates to webmail

  \begin{itemize}
  \item Prayer forwarding UI changes
  \item Hooks into \atcam\ database via \texttt{accountd}
  \end{itemize}

\item Jul onwards

  \begin{itemize}
  \item Better support for other University email systems
  \item Authentication of departed users
  \end{itemize}

\end{itemize}

%

\section{Technical details of autoreply implementation}

The mechanism should conform to RFC 3834 ``reccomendations for
automatic responses to electronic mail'' to avoid problems with mail
loops and sending replies to inappropriate places such as mailing
lists. There are further recommendations in RFC 5436 ``Sieve
notification mechanism: mailto''.

We may be able to arrange that messages are rejected outright if they
don't meet the criteria for a friendly reply. This will quell unwanted
mailing list subscriptions, and reduce the extent to which we attract
spam to invalid addresses - \texttt{esc.cam.ac.uk} still gets a
disproportionately large volume of spam owing to a period when they
accepted mail to any address.

The following subsections are a brief survey of the facilities in Exim
that might be used in the implementation.

\subsection{System filter vs routers}

There are a couple of options for determining when redirection
autoreplies should be sent. The informal model is that the autoreplies
should be similar to vacation messages, which suggests using Exim's
filter mechanism. However it isn't possible to implement per-recipient
logic using a system filter, and we need to handle messages sent to
both current and departed users. We can get all the filter
functionality just using routers in Exim's main configuration file,
which also avoids the need for another configuration file

Exim supports some of the RFC 3834 requirements via the ``personal''
test in its filter language. We can make use of this by embedding a
filter script fragment in a ``redirect'' router in the main
configuration file.

\subsection{Autoreply transport}

\begin{itemize}
\item Includes ``once'' mechanism to avoid suppress repeated friendly
  bounces in a given time interval
\item Requires multiple ``once'' databases, one for each departed user
  on each ppswitch machine
\item Separate replies for each departed recipient if a message has
  multiple recipients
\end{itemize}

\subsection{Pipe transport}

\begin{itemize}
\item Output of piped command is returned to sender
\item No built-in duplicate suppression
\item One reply covering all departed recipients
\end{itemize}

\subsection{Choice of transport}

The autoreply transport provides most of what we need, except that the
``once'' databases will not scale up gracefully. We can either extend
Exim to provide the right functionality in the autoreply transport, or
use the pipe transport and implement our custom autoreply logic in an
external process.

%

\end{document}

% eof