| Present: |
P. Hazel,
D.P. Carter,
R.J. Dowling,
F.A.N. Finch,
C.J. Jardine,
K.M. Jeary,
R.A.W. Mee
B.K. Omotani, R. J. Smith, P. Stewart, C.E. Thompson, J.M. Wilkins |
Date of next meeting: 15th March 2006 at 11:15 in C304
There have been no further hardware problem on Hermes or PPSW since the motherboard on Cyrus-22 was switched out before Christmas. There have also been no further kernel deadlocks on the Cyrus servers since a trivial (single line) patch was applied to the ReiserFS code.
Hermes-1 and 2 (the two systems hosting the Hermes Webmail and SSH server) have both had additional memory installed, allowing us increase the timeout on Webmail sessions. This has also allowed us to increase the maximum number of HTTP listeners on the system, which works around a very slow leak in the master process keeping track of the number of HTTP listeners.
10.5 TBytes of usable capacity (23x500 GByte disks configured as a single RAID-6 stripe plus a hot spare) have been added to the Hermes backup spooling system Otanes.
There are currently 113 Mailman lists on the system (an increase from 80 at the last MDCM, entirely by word of mouth). Given the large number of options that Mailman provides, we really need proper documentation before it can be advertised as a full service: DPC, SP and AK have had a preliminary meeting to discuss requirements.
Currently 7803 people are still allowed to make insecure IMAP, POP and SMTP connections to Hermes, and 1588 are allowed to use ~/mail and variants. Engineering have started to sort out their users at our request, and a another department (probably Education) will be nominated as a second trial to refine the whole process. FANF gave a talk about correct configuration of mail user agents to the Techlinks on 25th January 2006. Around 5 to 10 people in the Computing Service are left making insecure connections: they will be used as guinea pigs for a more aggressive notification system which will warn people every 30 minutes that they continue to use insecure settings.
The rate limiting project is currently on hold as there are too many difficult cases to deal with. In particular the system as it currently stands is not very forgiving for the kind of non-technical users who generate the majority of legitimate bulk email (e.g: alumni offices).
Earth Sciences are about to replace two existing mail systems with a single new one. This should hopefully cure the problems that we were seeing with the two old servers running different antispam policies in series.
Kings have been asked for a time estimate for the rationalisation of their current mail setup (an Exchange server hidden behind an escape route from the kings.cam.ac.uk managed mail domain). In the mean time FANF is planning a lockdown to prevent the escape route name (adminmail.kings.cam.ac.uk) from leaking out to unrelated parts of the Exim configuration on PPSW.
postmaster@cam.ac.uk currently redirects to a shared mailbox on CUS. With the demise of CUS this will clearly have to move. A shared mailbox on Hermes is an obvious part to the solution, however this may be a good opportunity to also move postmaster mail into a proper ticket tracking system such as RT. unix-support have now been running an RT system for their own mail for a number of months and we are aware of a number of other institutions (including our own Engineering department, and Oxford University Computing Services) who use RT for postmaster mail.
Similar arrangements may be sensible for network-support@ucs.cam.ac.uk, and other role addresses used around the Computing Service. A single unified system would be preferable to multiple competing systems.
A fair number of JCR and MCR Web sites have direct login forms to Hermes Webmail, bypassing the main Hermes login page. This is clearly a bad thing from a security standpoint as it encourages users to enter Hermes passwords into a page which is not part of Hermes. It also means that users miss any Webmail message of the day. People who put up such forms are relying on the current layout and behaviour of the HTML forms on the Webmail login page with no guarantees about future compatibility.
Code was added to Prayer last summer to spot and eventually block people who login through one of these forms. Currently around 5,000 logins a day (out of 130,000 logins a day total) to Webmail are via these external login pages. While we would like to block external logins as soon as practical, we clearly need to contact the 30 or so sites that would be affected first.
AOL blocked all messages sent from the 4 PPSW systems from about 11:30 on Sunday 29th January to 15:30 on the afternoon of the following day. AOL is one of the few domains where we still do sender callout verification, so this also blocked all email sent from AOL users to Cambridge addresses.
AOL clear blocks automatically after 24 hours. Postmaster@hermes.cam.ac.uk has been signed up to the AOL feedback system so that in future we are at least notified the next time that we are blocked.
Unlike previous occasions, there was no obvious compromised machine on the CUDN spewing spam messages through PPSW which caused them to blacklist us. The working hypothesis is that the block was caused by Cambridge users who forward email to AOL. The AOL mail interface includes a link to report messages as spam, unfortunately this interface appears to make no provision for people who forward email to AOL through innocent third parties. If this hypothesis is correct, then one option to reduce the fallout from these blocks in future would be to send messages redirected to external sites by Hermes via a separate set of interfaces on PPSW. This would mean that the blocks would only impact people who forward mail onto AOL, rather than normal Hermes users who are just trying to communicate with AOL users.
Given that it is very likely that we will be blocked again in future, we need to either discourage Cambridge users from forwarding email to AOL or discourage them from using the spam reporting system at the AOL end. Using the spam filter on Hermes to filter out messages before the remainder are redirected would decrease the number of messages that people can report as spam at the AOL end. However, this is likely to cause problems with false positives being filtered on Hermes: the messages would effectively disappear into a black hole as far as the user is concerned.
DPC 2006-02-01