Sending email from computers on the CUDN ======================================== $Cambridge: hermes/doc/misc/sending-from-cudn.txt,v 1.8 2005/10/28 15:30:05 fanf2 Exp $ This document explains how to configure your computer to send email from the CUDN without encountering problems. It discusses the choice of outgoing SMTP server, and some implications of our security policies. Hermes users ------------ Most people in the University use Hermes for email. The Hermes message submission service, called smtp.hermes.cam.ac.uk, is intended for Hermes users to send email from MUA software as documented at http://www.cam.ac.uk/cs/email/muasettings.html All email sent via smtp.hermes.cam.ac.uk should be securely authenticated. At the time of writing (Michaelmas 2005) this rule is not enforced, but we intend that insecure access will be withdrawn by Summer 2006. http://www.cam.ac.uk/cs/email/securehermes.html Using role addresses with Hermes -------------------------------- You do not have to use your @cam or @hermes address when sending email via Hermes. In particular, many departments, colleges, and research groups do not run their own email servers, and instead use the Managed Mail Domain service which provides special email addresses for use with Hermes. For example, the bursar of St Botolph's college might use the address , or the maintainer of a project web site may use the address . See leaflet G25 for more information about managed mail domains. http://www.cam.ac.uk/cs/docs/eleaflets/g25/ You can also use addresses at mail domains which are not managed by the Computing Service. The secure authentication requirement still applies whatever email address you use. Hermes ensures that authenticated senders are clearly identified in the messages' headers, to avoid confusion. Users of college or department email systems -------------------------------------------- Other email systems in the University will also provide a message submission service, similar to the facilities provided by Hermes. If you are a user of a department or college email system then you should configure your software to send email via that system's message submission service. The Computing Service cannot provide advice on the specifics - there are too many different systems out there for us to keep track of them all. Sending email from home or when travelling ------------------------------------------ Roaming users should send email via their "home" server if possible. For example, Hermes users can send email via smtp.hermes.cam.ac.uk from anywhere on the Internet if their software is correctly configured for secure authentication, as in http://www.cam.ac.uk/cs/email/muasettings.html In the past, before this facility was available, we said that roaming users should use the local ISP's smart host. This is no longer recommended because it is increasingly the case that anti-spam protocols check that the "from" address on messages corresponds to the originating server. However, some ISPs (particularly mobile phone operators) block all the ports that may be used to send email; in this situation you must use the ISP's smart host, and you must turn off any security and authentication options for SMTP, but leave them on for IMAP or POP. Users of college or department email systems should consult their computer officers for advice about message submission when away from the University. Users of non-University mail systems ------------------------------------ Similarly, users of external email services should configure their software to use that service's message submission servers. For example, if you are a visitor from Oxford who uses Herald for email, you should send via smtp.ox.ac.uk. However, note that because of the port 25 block (see below), you must configure your software to use an alternative port, usually one of 587 or 465 depending on what your software and email service supports. If your email service doesn't provide a message submission server on an alternative port, you can use the smart host ppsw.cam.ac.uk port 25 to send email. In this case you must turn off any security or authentication options for SMTP, but leave them on for IMAP or POP. Other cases ----------- The preceding sections cover email sent manually by individual users. It is also common for email to be sent in a way which is not manual, or which is not associated with a particular individual. This includes: * Email from college or department email servers which provide service to many people; * Email sent from a form on a web server, or some other multi-user application; * Email sent by automated jobs, such from the "cron" program on Unix; * Email sent by a mailing list system (whether in large quantities or not). Some examples from the Computing Service's own systems include email from cus.cam.ac.uk; email from the help-desk ticket system; nightly maintenance jobs on many of our computers; and lists.cam.ac.uk. In all these cases, email should be sent via the smart host ppsw.cam.ac.uk port 25, unless arrangements have been made to send directly to the public Internet. If the volume of email from a computer is likely to be large - peak rates of more than 60 messages per hour - you should contact us so that we can ensure that our rate limiting system doesn't interfere. For more information, see http://www.cam.ac.uk/cs/...TBD... Restrictions on sending email ----------------------------- Although we don't restrict your choice of email address, the central email systems are quite strict about it being correct. This is to ensure that if there is an error it is detected as soon as possible, and that if a message cannot be delivered its sender gets a failure report. Furthermore, we require that email domains within Cambridge are registered in the DNS with an MX record; a cam.ac.uk host name may not be used in an email address. (See leaflet G14 http://www.cam.ac.uk/cs/docs/eleaflets/g14/) We are happy to provide advice on configuring your software to work with these rules. The CUDN routers block port 25, which is the port used to send email between organizations across the Internet. This means that computers on the CUDN cannot send email directly to computers outside the University, but must instead send it via a message submission server or smart host as described in the preceding sections. This block helps us to ensure that email from the University is of a reasonably good quality, from the point of view of conformance to technical standards. In particular, it makes it considerably harder for insecure computers on the CUDN to be exploited to send spam or viruses. There is also a mechanism to limit the rate of outgoing email in order to protect the central email systems against the consequences of spews of junk email sent through smtp.hermes.cam.ac.uk or ppsw.cam.ac.uk. Although this mechanism should not affect normal email, it may affect bulk email, so you should contact us if you need to send a lot of email. We will be happy to adjust the limits to accommodate your requirements. There is more information about the rate limiting system at http://www.cam.ac.uk/cs/...TBD... Further information ------------------- Email software settings for Hermes http://www.cam.ac.uk/cs/email/muasettings.html Withdrawal of insecure access to Hermes http://www.cam.ac.uk/cs/email/securehermes.html Rules for administering a mail domain http://www.cam.ac.uk/cs/docs/eleaflets/g14/ Managed mail domains http://www.cam.ac.uk/cs/docs/eleaflets/g25/ Bulk email and rate limiting http://www.cam.ac.uk/cs/...TBD... ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Bulk email and rate limiting ============================ Introduction ------------ The IT syndicate has issued some guidelines on the use of bulk email in Cambridge, which state that you should seek advice if you need to send a large mailshot. This document explains why in more detail. http://www.cam.ac.uk/cs/itsyndicate/bulkemail.html Both ppsw.cam.ac.uk and smtp.hermes.cam.ac.uk have a system to limit the rate at which email can be sent. This is to protect us against the bad effects of an insecure computer on the CUDN or a stolen Hermes username and password being exploited by a spammer. If a flood of junk email is sent through the central email systems from a compromised machine or Hermes account, the whole University may be blacklisted. This has happened in the past and it is very inconvenient, so we would like to reduce the chance of it happening again. At the same time, we do not want to restrict the legitimate use of email. Fortunately we can set generous rate limits that do not affect normal email use, and still significantly throttle unexpected floods. Therefore the rate limiting system should not be noticed by most users. However there are a number of cases in which it is necessary for a user or computer to send large amounts of email. The rest of this section explains how we accommodate this. Configuration requirements -------------------------- Computers that send large amounts of email are typically either email servers in their own right, or they are sending mailshots using mailing list software. As such they should be configured to send email via ppsw.cam.ac.uk, *not* smtp.hermes.cam.ac.uk (which cannot accommodate bulk email). There is more information about the choice of outgoing SMTP server at http://www.cam.ac.uk/cs/...TBD... If you are going to send large volumes of email from a computer that does not normally do so - a large volume being peak rates of more than 60 messages in an hour - you should contact us so that we can ensure that our rate limiting system doesn't interfere. There is a default maximum rate for each computer which sends email through ppsw.cam.ac.uk, and we maintain a list of exceptional computers which are able to send at higher rates than the default. We need to be informed if you are going to send lots of email from a computer which is not on this list. Note that the limit is the number of messages, not the number of recipients. If you are running a mailing list and you do not need to customize the messages for each recipient, then you can send a few messages with a large number of recipients each (e.g. using BCC) in order to avoid hitting the rate limit. You might consider using lists.cam.ac.uk which works in this way; see leaflet G90 for details. http://www.cam.ac.uk/cs/docs/leaflets/g90/ Because spammers tend to time their attacks when the email systems are running unattended, we generally would not find out about an attack until it was too late to do anything about it. Therefore the rate limiting system restricts floods of email without manual intervention. This is why we recommend that you give us advance notice. However, we will not add NAT firewalls to the exception list, because that would weaken the security model. We want to reduce the likelihood that a compromised machine is able to send lots of email, which means minimizing the number of machines on the exception list. Computers that need to send lots of email must have their own IP addresses. What happens to fast senders ---------------------------- If a system sends email faster than its permitted rate, then ppswitch will temporarily refuse messages with an SMTP 450 reply, which indicates that the sending system should try again later. Some messages will still be accepted, such that the overall rate of successfully sent messages is no more than the maximum - that is, over-limit computers are not completely locked out, so messages will get through eventually as they are re-tried. However, some email software does not implement the queue-and-retry part of SMTP. This is particularly common with MUA software, which also has a reputation for poor error handling. So although the rate limiting system is designed not to lose email, it unavoidably assumes that the sending software has a reasonably complete and competent SMTP implementation. Of course, this problem doesn't exist if we are informed in advance so that we can configure the rate limits to be greater than usual. Our auditing software notifies us if a rate limit is reached, in which case we will check whether it is the result of an attack or unexpected legitimate bulk email. In the former case we will notify CERT, and in the latter case we will get in touch with the relevant people so that we can configure our systems appropriately for future mailshots. How rates are measured ---------------------- On smtp.hermes.cam.ac.uk the rate limiting system uses the authenticated username to identify the sender, so the rate it measures for a particular user includes all computers which they use to send email, including webmail. This is because we are defending against the theft of a username and password, rather than the compromise of any particular computer. Every user has the same limit. On ppsw.cam.ac.uk the rate limiting system uses the IP address of the client computer, because in this case we are defending against security problems on individual machines. There is a default limit per IP address, and an exception list of computers that are allowed to send at higher rates. Note that this means that all the machines behind a NAT gateway that send via ppsw.cam.ac.uk are contributing to the same rate measurement. Therefore, machines that send a lot of email should have their own IP addresses. (Hermes users behind a NAT will have their own individual rate measurements based on their username.) Measured rates are averaged over a period of an hour, which allows the limiter to accommodate short high-speed bursts. The maximum number of messages per hour is at least 60 (the exact number is subject to change); this is also the maximum number of messages in a high speed burst. (The mathematical model is an exponentially-weighted moving average, so your rate an hour ago contributes at most 37% to your current measured rate, and any activity more than four hours ago is effectively forgotten.) There is also a daily limit which is less than 24 times the hourly limit; this acts as a back-stop to protect against floods that last several hours (e.g. over a weekend). Further information ------------------- Sending email from computers on the CUDN http://www.cam.ac.uk/cs/...TBD... Guidelines on Use of Bulk Email in Cambridge http://www.cam.ac.uk/cs/itsyndicate/bulkemail.html User-driven mailing lists http://www.cam.ac.uk/cs/docs/leaflets/g90/