unchanged: --- exim-doc/doc-txt/ChangeLog 23 May 2005 15:28:37 -0000 1.140 +++ exim-doc/doc-txt/ChangeLog 23 May 2005 16:58:55 -0000 1.141 @@ -1,4 +1,4 @@ -$Cambridge: exim/exim-doc/doc-txt/ChangeLog,v 1.140 2005/05/23 15:28:37 fanf2 Exp $ +$Cambridge: exim/exim-doc/doc-txt/ChangeLog,v 1.141 2005/05/23 16:58:55 fanf2 Exp $ Change log file for Exim from version 4.21 ------------------------------------------- @@ -34,6 +34,8 @@ in the header line. TF/03 Added the control = fakedefer ACL modifier. + +TF/04 Added the ratelimit ACL condition. See NewStuff for details. Exim version 4.51 diff -u exim-doc/doc-txt/NewStuff exim-doc/doc-txt/NewStuff --- exim-doc/doc-txt/NewStuff 23 May 2005 16:58:55 -0000 1.45 +++ exim-doc/doc-txt/NewStuff 25 May 2005 09:58:16 -0000 1.46 @@ -1,4 +1,4 @@ -$Cambridge: exim/exim-doc/doc-txt/NewStuff,v 1.44 2005/05/23 15:44:06 fanf2 Exp $ +$Cambridge: exim/exim-doc/doc-txt/NewStuff,v 1.46 2005/05/25 09:58:16 fanf2 Exp $ New Features in Exim -------------------- @@ -120,6 +120,120 @@ You must take care when using fakedefer because it will cause messages to be duplicated when the sender retries. Therefore you should not use fakedefer if the message will be delivered normally. + +TF/04 There is a new ratelimit ACL condition which can be used to measure + and control the rate at which clients can send email. This is more + powerful than the existing smtp_ratelimit_* options, because those + options only control the rate of commands in a single SMTP session, + whereas the new ratelimit condition works across all connections + (concurrent and sequential) to the same host. + + The syntax of the ratelimit condition is: + + ratelimit = /

/ / + + If the average client sending rate is less than m messages per time + period p then the condition is false, otherwise it is true. + + The parameter p is the smoothing time constant, in the form of an Exim + time interval e.g. 8h for eight hours. A larger time constant means it + takes Exim longer to forget a client's past behaviour. The parameter m is + the maximum number of messages that a client can send in a fast burst. By + increasing both m and p but keeping m/p constant, you can allow a client + to send more messages in a burst without changing its overall sending + rate limit. Conversely, if m and p are both small then messages must be + sent at an even rate. + + The key is used to look up the data used to calcluate the client's + average sending rate. This data is stored in a database maintained by + Exim in its spool directory alongside the retry database etc. For + example, you can limit the sending rate of each authenticated user, + independent of the computer they are sending from, by setting the key + to $authenticated_id. The default key is $sender_host_address. + + Each ratelimit condition can have up to two options. The first option + specifies what Exim measures the rate of, and the second specifies how + Exim handles excessively fast clients. + + The per_mail option means that it measures the client's rate of sending + messages. This is the default if none of the per_* options is specified. + + The per_conn option means that it measures the client's connection rate. + + The per_byte option limits the sender's email bandwidth. Note that it + is best to use this option in the DATA ACL; if it is used in an earlier + ACL it relies on the SIZE parameter on the MAIL command, which may be + inaccurate or completely missing. You can follow the limit m in the + configuration with K, M, or G to specify limits in kilobytes, + megabytes, or gigabytes respectively. + + The per_cmd option means that Exim recomputes the rate every time the + condition is processed, which can be used to limit the SMTP command rate. + The alias per_rcpt is provided for use in the RCPT ACL instead of per_cmd + to make it clear that the effect is to limit the rate at which recipients + are accepted. Note that in this case the rate limiting engine will see a + message with many recipients as a large high-speed burst. + + If a client's average rate is greater than the maximum, the rate + limiting engine can react in two possible ways, depending on the + presence of the strict or leaky options. This is independent of the + other counter-measures (e.g. rejecting the message) that may be + specified by the rest of the ACL. The default mode is leaky, which + avoids a sender's over-aggressive retry rate preventing it from getting + any email through. + + The strict option means that the client's recorded rate is always + updated. The effect of this is that Exim measures the client's average + rate of attempts to send email, which can be much higher than the + maximum. If the client is over the limit it will be subjected to + counter-measures until it slows down below the maximum rate. + + The leaky option means that the client's recorded rate is not updated + if it is above the limit. The effect of this is that Exim measures the + client's average rate of successfully sent email, which cannot be + greater than the maximum. If the client is over the limit it will + suffer some counter-measures, but it will still be able to send email + at the configured maximum rate, whatever the rate of its attempts. + + As a side-effect, the ratelimit condition will set the expansion + variables $sender_rate containing the client's computed rate, + $sender_rate_limit containing the configured value of m, and + $sender_rate_period containing the configured value of p. + + Exim's other ACL facilities are used to define what counter-measures + are taken when the rate limit is exceeded. This might be anything from + logging a warning (e.g. while measuring existing sending rates in order + to define our policy), through time delays to slow down fast senders, + up to rejecting the message. For example, + + # Log all senders' rates + warn + ratelimit = 0 / 1h / strict + log_message = \ + Sender rate $sender_rate > $sender_rate_limit / $sender_rate_period + + # Slow down fast senders + warn + ratelimit = 100 / 1h / per_rcpt / strict + delay = ${eval: 10 * ($sender_rate - $sender_rate_limit) } + + # Keep authenticated users under control + deny + ratelimit = 100 / 1d / strict / $authenticated_id + + # System-wide rate limit + defer + message = Sorry, too busy. Try again later. + ratelimit = 10 / 1s / $primary_hostname + + # Restrict incoming rate from each host, with a default rate limit + # set using a macro and special cases looked up in a table. + defer + message = Sender rate $sender_rate exceeds \ + $sender_rate_limit messages per $sender_rate_period + ratelimit = ${lookup {$sender_host_address} \ + cdb {DB/ratelimits.cdb} \ + {$value} {RATELIMIT} } Version 4.51 unchanged: --- exim-src/OS/Makefile-AIX 16 Feb 2005 16:40:22 -0000 1.2 +++ exim-src/OS/Makefile-AIX 23 May 2005 16:58:55 -0000 1.3 @@ -1,4 +1,4 @@ -# $Cambridge: exim/exim-src/OS/Makefile-AIX,v 1.2 2005/02/16 16:40:22 ph10 Exp $ +# $Cambridge: exim/exim-src/OS/Makefile-AIX,v 1.3 2005/05/23 16:58:55 fanf2 Exp $ # Exim: OS-specific make file for AIX # Written by Nick Waterman (nick@cimio.co.uk) @@ -25,6 +25,6 @@ # Needed for vfork() and vfork() only? -LIBS = -lbsd +LIBS = -lbsd -lm # End unchanged: --- exim-src/OS/Makefile-BSDI 6 Oct 2004 15:07:39 -0000 1.1 +++ exim-src/OS/Makefile-BSDI 23 May 2005 16:58:55 -0000 1.2 @@ -1,4 +1,4 @@ -# $Cambridge: exim/exim-src/OS/Makefile-BSDI,v 1.1 2004/10/06 15:07:39 ph10 Exp $ +# $Cambridge: exim/exim-src/OS/Makefile-BSDI,v 1.2 2005/05/23 16:58:55 fanf2 Exp $ # Exim: OS-specific make file for BSDI. Its antique link editor # cannot handle the TextPop overriding. @@ -13,7 +13,7 @@ XLFLAGS=-L$(X11)/lib X11_LD_LIB=$(X11)/lib -LIBS_EXIMON=-lSM -lICE -lipc +LIBS_EXIMON=-lSM -lICE -lipc -lm EXIMON_TEXTPOP= EXIWHAT_PS_ARG=-ax unchanged: --- exim-src/OS/Makefile-CYGWIN 10 Nov 2004 10:36:48 -0000 1.2 +++ exim-src/OS/Makefile-CYGWIN 23 May 2005 16:58:55 -0000 1.3 @@ -1,4 +1,4 @@ -# $Cambridge: exim/exim-src/OS/Makefile-CYGWIN,v 1.2 2004/11/10 10:36:48 ph10 Exp $ +# $Cambridge: exim/exim-src/OS/Makefile-CYGWIN,v 1.3 2005/05/23 16:58:55 fanf2 Exp $ # OS-specific file for Cygwin. @@ -6,7 +6,7 @@ HAVE_ICONV = yes CFLAGS= -g -Wall -O2 -LIBS= -lcrypt -lresolv +LIBS= -lcrypt -lresolv -lm LIBS_EXIM= -liconv EXIWHAT_PS_ARG=-as EXIWHAT_KILL_SIGNAL=-USR1 unchanged: --- exim-src/OS/Makefile-DGUX 6 Oct 2004 15:07:39 -0000 1.1 +++ exim-src/OS/Makefile-DGUX 23 May 2005 16:58:55 -0000 1.2 @@ -1,4 +1,4 @@ -# $Cambridge: exim/exim-src/OS/Makefile-DGUX,v 1.1 2004/10/06 15:07:39 ph10 Exp $ +# $Cambridge: exim/exim-src/OS/Makefile-DGUX,v 1.2 2005/05/23 16:58:55 fanf2 Exp $ # Exim: OS-specific make file for DGUX # @@ -25,7 +25,7 @@ CFLAGS=-O2 RANLIB=@true -LIBS=-lsocket -lnsl +LIBS=-lsocket -lnsl -lm LIBRESOLV=-lresolv DBMLIB=-ldbm unchanged: --- exim-src/OS/Makefile-FreeBSD 6 Oct 2004 15:07:39 -0000 1.1 +++ exim-src/OS/Makefile-FreeBSD 23 May 2005 16:58:55 -0000 1.2 @@ -1,4 +1,4 @@ -# $Cambridge: exim/exim-src/OS/Makefile-FreeBSD,v 1.1 2004/10/06 15:07:39 ph10 Exp $ +# $Cambridge: exim/exim-src/OS/Makefile-FreeBSD,v 1.2 2005/05/23 16:58:55 fanf2 Exp $ # Exim: OS-specific make file for FreeBSD # There's no setting of CFLAGS here, to allow the system default @@ -11,7 +11,7 @@ HAVE_SA_LEN=YES # crypt() is in a separate library -LIBS=-lcrypt +LIBS=-lcrypt -lm # FreeBSD always ships with Berkeley DB USE_DB=yes unchanged: --- exim-src/OS/Makefile-GNU 12 Jan 2005 12:25:56 -0000 1.3 +++ exim-src/OS/Makefile-GNU 23 May 2005 16:58:55 -0000 1.4 @@ -1,4 +1,4 @@ -# $Cambridge: exim/exim-src/OS/Makefile-GNU,v 1.3 2005/01/12 12:25:56 ph10 Exp $ +# $Cambridge: exim/exim-src/OS/Makefile-GNU,v 1.4 2005/05/23 16:58:55 fanf2 Exp $ # Exim: OS-specific make file for GNU and variants. @@ -13,7 +13,7 @@ DBMLIB = -ldb USE_DB = yes -LIBS = -lnsl -lcrypt +LIBS = -lnsl -lcrypt -lm LIBRESOLV = -lresolv X11=/usr/X11R6 unchanged: --- exim-src/OS/Makefile-GNUkFreeBSD 4 Jan 2005 10:25:58 -0000 1.1 +++ exim-src/OS/Makefile-GNUkFreeBSD 23 May 2005 16:58:55 -0000 1.2 @@ -1,4 +1,4 @@ -# $Cambridge: exim/exim-src/OS/Makefile-GNUkFreeBSD,v 1.1 2005/01/04 10:25:58 ph10 Exp $ +# $Cambridge: exim/exim-src/OS/Makefile-GNUkFreeBSD,v 1.2 2005/05/23 16:58:55 fanf2 Exp $ # Exim: OS-specific make file for GNU and variants. @@ -13,7 +13,7 @@ DBMLIB = -ldb USE_DB = yes -LIBS = -lnsl -lcrypt +LIBS = -lnsl -lcrypt -lm LIBRESOLV = -lresolv X11=/usr/X11R6 unchanged: --- exim-src/OS/Makefile-GNUkNetBSD 4 Jan 2005 10:25:58 -0000 1.1 +++ exim-src/OS/Makefile-GNUkNetBSD 23 May 2005 16:58:55 -0000 1.2 @@ -1,4 +1,4 @@ -# $Cambridge: exim/exim-src/OS/Makefile-GNUkNetBSD,v 1.1 2005/01/04 10:25:58 ph10 Exp $ +# $Cambridge: exim/exim-src/OS/Makefile-GNUkNetBSD,v 1.2 2005/05/23 16:58:55 fanf2 Exp $ # Exim: OS-specific make file for GNU and variants. @@ -13,7 +13,7 @@ DBMLIB = -ldb USE_DB = yes -LIBS = -lnsl -lcrypt +LIBS = -lnsl -lcrypt -lm LIBRESOLV = -lresolv X11=/usr/X11R6 unchanged: --- exim-src/OS/Makefile-IRIX 6 Oct 2004 15:07:39 -0000 1.1 +++ exim-src/OS/Makefile-IRIX 23 May 2005 16:58:55 -0000 1.2 @@ -1,4 +1,4 @@ -# $Cambridge: exim/exim-src/OS/Makefile-IRIX,v 1.1 2004/10/06 15:07:39 ph10 Exp $ +# $Cambridge: exim/exim-src/OS/Makefile-IRIX,v 1.2 2005/05/23 16:58:55 fanf2 Exp $ # Exim: OS-specific make file for IRIX @@ -6,7 +6,7 @@ BASENAME_COMMAND=/sbin/basename HOSTNAME_COMMAND=/usr/bsd/hostname CFLAGS=-OPT:Olimit=1500 -LIBS=-lmld +LIBS=-lmld -lm XINCLUDE=-I/usr/include/X11 vfork=fork RANLIB=@true unchanged: --- exim-src/OS/Makefile-IRIX6 6 Oct 2004 15:07:39 -0000 1.1 +++ exim-src/OS/Makefile-IRIX6 23 May 2005 16:58:55 -0000 1.2 @@ -1,4 +1,4 @@ -# $Cambridge: exim/exim-src/OS/Makefile-IRIX6,v 1.1 2004/10/06 15:07:39 ph10 Exp $ +# $Cambridge: exim/exim-src/OS/Makefile-IRIX6,v 1.2 2005/05/23 16:58:55 fanf2 Exp $ # Exim: OS-specific make file for IRIX6 on 64-bit systems @@ -6,7 +6,7 @@ HOSTNAME_COMMAND=/usr/bsd/hostname CFLAGS=-O2 -n32 -OPT:Olimit=4000 LFLAGS=-n32 -LIBS=-lelf +LIBS=-lelf -lm XINCLUDE=-I/usr/include/X11 XLFLAGS= vfork=fork unchanged: --- exim-src/OS/Makefile-IRIX632 6 Oct 2004 15:07:39 -0000 1.1 +++ exim-src/OS/Makefile-IRIX632 23 May 2005 16:58:55 -0000 1.2 @@ -1,4 +1,4 @@ -# $Cambridge: exim/exim-src/OS/Makefile-IRIX632,v 1.1 2004/10/06 15:07:39 ph10 Exp $ +# $Cambridge: exim/exim-src/OS/Makefile-IRIX632,v 1.2 2005/05/23 16:58:55 fanf2 Exp $ # Exim: OS-specific make file for IRIX 6 on 32-bit systems. # There seems to be some variation. The commented settings show @@ -10,7 +10,7 @@ CFLAGS=-32 LFLAGS=-32 #LIBS=-lmld -LIBS=-lelf +LIBS=-lelf -lm XINCLUDE=-I/usr/include/X11 vfork=fork RANLIB=@true unchanged: --- exim-src/OS/Makefile-IRIX65 6 Oct 2004 15:07:39 -0000 1.1 +++ exim-src/OS/Makefile-IRIX65 23 May 2005 16:58:55 -0000 1.2 @@ -1,4 +1,4 @@ -# $Cambridge: exim/exim-src/OS/Makefile-IRIX65,v 1.1 2004/10/06 15:07:39 ph10 Exp $ +# $Cambridge: exim/exim-src/OS/Makefile-IRIX65,v 1.2 2005/05/23 16:58:55 fanf2 Exp $ # Exim: OS-specific make file for IRIX 6.5 @@ -10,7 +10,7 @@ LFLAGS=-Wl,-LD_MSG:off=85 LFLAGS= # nlist has moved from libmld to libelf -LIBS=-lelf +LIBS=-lelf -lm XINCLUDE=-I/usr/include/X11 vfork=fork RANLIB=@true unchanged: --- exim-src/OS/Makefile-Linux 6 Oct 2004 15:07:39 -0000 1.1 +++ exim-src/OS/Makefile-Linux 23 May 2005 16:58:55 -0000 1.2 @@ -1,4 +1,4 @@ -# $Cambridge: exim/exim-src/OS/Makefile-Linux,v 1.1 2004/10/06 15:07:39 ph10 Exp $ +# $Cambridge: exim/exim-src/OS/Makefile-Linux,v 1.2 2005/05/23 16:58:55 fanf2 Exp $ # Exim: OS-specific make file for Linux. This is for modern Linuxes, # which use libc6. @@ -14,7 +14,7 @@ DBMLIB = -ldb USE_DB = yes -LIBS = -lnsl -lcrypt +LIBS = -lnsl -lcrypt -lm LIBRESOLV = -lresolv X11=/usr/X11R6 unchanged: --- exim-src/OS/Makefile-NetBSD 6 Oct 2004 15:07:39 -0000 1.1 +++ exim-src/OS/Makefile-NetBSD 23 May 2005 16:58:55 -0000 1.2 @@ -1,4 +1,4 @@ -# $Cambridge: exim/exim-src/OS/Makefile-NetBSD,v 1.1 2004/10/06 15:07:39 ph10 Exp $ +# $Cambridge: exim/exim-src/OS/Makefile-NetBSD,v 1.2 2005/05/23 16:58:55 fanf2 Exp $ # Exim: OS-specific make file for NetBSD (ELF object format) @@ -7,7 +7,7 @@ HAVE_SA_LEN=YES HAVE_IPV6=YES -LIBS=-lcrypt +LIBS=-lcrypt -lm X11=/usr/X11R6 XINCLUDE=-I$(X11)/include unchanged: --- exim-src/OS/Makefile-NetBSD-a.out 6 Oct 2004 15:07:39 -0000 1.1 +++ exim-src/OS/Makefile-NetBSD-a.out 23 May 2005 16:58:55 -0000 1.2 @@ -1,4 +1,4 @@ -# $Cambridge: exim/exim-src/OS/Makefile-NetBSD-a.out,v 1.1 2004/10/06 15:07:39 ph10 Exp $ +# $Cambridge: exim/exim-src/OS/Makefile-NetBSD-a.out,v 1.2 2005/05/23 16:58:55 fanf2 Exp $ # Exim: OS-specific make file for NetBSD (a.out/COFF object format) @@ -7,7 +7,7 @@ HAVE_SA_LEN=YES HAVE_IPV6=YES -LIBS=-lcrypt +LIBS=-lcrypt -lm X11=/usr/X11R6 XINCLUDE=-I$(X11)/include unchanged: --- exim-src/OS/Makefile-OSF1 6 Oct 2004 15:07:39 -0000 1.1 +++ exim-src/OS/Makefile-OSF1 23 May 2005 16:58:55 -0000 1.2 @@ -1,9 +1,9 @@ -# $Cambridge: exim/exim-src/OS/Makefile-OSF1,v 1.1 2004/10/06 15:07:39 ph10 Exp $ +# $Cambridge: exim/exim-src/OS/Makefile-OSF1,v 1.2 2005/05/23 16:58:55 fanf2 Exp $ # Exim: OS-specific make file for OSF1 CFLAGS=-O -LIBS=-liconv +LIBS=-liconv -lm HAVE_CRYPT16=yes HAVE_ICONV=yes HOSTNAME_COMMAND=/usr/bin/hostname unchanged: --- exim-src/OS/Makefile-OpenUNIX 6 Oct 2004 15:07:39 -0000 1.1 +++ exim-src/OS/Makefile-OpenUNIX 23 May 2005 16:58:55 -0000 1.2 @@ -1,4 +1,4 @@ -# $Cambridge: exim/exim-src/OS/Makefile-OpenUNIX,v 1.1 2004/10/06 15:07:39 ph10 Exp $ +# $Cambridge: exim/exim-src/OS/Makefile-OpenUNIX,v 1.2 2005/05/23 16:58:55 fanf2 Exp $ # Exim: OS-specific make file for OpenUNIX @@ -6,7 +6,7 @@ CFLAGS=-O -I/usr/local/include LFLAGS=-L/usr/local/lib -LIBS=-lsocket -lnsl -lelf -lgen -lresolv +LIBS=-lsocket -lnsl -lelf -lgen -lresolv -lm EXTRALIBS_EXIMON=-lICE -lSM RANLIB=@true unchanged: --- exim-src/OS/Makefile-QNX 6 Oct 2004 15:07:39 -0000 1.1 +++ exim-src/OS/Makefile-QNX 23 May 2005 16:58:55 -0000 1.2 @@ -1,4 +1,4 @@ -# $Cambridge: exim/exim-src/OS/Makefile-QNX,v 1.1 2004/10/06 15:07:39 ph10 Exp $ +# $Cambridge: exim/exim-src/OS/Makefile-QNX,v 1.2 2005/05/23 16:58:55 fanf2 Exp $ # Exim: OS-specific makefile for QNX @@ -21,7 +21,7 @@ RANLIB=@true DBMLIB=-ldb USE_DB=yes -LIBS=-lsocket +LIBS=-lsocket -lm X11=/usr/X11R6 XINCLUDE=-I$(X11)/include unchanged: --- exim-src/OS/Makefile-SCO 6 Oct 2004 15:07:39 -0000 1.1 +++ exim-src/OS/Makefile-SCO 23 May 2005 16:58:55 -0000 1.2 @@ -1,4 +1,4 @@ -# $Cambridge: exim/exim-src/OS/Makefile-SCO,v 1.1 2004/10/06 15:07:39 ph10 Exp $ +# $Cambridge: exim/exim-src/OS/Makefile-SCO,v 1.2 2005/05/23 16:58:55 fanf2 Exp $ # Exim: OS-specific make file for SCO @@ -10,7 +10,7 @@ RANLIB=@true DBMLIB=-lndbm ERRNO_QUOTA=0 -LIBS=-lsocket +LIBS=-lsocket -lm HAVE_ICONV=yes X11=/usr/lib/X11 unchanged: --- exim-src/OS/Makefile-SCO_SV 6 Oct 2004 15:07:39 -0000 1.1 +++ exim-src/OS/Makefile-SCO_SV 23 May 2005 16:58:55 -0000 1.2 @@ -1,4 +1,4 @@ -# $Cambridge: exim/exim-src/OS/Makefile-SCO_SV,v 1.1 2004/10/06 15:07:39 ph10 Exp $ +# $Cambridge: exim/exim-src/OS/Makefile-SCO_SV,v 1.2 2005/05/23 16:58:55 fanf2 Exp $ # Exim: OS-specific make file for SCO_SV release 5 (tested on 5.0.5 & 5.0.5) # (see the UNIX_SV files for SCO 4.2) @@ -11,7 +11,7 @@ CFLAGS=-melf -O3 -m486 LFLAGS=-L/lib -L/usr/lib -L/usr/local/lib -LIBS=-ltinfo -lm -lsocket +LIBS=-ltinfo -lsocket -lm HAVE_ICONV=yes unchanged: --- exim-src/OS/Makefile-SunOS5 6 Oct 2004 15:07:39 -0000 1.1 +++ exim-src/OS/Makefile-SunOS5 23 May 2005 16:58:55 -0000 1.2 @@ -1,4 +1,4 @@ -# $Cambridge: exim/exim-src/OS/Makefile-SunOS5,v 1.1 2004/10/06 15:07:39 ph10 Exp $ +# $Cambridge: exim/exim-src/OS/Makefile-SunOS5,v 1.2 2005/05/23 16:58:55 fanf2 Exp $ # Exim: OS-specific make file for SunOS5 @@ -10,7 +10,7 @@ HOSTNAME_COMMAND=look_for_it RANLIB=@true -LIBS=-lsocket -lnsl -lkstat +LIBS=-lsocket -lnsl -lkstat -lm LIBRESOLV=-lresolv EXIWHAT_MULTIKILL_CMD=pkill unchanged: --- exim-src/OS/Makefile-SunOS5-hal 6 Oct 2004 15:07:39 -0000 1.1 +++ exim-src/OS/Makefile-SunOS5-hal 23 May 2005 16:58:55 -0000 1.2 @@ -1,4 +1,4 @@ -# $Cambridge: exim/exim-src/OS/Makefile-SunOS5-hal,v 1.1 2004/10/06 15:07:39 ph10 Exp $ +# $Cambridge: exim/exim-src/OS/Makefile-SunOS5-hal,v 1.2 2005/05/23 16:58:55 fanf2 Exp $ # Exim: OS-specific make file for SunOS5 on a HAL @@ -11,7 +11,7 @@ LIBIDENTCFLAGS="-KV7 -O -DHAVE_ANSIHEADERS" LIBIDENTNAME=sunos5 RANLIB=@true -LIBS=-lsocket -lnsl -lkstat +LIBS=-lsocket -lnsl -lkstat -lm LIBRESOLV=-lresolv X11=/usr/X11R6 XINCLUDE=-I$(X11)/include unchanged: --- exim-src/OS/Makefile-UNIX_SV 6 Oct 2004 15:07:39 -0000 1.1 +++ exim-src/OS/Makefile-UNIX_SV 23 May 2005 16:58:55 -0000 1.2 @@ -1,4 +1,4 @@ -# $Cambridge: exim/exim-src/OS/Makefile-UNIX_SV,v 1.1 2004/10/06 15:07:39 ph10 Exp $ +# $Cambridge: exim/exim-src/OS/Makefile-UNIX_SV,v 1.2 2005/05/23 16:58:55 fanf2 Exp $ # Exim: OS-specific make file for SCO SVR4.2MP (and maybe Unixware) # @@ -17,7 +17,7 @@ RANLIB=@true DBMLIB=-lgdbm -L/usr/local/lib ERRNO_QUOTA=0 -LIBS=-lsocket -lelf -lgen -lnsl -lresolv +LIBS=-lsocket -lelf -lgen -lnsl -lresolv -lm X11=/usr/lib/X11 XINCLUDE=-I/usr/include/X11 unchanged: --- exim-src/OS/Makefile-USG 6 Oct 2004 15:07:39 -0000 1.1 +++ exim-src/OS/Makefile-USG 23 May 2005 16:58:55 -0000 1.2 @@ -1,4 +1,4 @@ -# $Cambridge: exim/exim-src/OS/Makefile-USG,v 1.1 2004/10/06 15:07:39 ph10 Exp $ +# $Cambridge: exim/exim-src/OS/Makefile-USG,v 1.2 2005/05/23 16:58:55 fanf2 Exp $ # Exim: OS-specific make file for Unixware 2.x # @@ -25,7 +25,7 @@ DBMLIB=-ldb -L/usr/local/lib USE_DB=YES ERRNO_QUOTA=0 -LIBS=-lsocket -lelf -lgen -lnsl -lresolv +LIBS=-lsocket -lelf -lgen -lnsl -lresolv -lm X11=/usr/lib/X11 XINCLUDE=-I/usr/include/X11 unchanged: --- exim-src/OS/Makefile-Unixware7 6 Oct 2004 15:07:39 -0000 1.1 +++ exim-src/OS/Makefile-Unixware7 23 May 2005 16:58:55 -0000 1.2 @@ -1,4 +1,4 @@ -# $Cambridge: exim/exim-src/OS/Makefile-Unixware7,v 1.1 2004/10/06 15:07:39 ph10 Exp $ +# $Cambridge: exim/exim-src/OS/Makefile-Unixware7,v 1.2 2005/05/23 16:58:55 fanf2 Exp $ # Exim: OS-specific make file for Unixware7 # Based on information from James FitzGibbon @@ -17,7 +17,7 @@ HAVE_ICONV=yes -LIBS=-lsocket -lnsl -lelf -lgen -lresolv +LIBS=-lsocket -lnsl -lelf -lgen -lresolv -lm # Removed on the advice of Larry Rosenman # EXTRALIBS=-lwrap unchanged: --- exim-src/OS/Makefile-mips 6 Oct 2004 15:07:39 -0000 1.1 +++ exim-src/OS/Makefile-mips 23 May 2005 16:58:55 -0000 1.2 @@ -1,4 +1,4 @@ -# $Cambridge: exim/exim-src/OS/Makefile-mips,v 1.1 2004/10/06 15:07:39 ph10 Exp $ +# $Cambridge: exim/exim-src/OS/Makefile-mips,v 1.2 2005/05/23 16:58:55 fanf2 Exp $ # Exim: OS-specific make file for RiscOS4bsd @@ -6,7 +6,7 @@ EXIT_FAILURE=1 EXIT_SUCCESS=0 LIBRESOLV=-lresolv -LIBS=-liberty +LIBS=-liberty -lm XINCLUDE=-I/usr/X11R6/include CFLAGS=-O diff -u exim-src/src/acl.c exim-src/src/acl.c --- exim-src/src/acl.c 23 May 2005 16:58:56 -0000 1.34 +++ exim-src/src/acl.c 25 May 2005 09:58:16 -0000 1.35 @@ -1,4 +1,4 @@ -/* $Cambridge: exim/exim-src/src/acl.c,v 1.33 2005/05/23 15:28:38 fanf2 Exp $ */ +/* $Cambridge: exim/exim-src/src/acl.c,v 1.35 2005/05/25 09:58:16 fanf2 Exp $ */ /************************************************* * Exim - an Internet mail transport agent * @@ -63,6 +63,7 @@ #ifdef WITH_CONTENT_SCAN ACLC_MIME_REGEX, #endif + ACLC_RATELIMIT, ACLC_RECIPIENTS, #ifdef WITH_CONTENT_SCAN ACLC_REGEX, @@ -110,6 +111,7 @@ #ifdef WITH_CONTENT_SCAN US"mime_regex", #endif + US"ratelimit", US"recipients", #ifdef WITH_CONTENT_SCAN US"regex", @@ -171,6 +173,7 @@ #ifdef WITH_CONTENT_SCAN TRUE, /* mime_regex */ #endif + TRUE, /* ratelimit */ FALSE, /* recipients */ #ifdef WITH_CONTENT_SCAN TRUE, /* regex */ @@ -227,6 +230,7 @@ #ifdef WITH_CONTENT_SCAN FALSE, /* mime_regex */ #endif + FALSE, /* ratelimit */ FALSE, /* recipients */ #ifdef WITH_CONTENT_SCAN FALSE, /* regex */ @@ -363,6 +367,8 @@ ~(1< 1 || per_byte + per_cmd + per_conn + per_mail > 1) + { + *log_msgptr = US"conflicting options for \"ratelimit\" condition"; + return ERROR; + } + +/* Default option values */ +if (!strict) leaky = TRUE; +if (!per_byte && !per_cmd && !per_conn) per_mail = TRUE; + +/* If there is no explicit key, use the sender_host_address. If there is no +sender_host_address (e.g. -bs or acl_not_smtp) then we simply omit it. */ + +if (!have_key && sender_host_address != NULL) + key = string_sprintf("%s / %s", key, sender_host_address); + +HDEBUG(D_acl) debug_printf("ratelimit condition limit=%.0f period=%.0f key=%s\n", + limit, period, key); + +/* If we are dealing with rate limits per connection, per message, or per byte, +see if we have already computed the rate by looking in the relevant tree. For +per-connection rate limiting, store tree nodes and dbdata in the permanent pool +so that they survive across resets. */ + +anchor = NULL; +old_pool = store_pool; + +if (per_conn) + { + anchor = &ratelimiters_conn; + store_pool = POOL_PERM; + } +if (per_mail || per_byte) + anchor = &ratelimiters_mail; + +if (anchor != NULL && (t = tree_search(*anchor, key)) != NULL) + { + dbd = t->data.ptr; + /* The following few lines duplicate some of the code below. */ + if (dbd->rate < limit) rc = FAIL; + else rc = OK; + store_pool = old_pool; + sender_rate = string_sprintf("%.1f", dbd->rate); + HDEBUG(D_acl) + debug_printf("ratelimit found pre-computed rate %s\n", sender_rate); + return rc; + } + +/* We aren't using a pre-computed rate, so get a previously recorded +rate from the database, update it, and write it back. If there's no +previous rate for this key, create one. */ + +dbm = dbfn_open(US"ratelimit", O_RDWR, &dbblock, TRUE); +if (dbm == NULL) + { + store_pool = old_pool; + sender_rate = NULL; + HDEBUG(D_acl) debug_printf("ratelimit database not available\n"); + *log_msgptr = US"ratelimit database not available"; + return DEFER; + } +dbd = dbfn_read(dbm, key); + +gettimeofday(&tv, NULL); + +if (dbd == NULL) + { + HDEBUG(D_acl) debug_printf("ratelimit initializing new key's data\n"); + dbd = store_get(sizeof(dbdata_ratelimit)); + dbd->time_stamp = tv.tv_sec; + dbd->time_usec = tv.tv_usec; + dbd->rate = 0.0; + } +else + { + /* The smoothed rate is computed using an exponentially weighted moving + average adjusted for variable sampling intervals. The standard EWMA for + a fixed sampling interval is: f'(t) = (1 - a) * f(t) + a * f'(t - 1) + where f() is the measured value and f'() is the smoothed value. + + Old data decays out of the smoothed value exponentially, such that data n + samples old is multiplied by a^n. The exponential decay time constant p + is defined such that data p samples old is multiplied by 1/e, which means + that a = exp(-1/p). We can maintain the same time constant for a variable + sampling interval i by using a = exp(-i/p). + + The rate we are measuring is messages per period, suitable for directly + comparing with the limit. The average rate between now and the previous + message is period / interval, which we feed into the EWMA as the sample. + + It turns out that the number of messages required for the smoothed rate + to reach the limit when they are sent in a burst is equal to the limit. + This can be seen by analysing the value of the smoothed rate after N + messages sent at even intervals. Let k = (1 - a) * p/i + + rate_1 = (1 - a) * p/i + a * rate_0 + = k + a * rate_0 + rate_2 = k + a * rate_1 + = k + a * k + a^2 * rate_0 + rate_3 = k + a * k + a^2 * k + a^3 * rate_0 + rate_N = rate_0 * a^N + k * SUM(x=0..N-1)(a^x) + = rate_0 * a^N + k * (1 - a^N) / (1 - a) + = rate_0 * a^N + p/i * (1 - a^N) + + When N is large, a^N -> 0 so rate_N -> p/i as desired. + + rate_N = p/i + (rate_0 - p/i) * a^N + a^N = (rate_N - p/i) / (rate_0 - p/i) + N * -i/p = log((rate_N - p/i) / (rate_0 - p/i)) + N = p/i * log((rate_0 - p/i) / (rate_N - p/i)) + + Numerical analysis of the above equation, setting the computed rate to + increase from rate_0 = 0 to rate_N = limit, shows that for large sending + rates, p/i, the number of messages N = limit. So limit serves as both the + maximum rate measured in messages per period, and the maximum number of + messages that can be sent in a fast burst. */ + + double this_time = (double)tv.tv_sec + + (double)tv.tv_usec / 1000000.0; + double prev_time = (double)dbd->time_stamp + + (double)dbd->time_usec / 1000000.0; + double interval = this_time - prev_time; + + double i_over_p = interval / period; + double a = exp(-i_over_p); + + /* We must avoid division by zero, and deal gracefully with the clock going + backwards. If we blunder ahead when time is in reverse then the computed + rate will become bogusly huge. Clamp i/p to a very small number instead. */ + + if (i_over_p <= 0.0) i_over_p = 1e-9; + + dbd->time_stamp = tv.tv_sec; + dbd->time_usec = tv.tv_usec; + + /* If we are measuring the rate in bytes per period, multiply the + measured rate by the message size. If we don't know the message size + then it's safe to just use a value of zero and let the recorded rate + decay as if nothing happened. */ + + if (per_byte) + dbd->rate = (message_size < 0 ? 0.0 : (double)message_size) + * (1 - a) / i_over_p + a * dbd->rate; + else + dbd->rate = (1 - a) / i_over_p + a * dbd->rate; + } + +/* Clients sending at the limit are considered to be over the limit. This +matters for edge cases such the first message sent by a client (which gets +the initial rate of 0.0) when the rate limit is zero (i.e. the client should +be completely blocked). */ + +if (dbd->rate < limit) rc = FAIL; + else rc = OK; + +/* Update the state if the rate is low or if we are being strict. If we +are in leaky mode and the sender's rate is too high, we do not update +the recorded rate in order to avoid an over-aggressive sender's retry +rate preventing them from getting any email through. */ + +if (rc == FAIL || !leaky) + dbfn_write(dbm, key, dbd, sizeof(dbdata_ratelimit)); +dbfn_close(dbm); + +/* Store the result in the tree for future reference, if necessary. */ + +if (anchor != NULL) + { + t = store_get(sizeof(tree_node) + Ustrlen(key)); + t->data.ptr = dbd; + Ustrcpy(t->name, key); + (void)tree_insertnode(anchor, t); + } + +/* We create the formatted version of the sender's rate very late in +order to ensure that it is done using the correct storage pool. */ + +store_pool = old_pool; +sender_rate = string_sprintf("%.1f", dbd->rate); + +HDEBUG(D_acl) + debug_printf("ratelimit computed rate %s\n", sender_rate); + +return rc; +} + + + +/************************************************* * Handle conditions/modifiers on an ACL item * *************************************************/ @@ -2414,6 +2702,10 @@ rc = mime_regex(&arg); break; #endif + + case ACLC_RATELIMIT: + rc = acl_ratelimit(arg, log_msgptr); + break; case ACLC_RECIPIENTS: rc = match_address_list(addr->address, TRUE, TRUE, &arg, NULL, -1, 0, unchanged: --- exim-src/src/dbstuff.h 4 Jan 2005 10:00:42 -0000 1.2 +++ exim-src/src/dbstuff.h 23 May 2005 16:58:56 -0000 1.3 @@ -1,4 +1,4 @@ -/* $Cambridge: exim/exim-src/src/dbstuff.h,v 1.2 2005/01/04 10:00:42 ph10 Exp $ */ +/* $Cambridge: exim/exim-src/src/dbstuff.h,v 1.3 2005/05/23 16:58:56 fanf2 Exp $ */ /************************************************* * Exim - an Internet mail transport agent * @@ -363,8 +363,8 @@ /* Basic DB type */ typedef struct { - GDBM_FILE gdbm; /* Database */ - datum lkey; /* Last key, for scans */ + GDBM_FILE gdbm; /* Database */ + datum lkey; /* Last key, for scans */ } EXIM_DB; /* Cursor type, not used with gdbm: just set up a dummy */ @@ -629,6 +629,17 @@ /*************/ int count; /* Reserved for possible connection count */ } dbdata_serialize; + + +/* This structure records the information required for the ratelimit +ACL condition. */ + +typedef struct { + time_t time_stamp; + /*************/ + int time_usec; /* Fractional part of time, from gettimeofday() */ + double rate; /* Smoothed sending rate at that time */ +} dbdata_ratelimit; /* End of dbstuff.h */ unchanged: --- exim-src/src/exim.h 10 May 2005 22:39:20 -0000 1.13 +++ exim-src/src/exim.h 23 May 2005 16:58:56 -0000 1.14 @@ -1,4 +1,4 @@ -/* $Cambridge: exim/exim-src/src/exim.h,v 1.13 2005/05/10 22:39:20 tom Exp $ */ +/* $Cambridge: exim/exim-src/src/exim.h,v 1.14 2005/05/23 16:58:56 fanf2 Exp $ */ /************************************************* * Exim - an Internet mail transport agent * @@ -49,6 +49,7 @@ #include #include +#include #include #include #include unchanged: --- exim-src/src/exim_dbutil.c 4 Jan 2005 10:00:42 -0000 1.3 +++ exim-src/src/exim_dbutil.c 23 May 2005 16:58:56 -0000 1.4 @@ -1,4 +1,4 @@ -/* $Cambridge: exim/exim-src/src/exim_dbutil.c,v 1.3 2005/01/04 10:00:42 ph10 Exp $ */ +/* $Cambridge: exim/exim-src/src/exim_dbutil.c,v 1.4 2005/05/23 16:58:56 fanf2 Exp $ */ /************************************************* * Exim - an Internet mail transport agent * @@ -63,10 +63,11 @@ /* Identifiers for the different database types. */ -#define type_retry 1 -#define type_wait 2 -#define type_misc 3 -#define type_callout 4 +#define type_retry 1 +#define type_wait 2 +#define type_misc 3 +#define type_callout 4 +#define type_ratelimit 5 @@ -113,7 +114,7 @@ usage(uschar *name, uschar *options) { printf("Usage: exim_%s%s \n", name, options); -printf(" = retry | misc | wait- | callout\n"); +printf(" = retry | misc | wait- | callout | ratelimit\n"); exit(1); } @@ -135,6 +136,7 @@ if (Ustrcmp(argv[2], "misc") == 0) return type_misc; if (Ustrncmp(argv[2], "wait-", 5) == 0) return type_wait; if (Ustrcmp(argv[2], "callout") == 0) return type_callout; + if (Ustrcmp(argv[2], "ratelimit") == 0) return type_ratelimit; } usage(name, options); return -1; /* Never obeyed */ @@ -536,6 +538,7 @@ dbdata_retry *retry; dbdata_wait *wait; dbdata_callout_cache *callout; + dbdata_ratelimit *ratelimit; int count_bad = 0; int i, length; uschar *t; @@ -662,6 +665,15 @@ } break; + + case type_ratelimit: + ratelimit = (dbdata_ratelimit *)value; + + printf("%s.%06d rate: %10.3f key: %s\n", + print_time(ratelimit->time_stamp), ratelimit->time_usec, + ratelimit->rate, keybuffer); + + break; } store_reset(value); } @@ -733,6 +745,7 @@ dbdata_retry *retry; dbdata_wait *wait; dbdata_callout_cache *callout; + dbdata_ratelimit *ratelimit; int i, oldlength; uschar *t; uschar field[256], value[256]; @@ -873,6 +886,29 @@ break; } break; + + case type_ratelimit: + ratelimit = (dbdata_ratelimit *)value; + switch(fieldno) + { + case 0: + if ((tt = read_time(value)) > 0) ratelimit->time_stamp = tt; + else printf("bad time value\n"); + break; + + case 1: + ratelimit->time_usec = Uatoi(value); + + case 2: + ratelimit->rate = Ustrtod(value, NULL); + break; + + default: + printf("unknown field number\n"); + verify = 0; + break; + } + break; } dbfn_write(dbm, name, record, length); @@ -969,6 +1005,13 @@ printf("2 random: %s (%d)\n", print_cache(callout->random_result), callout->random_result); } + break; + + case type_ratelimit: + ratelimit = (dbdata_ratelimit *)value; + printf("0 time stamp: %s\n", print_time(ratelimit->time_stamp)); + printf("1 fract. time: .%06d\n", ratelimit->time_usec); + printf("2 sender rate: % .3f\n", ratelimit->rate); break; } } unchanged: --- exim-src/src/expand.c 10 May 2005 10:19:11 -0000 1.21 +++ exim-src/src/expand.c 23 May 2005 16:58:56 -0000 1.22 @@ -1,4 +1,4 @@ -/* $Cambridge: exim/exim-src/src/expand.c,v 1.21 2005/05/10 10:19:11 ph10 Exp $ */ +/* $Cambridge: exim/exim-src/src/expand.c,v 1.22 2005/05/23 16:58:56 fanf2 Exp $ */ /************************************************* * Exim - an Internet mail transport agent * @@ -466,6 +466,9 @@ { "sender_host_name", vtype_host_lookup, NULL }, { "sender_host_port", vtype_int, &sender_host_port }, { "sender_ident", vtype_stringptr, &sender_ident }, + { "sender_rate", vtype_stringptr, &sender_rate }, + { "sender_rate_limit", vtype_stringptr, &sender_rate_limit }, + { "sender_rate_period", vtype_stringptr, &sender_rate_period }, { "sender_rcvhost", vtype_stringptr, &sender_rcvhost }, { "sender_verify_failure",vtype_stringptr, &sender_verify_failure }, { "smtp_active_hostname", vtype_stringptr, &smtp_active_hostname }, unchanged: --- exim-src/src/globals.c 23 May 2005 15:28:38 -0000 1.25 +++ exim-src/src/globals.c 23 May 2005 16:58:56 -0000 1.26 @@ -1,4 +1,4 @@ -/* $Cambridge: exim/exim-src/src/globals.c,v 1.25 2005/05/23 15:28:38 fanf2 Exp $ */ +/* $Cambridge: exim/exim-src/src/globals.c,v 1.26 2005/05/23 16:58:56 fanf2 Exp $ */ /************************************************* * Exim - an Internet mail transport agent * @@ -777,6 +777,8 @@ uschar *queue_smtp_domains = NULL; unsigned int random_seed = 0; +tree_node *ratelimiters_conn = NULL; +tree_node *ratelimiters_mail = NULL; uschar *raw_active_hostname = NULL; uschar *raw_sender = NULL; uschar **raw_recipients = NULL; @@ -964,6 +966,9 @@ BOOL sender_host_unknown = FALSE; uschar *sender_ident = NULL; BOOL sender_local = FALSE; +uschar *sender_rate = NULL; +uschar *sender_rate_limit = NULL; +uschar *sender_rate_period = NULL; uschar *sender_rcvhost = NULL; BOOL sender_set_untrusted = FALSE; uschar *sender_unqualified_hosts = NULL; unchanged: --- exim-src/src/globals.h 23 May 2005 15:28:38 -0000 1.17 +++ exim-src/src/globals.h 23 May 2005 16:58:56 -0000 1.18 @@ -1,4 +1,4 @@ -/* $Cambridge: exim/exim-src/src/globals.h,v 1.17 2005/05/23 15:28:38 fanf2 Exp $ */ +/* $Cambridge: exim/exim-src/src/globals.h,v 1.18 2005/05/23 16:58:56 fanf2 Exp $ */ /************************************************* * Exim - an Internet mail transport agent * @@ -505,6 +505,8 @@ extern uschar *queue_smtp_domains; /* Ditto, for these domains */ extern unsigned int random_seed; /* Seed for random numbers */ +extern tree_node *ratelimiters_conn; /* Results of connection ratelimit checks */ +extern tree_node *ratelimiters_mail; /* Results of per-mail ratelimit checks */ extern uschar *raw_active_hostname; /* Pre-expansion */ extern uschar *raw_sender; /* Before rewriting */ extern uschar **raw_recipients; /* Before rewriting */ @@ -579,6 +581,9 @@ extern BOOL sender_host_unknown; /* TRUE for -bs and -bS except inetd */ extern uschar *sender_ident; /* Sender identity via RFC 1413 */ extern BOOL sender_local; /* TRUE for local senders */ +extern uschar *sender_rate; /* Sender rate computed by ACL */ +extern uschar *sender_rate_limit; /* Configured rate limit */ +extern uschar *sender_rate_period; /* Configured smoothing period */ extern uschar *sender_rcvhost; /* Host data for Received: */ extern BOOL sender_set_untrusted; /* Sender set by untrusted caller */ extern uschar *sender_unqualified_hosts; /* Permitted unqualified senders */ unchanged: --- exim-src/src/smtp_in.c 23 May 2005 15:28:38 -0000 1.18 +++ exim-src/src/smtp_in.c 23 May 2005 16:58:56 -0000 1.19 @@ -1,4 +1,4 @@ -/* $Cambridge: exim/exim-src/src/smtp_in.c,v 1.18 2005/05/23 15:28:38 fanf2 Exp $ */ +/* $Cambridge: exim/exim-src/src/smtp_in.c,v 1.19 2005/05/23 16:58:56 fanf2 Exp $ */ /************************************************* * Exim - an Internet mail transport agent * @@ -837,6 +837,10 @@ spf_smtp_comment = NULL; #endif body_linecount = body_zerocount = 0; + +sender_rate = sender_rate_limit = sender_rate_period = NULL; +ratelimiters_mail = NULL; /* Updated by ratelimit ACL condition */ + /* Note that ratelimiters_conn persists across resets. */ for (i = 0; i < ACL_M_MAX; i++) acl_var[ACL_C_MAX + i] = NULL; unchanged: --- exim-src/src/string.c 4 Jan 2005 10:00:42 -0000 1.2 +++ exim-src/src/string.c 23 May 2005 16:58:56 -0000 1.3 @@ -1,4 +1,4 @@ -/* $Cambridge: exim/exim-src/src/string.c,v 1.2 2005/01/04 10:00:42 ph10 Exp $ */ +/* $Cambridge: exim/exim-src/src/string.c,v 1.3 2005/05/23 16:58:56 fanf2 Exp $ */ /************************************************* * Exim - an Internet mail transport agent * @@ -1070,9 +1070,11 @@ break; /* %f format is inherently insecure if the numbers that it may be - handed are unknown (e.g. 1e300). However, in Exim, the only use of %f - is for printing load averages, and these are actually stored as integers - (load average * 1000) so the size of the numbers is constrained. */ + handed are unknown (e.g. 1e300). However, in Exim, %f is used for + printing load averages, and these are actually stored as integers + (load average * 1000) so the size of the numbers is constrained. + It is also used for formatting sending rates, where the simplicity + of the format prevents overflow. */ case 'f': case 'e':