$Cambridge: exim/doc/doc-txt/ChangeLog,v 1.498 2007/03/21 15:15:12 ph10 Exp $ Change log file for Exim from version 4.21 ------------------------------------------- Exim version 4.67 ----------------- MH/01 Fix for bug #448, segfault in Dovecot authenticator when interface_address is unset (happens when testing with -bh and -oMi isn't used). Thanks to Jan Srzednicki. PH/01 Added a new log selector smtp_no_mail, to log SMTP sessions that do not issue a MAIL command. PH/02 In an ACL statement such as deny dnslists = X!=127.0.0.2 : X=127.0.0.2 if a client was not listed at all, or was listed with a value other than 127.0.0.2, in the X list, but was listed with 127.0.0.2 in the Y list, the condition was not true (as it should be), so access was not denied. The bug was that the ! inversion was incorrectly passed on to the second item. This has been fixed. PH/03 Added additional dnslists conditions == and =& which are different from = and & when the dns lookup returns more than one IP address. PH/04 Added gnutls_require_{kx,mac,protocols} to give more control over the cipher suites used by GnuTLS. These options are ignored by OpenSSL. PH/05 After discussion on the list, added a compile time option ENABLE_DISABLE_ FSYNC, which compiles an option called disable_fsync that allows for bypassing fsync(). The documentation is heavily laced with warnings. SC/01 Updated eximstats to collate all SpamAssassin rejects into one bucket. PH/06 Some tidies to the infrastructure of the Test Suite that is concerned with the auxiliary C programs that it uses: (1) Arrange for BIND_8_COMPAT to be defined when compiling on OSX (Darwin); (2) Tidies to the Makefile, including adding "make clean"; (3) Added -fPIC when compiling the test dynamically loaded module, to get rid of a warning. MH/02 Fix for bug #451, causing paniclog entries to be written if a bounce message fails, move_frozen_messages = true and ignore_bounce_errors_after = 0s. The bug is otherwise harmless. PH/07 There was a bug in the dovecot authenticator such that the value of $auth1 could be overwritten, and so not correctly preserved, after a successful authentication. This usually meant that the value preserved by the server_setid option was incorrect. PH/08 Added $smtp_count_at_connection_start, deliberately with a long name. PH/09 Installed PCRE release 7.0. PH/10 The acl_not_smtp_start ACL was, contrary to the documentation, not being run for batched SMTP input. It is now run at the start of every message in the batch. While fixing this I discovered that the process information (output by running exiwhat) was not always getting set for -bs and -bS input. This is fixed, and it now also says "batched" for BSMTP. PH/11 Added control=no_pipelining. PH/12 Added $sending_ip_address and $sending_port (mostly Magnus Holmgren's patch, slightly modified), and move the expansion of helo_data till after the connection is made in the smtp transport (so it can use these values). PH/13 Added ${rfc2047d: to decoded RFC 2047 strings. PH/14 Added log_selector = +pid. PH/15 Flush SMTP output before delaying, unless control=no_delay_flush is set. PH/16 Add ${if forany and ${if forall. PH/17 Added dsn_from option to vary the From: line in DSNs. PH/18 Flush SMTP output before performing a callout, unless control = no_callout_flush is set. PH/19 Change 4.64/PH/36 introduced a bug: when address_retry_include_sender was true (the default) a successful delivery failed to delete the retry item, thus causing premature timeout of the address. The bug is now fixed. PH/20 Added hosts_avoid_pipelining to the smtp transport. PH/21 Long custom messages for fakedefer and fakereject are now split up into multiline reponses in the same way that messages for "deny" and other ACL rejections are. PH/22 Applied Jori Hamalainen's speed-up changes and typo fixes to exigrep, with slight modification. PH/23 Applied sieve patches from the maintainer "tracking the latest notify draft, changing the syntax and factoring some duplicate code". PH/24 When the log selector "outgoing_port" was set, the port was shown as -1 for deliveries of the second and subsequent messages over the same SMTP connection. PH/25 Applied Magnus Holmgren's patch for ${addresses, ${map, ${filter, and ${reduce, with only minor "tidies". SC/02 Applied Daniel Tiefnig's patch to improve the '($parent) =' pattern match. PH/26 Added a "continue" ACL modifier that does nothing, for the benefit of its expansion side effects. PH/27 When a message times out after an over-quota error from an Exim-imposed quota, the bounce message says "mailbox is full". This message was not being given when it was a system quota that was exceeded. It now should be the same. MH/03 Made $recipients available in local_scan(). local_scan() already has better access to the recipient list through recipients_list[], but $recipients can be useful in postmaster-provided expansion strings. PH/28 The $smtp_command and $smtp_command_argument variables were not correct in the case of a MAIL command with additional options following the address, for example: MAIL FROM: SIZE=1234. The option settings were accidentally chopped off. PH/29 SMTP synchronization checks are implemented when a command is read - there is a check that no more input is waiting when there shouldn't be any. However, for some commands, a delay in an ACL can mean that it is some time before the response is written. In this time, more input might arrive, invalidly. So now there are extra checks after an ACL has run for HELO/EHLO and after the predata ACL, and likewise for MAIL and RCPT when pipelining has not been advertised. PH/30 MH's patch to allow iscntrl() characters to be list separators. PH/31 Unlike :fail:, a custom message specified with :defer: was not being returned in the SMTP response when smtp_return_error_details was false. This has been fixed. PH/32 Change the Dovecot authenticator to use read() and write() on the socket instead of the C I/O that was originally supplied, because problems were reported on Solaris. PH/33 Compile failed with OpenSSL 0.9.8e. This was due to a coding error in Exim which did not show up earlier: it was assuming that a call to SSL_CTX_set_info_callback() might give an error value. In fact, there is no error. In previous releases of OpenSSL, SSL_CTX_set_info_callback() was a macro that became an assignment, so it seemed to work. This has changed to a proper function call with a void return, hence the compile error. Exim's code has been fixed. PH/34 Change HDA_SIZE in oracle.c from 256 to 512. This is needed for 64-bit cpus. PH/35 Applied a patch from the Sieve maintainer which fixes a bug in "notify". PH/36 Applied John Jetmore's patch to add -v functionality to exigrep. PH/37 If a message is not accepted after it has had an id assigned (e.g. because it turns out to be too big or there is a timeout) there is no "Completed" line in the log. When some messages of this type were selected by exigrep, they were listed as "not completed". Others were picked up by some special patterns. I have improved the selection criteria to be more general. PH/38 The host_find_failed option in the manualroute router can now be set to "ignore", to completely ignore a host whose IP address cannot be found. If all hosts are ignored, the behaviour is controlled by the new host_all_ignored option. PH/39 In a list of hosts for manualroute, if one item (either because of multi- homing or because of multiple MX records with /mx) generated more than one IP address, and the following item turned out to be the local host, all the secondary addresses of the first item were incorrectly removed from the list, along with the local host and any following hosts (which is what is supposed to happen). PH/40 When Exim receives a message, it writes the login name, uid, and gid of whoever called Exim into the -H file. In the case of the daemon it was behaving confusingly. When first started, it used values for whoever started the daemon, but after a SIGHUP it used the Exim user (because it calls itself on a restart). I have changed the code so that it now always uses the Exim user. PH/41 (Following a suggestion from Tony Finch) If all the RCPT commands in a message are rejected with the same error (e.g. no authentication or bad sender address), and a DATA command is nevertheless sent (as can happen with PIPELINING or a stupid MUA), the error message that was given to the RCPT commands is included in the rejection of the DATA command. This is intended to be helpful for MUAs that show only the final error to their users. PH/42 Another patch from the Sieve maintainer. Exim version 4.66 ----------------- PH/01 Two more bugs that were introduced by 4.64/PH/07, in addition to the one fixed by 4.65/MH/01 (is this a record?) are fixed: (i) An empty string was always treated as zero by the numeric comparison operators. This behaviour has been restored. (ii) It is documented that the numeric comparison operators always treat their arguments as decimal numbers. This was broken in that numbers starting with 0 were being interpreted as octal. While fixing these problems I realized that there was another issue that hadn't been noticed. Values of message_size_limit (both the global option and the transport option) were treated as octal if they started with 0. The documentation was vague. These values are now always treated as decimal, and I will make that clear in the documentation. Exim version 4.65 ----------------- TK/01 Disable default definition of HAVE_LINUX_SENDFILE. Clashes with Linux large file support (_FILE_OFFSET_BITS=64) on older glibc versions. (#438) MH/01 Don't check that the operands of numeric comparison operators are integers when their expansion is in "skipping" mode (fixes bug introduced by 4.64-PH/07). PH/01 If a system filter or a router generates more than SHRT_MAX (32767) child addresses, Exim now panics and dies. Previously, because the count is held in a short int, deliveries were likely to be lost. As such a large number of recipients for a single message is ridiculous (performance will be very, very poor), I have chosen to impose a limit rather than extend the field. Exim version 4.64 ----------------- TK/01 Bugzilla #401. Fix DK spooling code so that it can overwrite a leftover -K file (the existence of which was triggered by #402). While we were at it, introduced process PID as part of the -K filename. This should rule out race conditions when creating these files. TK/02 Bugzilla #402. Apply patch from Simon Arlott, speeding up DK signing processing considerably. Previous code took too long for large mails, triggering a timeout which in turn triggers #401. TK/03 Introduced HAVE_LINUX_SENDFILE to os.h-Linux. Currently only used in the DK code in transports.c. sendfile() is not really portable, hence the _LINUX specificness. TF/01 In the add_headers option to the mail command in an Exim filter, there was a bug that Exim would claim a syntax error in any header after the first one which had an odd number of characters in the field name. PH/01 If a server that rejects MAIL FROM:<> was the target of a sender callout verification, Exim cached a "reject" for the entire domain. This is correct for most verifications, but it is not correct for a recipient verification with use_sender or use_postmaster set, because in that case the callout does not use MAIL FROM:<>. Exim now distinguishes the special case of MAIL FROM:<> rejection from other early rejections (e.g. rejection of HELO). When verifying a recipient using a non-null MAIL address, the cache is ignored if it shows MAIL FROM:<> rejection. Whatever the result of the callout, the value of the domain cache is left unchanged (for any other kind of callout, getting as far as trying RCPT means that the domain itself is ok). PH/02 Tidied a number of unused variable and signed/unsigned warnings that gcc 4.1.1 threw up. PH/03 On Solaris, an unexpectedly close socket (dropped connection) can manifest itself as EPIPE rather than ECONNECT. When tidying away a session, the daemon ignores ECONNECT errors and logs others; it now ignores EPIPE as well. PH/04 Applied Nico Erfurth's refactoring patch to tidy up mime.c (quoted-printable decoding). PH/05 Applied Nico Erfurth's refactoring patch to tidy up spool_mbox.c, and later the small subsequent patch to fix an introduced bug. PH/06 Installed the latest Cygwin Makefile from the Cygwin maintainer. PH/07 There was no check for overflow in expansions such as ${if >{1}{4096M}}. PH/08 An error is now given if message_size_limit is specified negative. PH/09 Applied and tidied up Jakob Hirsch's patch for allowing ACL variables to be given (somewhat) arbitrary names. JJ/01 exipick 20060919.0, allow for arbitrary acl_ variables introduced in 4.64-PH/09. JJ/02 exipick 20060919.0, --show-vars args can now be regular expressions, miscellaneous code fixes PH/10 Added the log_reject_target ACL modifier to specify where to log rejections. PH/11 Callouts were setting the name used for EHLO/HELO from $smtp_active_ hostname. This is wrong, because it relates to the incoming message (and probably the interface on which it is arriving) and not to the outgoing callout (which could be using a different interface). This has been changed to use the value of the helo_data option from the smtp transport instead - this is what is used when a message is actually being sent. If there is no remote transport (possible with a router that sets up host addresses), $smtp_active_hostname is used. PH/12 Installed Andrey Panin's patch to add a dovecot authenticator. Various tweaks were necessary in order to get it to work (see also 21 below): (a) The code assumed that strncpy() returns a negative number on buffer overflow, which isn't the case. Replaced with Exim's string_format() function. (b) There were several signed/unsigned issues. I just did the minimum hacking in of casts. There is scope for a larger refactoring. (c) The code used strcasecmp() which is not a standard C function. Replaced with Exim's strcmpic() function. (d) The code set only $1; it now sets $auth1 as well. (e) A simple test gave the error "authentication client didn't specify service in request". It would seem that Dovecot has changed its interface. Fortunately there's a specification; I followed it and changed what the client sends and it appears to be working now. PH/13 Added $message_headers_raw to provide the headers without RFC 2047 decoding. PH/14 Corrected misleading output from -bv when -v was also used. Suppose the address A is aliased to B and C, where B exists and C does not. Without -v the output is "A verified" because verification stops after a successful redirection if more than one address is generated. However, with -v the child addresses are also verified. Exim was outputting "A failed to verify" and then showing the successful verification for C, with its parentage. It now outputs "B failed to verify", showing B's parentage before showing the successful verification of C. PH/15 Applied Michael Deutschmann's patch to allow DNS black list processing to look up a TXT record in a specific list after matching in a combined list. PH/16 It seems that the options setting for the resolver (RES_DEFNAMES and RES_DNSRCH) can affect the behaviour of gethostbyname() and friends when they consult the DNS. I had assumed they would set it the way they wanted; and indeed my experiments on Linux seem to show that in some cases they do (I could influence IPv6 lookups but not IPv4 lookups). To be on the safe side, however, I have now made the interface to host_find_byname() similar to host_find_bydns(), with an argument containing the DNS resolver options. The host_find_byname() function now sets these options at its start, just as host_find_bydns() does. The smtp transport options dns_qualify_single and dns_search_parents are passed to host_find_byname() when gethostbyname=TRUE in this transport. Other uses of host_find_byname() use the default settings of RES_DEFNAMES (qualify_single) but not RES_DNSRCH (search_parents). PH/17 Applied (a modified version of) Nico Erfurth's patch to make spool_read_header() do less string testing, by means of a preliminary switch on the second character of optional "-foo" lines. (This is overdue, caused by the large number of possibilities that now exist. Originally there were few.) While I was there, I also converted the str(n)cmp tests so they don't re-test the leading "-" and the first character, in the hope this might squeeze out yet more improvement. PH/18 Two problems with "group" syntax in header lines when verifying: (1) The flag allowing group syntax was set by the header_syntax check but not turned off, possible causing trouble later; (2) The flag was not being set at all for the header_verify test, causing "group"-style headers to be rejected. I have now set it in this case, and also caused header_ verify to ignore an empty address taken from a group. While doing this, I came across some other cases where the code for allowing group syntax while scanning a header line wasn't quite right (mostly, not resetting the flag correctly in the right place). These bugs could have caused trouble for malformed header lines. I hope it is now all correct. PH/19 The functions {pwcheck,saslauthd}_verify_password() are always called with the "reply" argument non-NULL. The code, however (which originally came from elsewhere) had *some* tests for NULL when it wrote to *reply, but it didn't always do it. This confused somebody who was copying the code for some other use. I have removed all the tests. PH/20 It was discovered that the GnuTLS code had support for RSA_EXPORT, a feature that was used to support insecure browsers during the U.S. crypto embargo. It requires special client support, and Exim is probably the only MTA that supported it -- and would never use it because real RSA is always available. This code has been removed, because it had the bad effect of slowing Exim down by computing (never used) parameters for the RSA_EXPORT functionality. PH/21 On the advice of Timo Sirainen, added a check to the dovecot authenticator to fail if there's a tab character in the incoming data (there should never be unless someone is messing about, as it's supposed to be base64-encoded). Also added, on Timo's advice, the "secured" option if the connection is using TLS or if the remote IP is the same as the local IP, and the "valid-client-cert option" if a client certificate has been verified. PH/22 As suggested by Dennis Davis, added a server_condition option to *all* authenticators. This can be used for authorization after authentication succeeds. (In the case of plaintext, it servers for both authentication and authorization.) PH/23 Testing for tls_required and lost_connection in a retry rule didn't work if any retry times were supplied. PH/24 Exim crashed if verify=helo was activated during an incoming -bs connection, where there is no client IP address to check. In this situation, the verify now always succeeds. PH/25 Applied John Jetmore's -Mset patch. PH/26 Added -bem to be like -Mset, but loading a message from a file. PH/27 In a string expansion for a processed (not raw) header when multiple headers of the same name were present, leading whitespace was being removed from all of them, but trailing whitespace was being removed only from the last one. Now trailing whitespace is removed from each header before concatenation. Completely empty headers in a concatenation (as before) are ignored. PH/28 Fixed bug in backwards-compatibility feature of PH/09 (thanks to John Jetmore). It would have mis-read ACL variables from pre-4.61 spool files. PH/29 [Removed. This was a change that I later backed out, and forgot to correct the ChangeLog entry (that I had efficiently created) before committing the later change.] PH/30 Exim was sometimes attempting to deliver messages that had suffered address errors (4xx response to RCPT) over the same connection as other messages routed to the same hosts. Such deliveries are always "forced", so retry times are not inspected. This resulted in far too many retries for the affected addresses. The effect occurred only when there were more hosts than the hosts_max_try setting in the smtp transport when it had the 4xx errors. Those hosts that it had tried were not added to the list of hosts for which the message was waiting, so if all were tried, there was no problem. Two fixes have been applied: (i) If there are any address or message errors in an SMTP delivery, none of the hosts (tried or untried) are now added to the list of hosts for which the message is waiting, so the message should not be a candidate for sending over the same connection that was used for a successful delivery of some other message. This seems entirely reasonable: after all the message is NOT "waiting for some host". This is so "obvious" that I'm not sure why it wasn't done previously. Hope I haven't missed anything, but it can't do any harm, as the worst effect is to miss an optimization. (ii) If, despite (i), such a delivery is accidentally attempted, the routing retry time is respected, so at least it doesn't keep hammering the server. PH/31 Installed Andrew Findlay's patch to close the writing end of the socket in ${readsocket because some servers need this prod. PH/32 Added some extra debug output when updating a wait-xxx database. PH/33 The hint "could be header name not terminated by colon", which has been given for certain expansion errors for a long time, was not being given for the ${if def:h_colon_omitted{... case. PH/34 The spec says: "With one important exception, whenever a domain list is being scanned, $domain contains the subject domain." There was at least one case where this was not true. PH/35 The error "getsockname() failed: connection reset by peer" was being written to the panic log as well as the main log, but it isn't really panic-worthy as it just means the connection died rather early on. I have removed the panic log writing for the ECONNRESET error when getsockname() fails. PH/36 After a 4xx response to a RCPT error, that address was delayed (in queue runs only) independently of the message's sender address. This meant that, if the 4xx error was in fact related to the sender, a different message to the same recipient with a different sender could confuse things. In particualar, this can happen when sending to a greylisting server, but other circumstances could also provoke similar problems. I have changed the default so that the retry time for these errors is now based a combination of the sender and recipient addresses. This change can be overridden by setting address_retry_include_sender=false in the smtp transport. PH/37 For LMTP over TCP/IP (the smtp transport), error responses from the remote server are returned as part of bounce messages. This was not happening for LMTP over a pipe (the lmtp transport), but now it is the same for both kinds of LMTP. PH/38 Despite being documented as not happening, Exim was rewriting addresses in header lines that were in fact CNAMEs. This is no longer the case. PH/39 If -R or -S was given with -q