diff options
Diffstat (limited to 'doc/doc-txt/NewStuff')
| -rw-r--r-- | doc/doc-txt/NewStuff | 605 |
1 files changed, 605 insertions, 0 deletions
diff --git a/doc/doc-txt/NewStuff b/doc/doc-txt/NewStuff new file mode 100644 index 000000000..dfcc5e711 --- /dev/null +++ b/doc/doc-txt/NewStuff @@ -0,0 +1,605 @@ +$Cambridge: exim/doc/doc-txt/NewStuff,v 1.1 2004/10/07 15:04:35 ph10 Exp $ + +New Features in Exim +-------------------- + +This file contains descriptions of new features that have been added to Exim, +but have not yet made it into the main manual (which is most conveniently +updated when there is a relatively large batch of changes). The doc/ChangeLog +file contains a listing of all changes, including bug fixes. + + +Version 4.43 +------------ + + 1. There is a new Boolean global option called mua_wrapper, defaulting false. + This causes Exim to run an a restricted mode, in order to provide a very + specific service. + + Background: On a personal computer, it is a common requirement for all + email to be sent to a smarthost. There are plenty of MUAs that can be + configured to operate that way, for all the popular operating systems. + However, there are MUAs for Unix-like systems that cannot be so configured: + they submit messages using the command line interface of + /usr/sbin/sendmail. In addition, utility programs such as cron submit + messages this way. + + Requirement: The requirement is for something that can provide the + /usr/sbin/sendmail interface and deliver messages to a smarthost, but not + provide any queueing or retrying facilities. Furthermore, the delivery to + the smarthost should be synchronous, so that if it fails, the sending MUA + is immediately informed. In other words, we want something that in effect + converts a command-line MUA into a TCP/SMTP MUA. + + Solutions: There are a number of applications (for example, ssmtp) that do + this job. However, people have found them to be lacking in various ways. + For instance, some sites want to allow aliasing and forwarding before + sending to the smarthost. + + Using Exim: Exim already had the necessary infrastructure for doing this + job. Just a few tweaks were needed to make it behave as required, though it + is somewhat of an overkill to use a fully-featured MTA for this purpose. + + Setting mua_wrapper=true causes Exim to run in a special mode where it + assumes that it is being used to "wrap" a command-line MUA in the manner + just described. + + If you set mua_wrapper=true, you also need to provide a compatible router + and transport configuration. Typically there will be just one router and + one transport, sending everything to a smarthost. + + When run in MUA wrapping mode, the behaviour of Exim changes in the + following ways: + + (a) A daemon cannot be run, nor will Exim accept incoming messages from + inetd. In other words, the only way to submit messages is via the + command line. + + (b) Each message is synchonously delivered as soon as it is received (-odi + is assumed). All queueing options (queue_only, queue_smtp_domains, + control=queue, control=freeze in an ACL etc.) are quietly ignored. The + Exim reception process does not finish until the delivery attempt is + complete. If the delivery was successful, a zero return code is given. + + (c) Address redirection is permitted, but the final routing for all + addresses must be to the same remote transport, and to the same list of + hosts. Furthermore, the return_address must be the same for all + recipients, as must any added or deleted header lines. In other words, + it must be possible to deliver the message in a single SMTP + transaction, however many recipients there are. + + (d) If the conditions in (c) are not met, or if routing any address results + in a failure or defer status, or if Exim is unable to deliver all the + recipients successfully to one of the hosts immediately, delivery of + the entire message fails. + + (e) Because no queueing is allowed, all failures are treated as permanent; + there is no distinction between 4xx and 5xx SMTP response codes from + the smarthost. Furthermore, because only a single yes/no response can + be given to the caller, it is not possible to deliver to some + recipients and not others. If there is an error (temporary or + permanent) for any recipient, all are failed. + + (f) If more than one host is listed, Exim will try another host after a + connection failure or a timeout, in the normal way. However, if this + kind of failure happens for all the hosts, the delivery fails. + + (g) When delivery fails, an error message is written to the standard error + stream (as well as to Exim's log), and Exim exits to the caller with a + return code value 1. The message is expunged from Exim's spool files. + No bounce messages are ever generated. + + (h) No retry data is maintained, and any retry rules are ignored. + + (i) A number of Exim options are overridden: deliver_drop_privilege is + forced true, max_rcpt in the smtp transport is forced to "unlimited", + remote_max_parallel is forced to one, and fallback hosts are ignored. + + The overall effect is that Exim makes a single synchronous attempt to + deliver the message, failing if there is any kind of problem. Because no + local deliveries are done and no daemon can be run, Exim does not need root + privilege. It should be possible to run it setuid=exim instead of + setuid=root. See section 48.3 in the 4.40 manual for a general discussion + about the advantages and disadvantages of running without root privilege. + + 2. There have been problems with DNS servers when SRV records are looked up. + Some mis-behaving servers return a DNS error or timeout when a non-existent + SRV record is sought. Similar problems have in the past been reported for + MX records. The global dns_again_means_nonexist option can help with this + problem, but it is heavy-handed because it is a global option. There are + now two new options for the dnslookup router. They are called + srv_fail_domains and mx_fail_domains. In each case, the value is a domain + list. If an attempt to look up an SRV or MX record results in a DNS failure + or "try again" response, and the domain matches the relevant list, Exim + behaves as if the DNS had responded "no such record". In the case of an SRV + lookup, this means that the router proceeds to look for MX records; in the + case of an MX lookup, it proceeds to look for A or AAAA records, unless the + domain matches mx_domains. + + 3. The following functions are now available in the local_scan() API: + + (a) void header_remove(int occurrence, uschar *name) + + This function removes header lines. If "occurrence" is zero or negative, + all occurrences of the header are removed. If occurrence is greater + than zero, that particular instance of the header is removed. If no + header(s) can be found that match the specification, the function does + nothing. + + (b) BOOL header_testname(header_line *hdr, uschar *name, int length, + BOOL notdel) + + This function tests whether the given header has the given name. It + is not just a string comparison, because whitespace is permitted + between the name and the colon. If the "notdel" argument is TRUE, a + FALSE return is forced for all "deleted" headers; otherwise they are + not treated specially. For example: + + if (header_testname(h, US"X-Spam", 6, TRUE)) ... + + (c) void header_add_at_position(BOOL after, uschar *name, BOOL topnot, + int type, char *format, ...) + + This function adds a new header line at a specified point in the header + chain. If "name" is NULL, the new header is added at the end of the + chain if "after" is TRUE, or at the start if "after" is FALSE. If + "name" is not NULL, the headers are searched for the first non-deleted + header that matches the name. If one is found, the new header is added + before it if "after" is FALSE. If "after" is true, the new header is + added after the found header and any adjacent subsequent ones with the + same name (even if marked "deleted"). If no matching non-deleted header + is found, the "topnot" option controls where the header is added. If it + is TRUE, addition is at the top; otherwise at the bottom. Thus, to add + a header after all the Received: headers, or at the top if there are no + Received: headers, you could use + + header_add_at_position(TRUE, US"Received", TRUE, ' ', "X-xxx: ..."); + + Normally, there is always at least one non-deleted Received: header, + but there may not be if received_header_text expands to an empty + string. + + (d) BOOL receive_remove_recipient(uschar *recipient) + + This is a convenience function to remove a named recipient from the + list of recipients. It returns TRUE if a recipient was removed, and + FALSE if no matching recipient could be found. The argument must be a + complete email address. + + 4. When an ACL "warn" statement adds one or more header lines to a message, + they are added at the end of the existing header lines by default. It is + now possible to specify that any particular header line should be added + right at the start (before all the Received: lines) or immediately after + the first block of Received: lines in the message. This is done by + specifying :at_start: or :after_received: (or, for completeness, :at_end:) + before the text of the header line. (Header text cannot start with a colon, + as there has to be a header name first.) For example: + + warn message = :after_received:X-My-Header: something or other... + + If more than one header is supplied in a single warn statement, each one is + treated independently and can therefore be placed differently. If you add + more than one line at the start, or after the Received: block, they will + end up in reverse order. + + Warning: This facility currently applies only to header lines that are + added in an ACL. It does NOT work for header lines that are added in a + system filter or in a router or transport. + + 5. There is now a new error code that can be used in retry rules. Its name is + "rcpt_4xx", and there are three forms. A literal "rcpt_4xx" matches any 4xx + error received for an outgoing SMTP RCPT command; alternatively, either the + first or both of the x's can be given as digits, for example: "rcpt_45x" or + "rcpt_436". If you want (say) to recognize 452 errors given to RCPT + commands by a particular host, and have only a one-hour retry for them, you + can set up a retry rule of this form: + + the.host.name rcpt_452 F,1h,10m + + Naturally, this rule must come before any others that would match. + + These new errors apply to both outgoing SMTP (the smtp transport) and + outgoing LMTP (either the lmtp transport, or the smtp transport in LMTP + mode). Note, however, that they apply only to responses to RCPT commands. + + 6. The "postmaster" option of the callout feature of address verification has + been extended to make it possible to use a non-empty MAIL FROM address when + checking a postmaster address. The new suboption is called "postmaster_ + mailfrom", and you use it like this: + + require verify = sender/callout=postmaster_mailfrom=abc@x.y.z + + Providing this suboption causes the postmaster check to be done using the + given address. The original "postmaster" option is equivalent to + + require verify = sender/callout=postmaster_mailfrom= + + If both suboptions are present, the rightmost one overrides. + + Important notes: + + (1) If you use a non-empty sender address for postmaster checking, there is + the likelihood that the remote host will itself initiate a callout + check back to your host to check that address. As this is a "normal" + callout check, the sender will most probably be empty, thus avoiding + possible callout loops. However, to be on the safe side it would be + best to set up your own ACLs so that they do not do sender verification + checks when the recipient is the address you use for postmaster callout + checking. + + (2) The caching arrangements for postmaster checking do NOT take account of + the sender address. It is assumed that either the empty address, or a + fixed non-empty address will be used. All that Exim remembers is that + the postmaster check for the domain succeeded or failed. + + 7. When verifying addresses in header lines using the verify=header_sender + option, Exim behaves by default as if the addresses are envelope sender + addresses from a message. Callout verification therefore tests to see + whether a bounce message could be delivered, by using an empty address in + the MAIL FROM command. However, it is arguable that these addresses might + never be used as envelope senders, and could therefore justifiably reject + bounce messages (empty senders). There is now an additional callout option + for verify=header_sender that allows you to specify what address to use in + the MAIL FROM command. You use it as in this example: + + require verify = header_sender/callout=mailfrom=abcd@x.y.z + + Important notes: + + (1) As in the case of postmaster_mailfrom (see above), you should think + about possible loops. + + (2) In this case, as in the case of recipient callouts with non-empty + senders (the use_sender option), caching is done on the basis of a + recipient/sender pair. + + 8. If you build Exim with USE_READLINE=yes in Local/Makefile, it will try to + load libreadline dynamically whenever the -be (test expansion) option is + used without command line arguments. If successful, it will then use + readline() for reading the test data. A line history is supported. By the + time Exim does this, it is running as the calling user, so this should not + cause any security problems. Security is the reason why this is NOT + supported for -bt or -bv, when Exim is running as root or exim, + respectively. Note that this option adds to the size of the Exim binary, + because the dynamic loading library is not otherwise included. On my + desktop it adds about 2.5K. You may need to add -ldl to EXTRA_LIBS when you + set USE_READLINE=yes. + + 9. Added ${str2b64:<string>} to the expansion operators. This operator + converts an arbitrary string into one that is base64 encoded. + +10. A new authenticator, called cyrus_sasl, has been added. This requires + the presence of the Cyrus SASL library; it authenticates by calling this + library, which supports a number of authentication mechanisms, including + PLAIN and LOGIN, but also several others that Exim does not support + directly. The code for this authenticator was provided by Matthew + Byng-Maddick of A L Digital Ltd (http://www.aldigital.co.uk). Here follows + draft documentation: + + xx. THE CYRUS_SASL AUTHENTICATOR + + The cyrus_sasl authenticator provides server support for the Cyrus library + Implementation of the RFC 2222 "Simple Authentication and Security Layer". + It provides a gatewaying mechanism directly to the Cyrus interface, so if + your Cyrus library can do, for example, CRAM-MD5, then so can the + cyrus_sasl authenticator. By default it uses the public name of the driver + to determine which mechanism to support. + + Where access to some kind of secret file is required, for example in GSSAPI + or CRAM-MD5, it is worth noting that the authenticator runs as the exim + user, and that the Cyrus SASL library has no way of escalating privileges + by default. You may also find you need to set environment variables, + depending on the driver you are using. + + xx.1 Using cyrus_sasl as a server + + The cyrus_sasl authenticator has four private options. It puts the username + (on a successful authentication) into $1. + + server_hostname Type: string* Default: $primary_hostname + + This option selects the hostname that is used when communicating with + the library. It is up to the underlying SASL plug-in what it does with + this data. + + server_mech Type: string Default: public_name + + This option selects the authentication mechanism this driver should + use. It allows you to use a different underlying mechanism from the + advertised name. For example: + + sasl: + driver = cyrus_sasl + public_name = X-ANYTHING + server_mech = CRAM-MD5 + server_set_id = $1 + + server_realm Type: string Default: unset + + This is the SASL realm that the server is claiming to be in. + + server_service Type: string Default: "smtp" + + This is the SASL service that the server claims to implement. + + For straigthforward cases, you do not need to set any of the + authenticator's private options. All you need to do is to specify an + appropriate mechanism as the public name. Thus, if you have a SASL library + that supports CRAM-MD5 and PLAIN, you might have two authenticators as + follows: + + sasl_cram_md5: + driver = cyrus_sasl + public_name = CRAM-MD5 + server_set_id = $1 + + sasl_plain: + driver = cyrus_sasl + public_name = PLAIN + server_set_id = $1 + +11. There is a new global option called tls_on_connect_ports. Its value must be + a list of port numbers; the most common use is expected to be + + tls_on_connect_ports = 465 + + Setting this option has the same effect as -tls-on-connect on the command + line, but only for the specified ports. It applies to all connections, both + via the daemon and via inetd. You still need to specify all the ports for + the daemon (using daemon_smtp_ports or local_interfaces or the -X command + line option) because this option does not add an extra port -- rather, it + specifies different behaviour on a port that is defined elsewhere. The + -tls-on-connect command line option overrides tls_on_connect_ports, and + forces tls-on-connect for all ports. + +12. There is a new ACL that is run when a DATA command is received, before the + data itself is received. The ACL is defined by acl_smtp_predata. (Compare + acl_smtp_data, which is run after the data has been received.) + This new ACL allows a negative response to be given to the DATA command + itself. Header lines added by MAIL or RCPT ACLs are not visible at this + time, but any that are defined here are visible when the acl_smtp_data ACL + is run. + +13. The "control=submission" ACL modifier has an option "/domain=xxx" which + specifies the domain to be used when creating From: or Sender: lines using + the authenticated id as a local part. If the option is supplied with an + empty domain, that is, just "/domain=", Exim assumes that the authenticated + id is a complete email address, and it uses it as is when creating From: + or Sender: lines. + +14. It is now possible to make retry rules that apply only when the failing + message has a specific sender. In particular, this can be used to define + retry rules that apply only to bounce messages. The syntax is to add a new + third item to a retry rule, of the form "senders=<address list>". The retry + timings themselves then become the fourth item. For example: + + * * senders=: F,1h,30m + + would match all bounce messages. If the address list contains white space, + it must be enclosed in quotes. For example: + + a.domain timeout senders="x@b.dom : y@c.dom" G,8h,10m,1.5 + + When testing retry rules using -brt, you can supply a sender using the -f + command line option, like this: + + exim -f "" -brt user@dom.ain + + If you do not set -f with -brt, a retry rule that contains a senders list + will never be matched. + +15. Two new control modifiers have been added to ACLs: "control = enforce_sync" + and "control = no_enforce_sync". This makes it possible to be selective + about when SMTP synchronization is enforced. The global option + smtp_enforce_sync now specifies the default state of the switch. These + controls can appear in any ACL, but the most obvious place to put them is + in the ACL defined by acl_smtp_connect, which is run at the start of an + incoming SMTP connection, before the first synchronization check. + +16. Another two new control modifiers are "control = caseful_local_part" and + "control = caselower_local_part". These are permitted only in the ACL + specified by acl_smtp_rcpt (i.e. during RCPT processing). By default, the + contents of $local_part are lower cased before ACL processing. + After "control = caseful_local_part", any uppercase letters in the original + local part are restored in $local_part for the rest of the ACL, or until + "control = caselower_local_part" is encountered. However, this applies only + to local part handling that takes place directly in the ACL (for example, + as a key in lookups). If a "verify = recipient" test is obeyed, the + case-related handling of the local part during the verification is + controlled by the router configuration (see the caseful_local_part generic + router option). + + This facility could be used, for example, to add a spam score to local + parts containing upper case letters. For example, using $acl_m4 to + accumulate the spam score: + + warn control = caseful_local_part + set acl_m4 = ${eval:\ + $acl_m4 + \ + ${if match{$local_part}{[A-Z]}{1}{0}}\ + } + control = caselower_local_part + + Notice that we put back the lower cased version afterwards, assuming that + is what is wanted for subsequent tests. + +17. The option hosts_connection_nolog is provided so that certain hosts can be + excepted from logging when the +smtp_connection log selector is set. For + example, you might want not to log SMTP connections from local processes, + or from 127.0.0.1, or from your local LAN. The option is a host list with + an unset default. Because it is consulted in the main loop of the daemon, + you should strive to restrict its value to a short inline list of IP + addresses and networks. To disable logging SMTP connections from local + processes, you must create a host list with an empty item. For example: + + hosts_connection_nolog = : + + If the +smtp_connection log selector is not set, this option has no effect. + +18. There is now an acl called acl_smtp_quit, which is run for the QUIT + command. The outcome of the ACL does not affect the response code to QUIT, + which is always 221. Thus, the ACL does not in fact control any access. + For this reason, the only verbs that are permitted are "accept" and "warn". + + The ACL can be used for tasks such as custom logging at the end of an SMTP + session. For example, you can use ACL variables in other ACLs to count + messages, recipients, etc., and log the totals at QUIT time using one or + more "logwrite" modifiers on a "warn" command. + + You do not need to have a final "accept", but if you do, you can use a + "message" modifier to specify custom text that is sent as part of the 221 + response. + + This ACL is run only for a "normal" QUIT. For certain kinds of disastrous + failure (for example, failure to open a log file, or when Exim is bombing + out because it has detected an unrecoverable error), all SMTP commands + from the client are given temporary error responses until QUIT is received + or the connection is closed. In these special cases, the ACL is not run. + +19. The appendfile transport has two new options, mailbox_size and mailbox_ + filecount. If either these options are set, it is expanded, and the result + is taken as the current size of the mailbox or the number of files in the + mailbox, respectively. This makes it possible to use some external means of + maintaining the data about the size of a mailbox for enforcing quota + limits. The result of expanding these option values must be a decimal + number, optionally followed by "K" or "M". + +20. It seems that there are broken clients in use that cannot handle multiline + SMTP responses. Can't people who implement these braindead programs read? + RFC 821 mentions multiline responses, and it is over 20 years old. They + must handle multiline responses for EHLO, or do they still use HELO? + Anyway, here is YAWFAB (yet another workaround for asinine brokenness). + There's a new ACL switch that can be set by + + control = no_multiline_responses + + If this is set, it suppresses multiline SMTP responses from ACL rejections. + One way of doing this would have been just to put out these responses as + one long line. However, RFC 2821 specifies a maximum of 512 bytes per + response ("use multiline responses for more" it says), and some of the + responses might get close to that. So I have implemented this by doing two + very easy things: + + (1) Extra information that is normally output as part of a rejection + caused by sender verification failure is omitted. Only the final line + (typically "sender verification failed") is now sent. + + (2) If a "message" modifier supplies a multiline response, only the first + line is output. + + The setting of the switch can, of course, be made conditional on the + calling host. + +21. There is now support for the libradius library that comes with FreeBSD. + This is an alternative to the radiusclient library that Exim already + supports. To use the FreeBSD library, you need to set + + RADIUS_LIB_TYPE=RADLIB + + in Local/Makefile, in addition to RADIUS_CONFIGURE_FILE, and you probably + also need -libradius in EXTRALIBS. + + +Version 4.42 +------------ + + 1. The "personal" filter test is brought up-to-date with recommendations from + the Sieve specification: (a) The list of non-personal From: addresses now + includes "listserv", "majordomo", and "*-request"; (b) If the message + contains any header line starting with "List=-" it is treated as + non-personal. + + 2. The Sieve functionality has been extended to support the "copy" and + "vacation" extensions, and comparison tests. + + 3. There is now an overall timeout for performing a callout verification. It + defaults to 4 times the callout timeout, which applies to individual SMTP + commands during the callout. The overall timeout applies when there is more + than one host that can be tried. The timeout is checked before trying the + next host. This prevents very long delays if there are a large number of + hosts and all are timing out (e.g. when the network connections are timing + out). The value of the overall timeout can be changed by specifying an + additional sub-option for "callout", called "maxwait". For example: + + verify = sender/callout=5s,maxwait=20s + + 4. Changes to the "personal" filter test: + + (1) The list of non-personal local parts in From: addresses has been + extended to include "listserv", "majordomo", "*-request", and "owner-*", + taken from the Sieve specification recommendations. + + (2) If the message contains any header line starting with "List-" it is + treated as non-personal. + + (3) The test for "circular" in the Subject: header line has been removed + because it now seems ill-conceived. + + 5. The autoreply transport has a new option called never_mail. This is an + address list. If any run of the transport creates a message with a + recipient that matches any item in the list, that recipient is quietly + discarded. If all recipients are discarded, no message is created. + + +Version 4.40 +------------ + +The documentation is up-to-date for the 4.40 release. What follows here is a +brief list of the new features that have been added since 4.30. + + 1. log_incoming_interface affects more log lines. + + 2. New ACL modifier "control = submission". + + 3. CONFIGURE_OWNER can be set at build time to define an alternative owner for + the configuration file, in addition to root and exim. + + 4. Added expansion variables $body_zerocount, $recipient_data, and + $sender_data. + + 5. The time of last modification of the "new" subdirectory is now used as the + "mailbox time last read" when there is a quota error for a maildir + delivery. + + 6. The special item "+ignore_unknown" may now appear in host lists. + + 7. The special domain-matching patterns @mx_any, @mx_primary, and + @mx_secondary can now be followed by "/ignore=<ip list>". + + 8. New expansion conditions: match_domain, match_address, match_local_part, + lt, lti, le, lei, gt, gti, ge, and new expansion operators time_interval, + eval10, and base62d. + + 9. New lookup type called "iplsearch". + +10. New log selectors ident_timeout, tls_certificate_verified, queue_time, + deliver_time, outgoing_port, return_path_on_delivery. + +11. New global options smtp_active_hostname and tls_require_ciphers. + +12. Exinext has -C and -D options. + +13. "domainlist_cache" forces caching of an apparently variable list. + +14. For compatibility with Sendmail, the command line option -prval:sval + is equivalent to -oMr rval -oMs sval. + +15. New callout options use_sender and use_postmaster for use when verifying + recipients. + +16. John Jetmore's "exipick" utility has been added to the distribution. + +17. The TLS code now supports CRLs. + +18. The dnslookup router and the dnsdb lookup type now support the use of SRV + records. + +19. The redirect router has a new option called qualify_domain. + +20. exigrep's output now also includes lines that are not related to any + particular message, but which do match the pattern. + +21. New global option write_rejectlog. If it is set false, Exim no longer + writes anything to the reject log. + +**** |