summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorPhilip Hazel <ph10@hermes.cam.ac.uk>2004-10-08 10:38:47 +0000
committerPhilip Hazel <ph10@hermes.cam.ac.uk>2004-10-08 10:38:47 +0000
commite05f33e0b79c14608757a60f2f3f8588008355f7 (patch)
tree0a8dd6eaa4b91c51b6b1b013eeba11ab1cf7dc13
parent495ae4b01f36d0d8bb0e34a1d7263c2b8224aa4a (diff)
Start
-rw-r--r--doc/doc-misc/ABOUT11
-rw-r--r--doc/doc-misc/Ext-maildir109
-rw-r--r--doc/doc-misc/Ext-maildir++394
-rw-r--r--doc/doc-misc/Ext-mbx-locking400
-rw-r--r--doc/doc-misc/LongTermIssues200
-rw-r--r--doc/doc-misc/RFC.conform401
-rw-r--r--doc/doc-misc/TexiNotes193
-rw-r--r--doc/doc-misc/WishList1727
-rw-r--r--doc/doc-scripts/ABOUT9
-rw-r--r--doc/doc-src/ABOUT11
-rw-r--r--doc/doc-txt/ABOUT9
-rw-r--r--src/ABOUT11
12 files changed, 3475 insertions, 0 deletions
diff --git a/doc/doc-misc/ABOUT b/doc/doc-misc/ABOUT
new file mode 100644
index 000000000..7f6e294c9
--- /dev/null
+++ b/doc/doc-misc/ABOUT
@@ -0,0 +1,11 @@
+$Cambridge: exim/doc/doc-misc/ABOUT,v 1.1 2004/10/08 10:38:47 ph10 Exp $
+
+CVS directory exim/exim-doc/doc-misc
+------------------------------------
+
+This directory contains some miscellaneous documentation files that do not form
+part of Exim distributions, but are related to its maintenance and development.
+Those whose names start with "Ext-" are external documents that won't be
+modified (and hence have no local CVS Ids).
+
+End
diff --git a/doc/doc-misc/Ext-maildir b/doc/doc-misc/Ext-maildir
new file mode 100644
index 000000000..b523dee4a
--- /dev/null
+++ b/doc/doc-misc/Ext-maildir
@@ -0,0 +1,109 @@
+The following information is from the maildir man page of qmail.
+
+INTRODUCTION
+ maildir is a structure for directories of incoming mail
+ messages. It solves the reliability problems that plague
+ mbox files and mh folders.
+
+RELIABILITY ISSUES
+ A machine may crash while it is delivering a message. For
+ both mbox files and mh folders this means that the message
+ will be silently truncated. Even worse: for mbox format,
+ if the message is truncated in the middle of a line, it
+ will be silently joined to the next message. The mail
+ transport agent will try again later to deliver the mes-
+ sage, but it is unacceptable that a corrupted message
+ should show up at all. In maildir, every message is guar-
+ anteed complete upon delivery.
+
+ A machine may have two programs simultaneously delivering
+ mail to the same user. The mbox and mh formats require
+ the programs to update a single central file. If the pro-
+ grams do not use some locking mechanism, the central file
+ will be corrupted. There are several mbox and mh locking
+ mechanisms, none of which work portably and reliably. In
+ contrast, in maildir, no locks are ever necessary. Dif-
+ ferent delivery processes never touch the same file.
+
+ A user may try to delete messages from his mailbox at the
+ same moment that the machine delivers a new message. For
+ mbox and mh formats, the user's mail-reading program must
+ know what locking mechanism the mail-delivery programs
+ use. In contrast, in maildir, any delivered message can
+ be safely updated or deleted by a mail-reading program.
+
+ Many sites use Sun's Network Failure System (NFS), presum-
+ ably because the operating system vendor does not offer
+ anything else. NFS exacerbates all of the above problems.
+ Some NFS implementations don't provide any reliable lock-
+ ing mechanism. With mbox and mh formats, if two machines
+ deliver mail to the same user, or if a user reads mail
+ anywhere except the delivery machine, the user's mail is
+ at risk. maildir works without trouble over NFS.
+
+THE MAILDIR STRUCTURE
+ A directory in maildir format has three subdirectories,
+ all on the same filesystem: tmp, new, and cur.
+
+ Each file in new is a newly delivered mail message. The
+ modification time of the file is the delivery date of the
+ message. The message is delivered without an extra UUCP-
+ style From_ line, without any >From quoting, and without
+ an extra blank line at the end. The message is normally
+ in RFC 822 format, starting with a Return-Path line and a
+ Delivered-To line, but it could contain arbitrary binary
+ data. It might not even end with a newline.
+
+ Files in cur are just like files in new. The big differ-
+ ence is that files in cur are no longer new mail: they
+ have been seen by the user's mail-reading program.
+
+HOW A MESSAGE IS DELIVERED
+ The tmp directory is used to ensure reliable delivery, as
+ discussed here.
+
+ A program delivers a mail message in six steps. First, it
+ chdir()s to the maildir directory. Second, it stat()s the
+ name tmp/time.pid.host, where time is the number of sec-
+ onds since the beginning of 1970 GMT, pid is the program's
+ process ID, and host is the host name. Third, if stat()
+ returned anything other than ENOENT, the program sleeps
+ for two seconds, updates time, and tries the stat() again,
+ a limited number of times. Fourth, the program creates
+ tmp/time.pid.host. Fifth, the program NFS-writes the mes-
+ sage to the file. Sixth, the program link()s the file to
+ new/time.pid.host. At that instant the message has been
+ successfully delivered.
+
+ The delivery program is required to start a 24-hour timer
+ before creating tmp/time.pid.host, and to abort the deliv-
+ ery if the timer expires. Upon error, timeout, or normal
+ completion, the delivery program may attempt to unlink()
+ tmp/time.pid.host.
+
+ NFS-writing means (1) as usual, checking the number of
+ bytes returned from each write() call; (2) calling fsync()
+ and checking its return value; (3) calling close() and
+ checking its return value. (Standard NFS implementations
+ handle fsync() incorrectly but make up for it by abusing
+ close().)
+
+HOW A MESSAGE IS READ
+ A mail reader operates as follows.
+
+ It looks through the new directory for new messages. Say
+ there is a new message, new/unique. The reader may freely
+ display the contents of new/unique, delete new/unique, or
+ rename new/unique as cur/unique:info. See
+ http://pobox.com/~djb/maildir.html for the meaning of
+ info.
+
+ The reader is also expected to look through the tmp direc-
+ tory and to clean up any old files found there. A file in
+ tmp may be safely removed if it has not been accessed in
+ 36 hours.
+
+ It is a good idea for readers to skip all filenames in new
+ and cur starting with a dot. Other than this, readers
+ should not attempt to parse filenames.
+###
diff --git a/doc/doc-misc/Ext-maildir++ b/doc/doc-misc/Ext-maildir++
new file mode 100644
index 000000000..b2fc58045
--- /dev/null
+++ b/doc/doc-misc/Ext-maildir++
@@ -0,0 +1,394 @@
+ Maildir++
+
+ In this document:
+ * HOWTO.maildirquota
+ * Mission statement
+ * Definitions and goals
+ * Contents of a maildirsize
+ * Calculating maildirsize
+ * Calculating the quota for a Maildir++
+ * Delivering to a Maildir++
+ * Reading from a Maildir++
+ * Bugs
+
+HOWTO.maildirquota
+
+ The remaining portion of this document is a technical description of
+ the maildir quota extension. This section is a brief overview of this
+ extension.
+
+ What is a maildirquota?
+
+ If you would like to have a quota on your maildir mailboxes, the best
+ solution is to always use filesystem-based quotas: per-user usage
+ quotas that is enforced by the operating system.
+
+ This is the best solution when the default Maildir is located in each
+ account's home directory. This solution will NOT work if Maildirs are
+ stored elsewhere, or if you have a large virtual domain setup where a
+ single userid is used to hold many individual Maildirs, one for each
+ virtual user.
+
+ This extension to the maildir format allows a "voluntary" maildir
+ quota implementation that does not rely on filesystem-based quotas.
+
+ When maildirquota will not work.
+
+ For this quota mechanism to work, all software that accesses a maildir
+ must observe this quota protocol. It follows that this quota mechanism
+ can be easily circumvented if users have direct (shell) access to the
+ filesystem containing the users' maildirs.
+
+ Furthermore, this quota mechanism is not 100% effective. It is
+ possible to have a situation where someone may go over quota. This
+ quota implementation uses a deliverate trade-off. It is necessary to
+ use some form of locking in order to have a complete bulletproof quota
+ enforcement, but maildirs mail stores were explicitly designed to
+ avoid any kind of locking. This quota approach does not use locking,
+ and the tradeoff is that sometimes it is possible for a few extra
+ messages to be delivered to the maildir, before the door is
+ permanently shot.
+
+ For best performance, all maildir clients should support this quota
+ extension, however there's a wide degree of tolerance here. As long as
+ the mail delivery agent that puts new messages into a Maildir uses
+ this extension, the quota will be enforced without excessive
+ degradation.
+
+ In the worst case scenario, quotas are automatically recalculated
+ every fifteen minutes. If a maildir goes over quota, and a mail client
+ that does not support this quota extension removes enough mail from
+ the maildir, the mail delivery agent will not be immediately informed
+ that the maildir is now under quota. However, eventually the correct
+ quota will be recalculated and mail delivery will resume.
+
+ Mail user agents sometimes put messages into the maildir themselves.
+ Messages added to a maildir by a mail user agent that does not
+ understand the quota extension will not be immediately counted towards
+ the overall quota, and may not be counted for an extensive period of
+ time. Additionally, if there are a lot of messages that have been
+ added to a maildir from these mail user agents, quota recalculation
+ may impose non-trivial load on the system, as the quota recalculator
+ will have to issue the stat system call for each message.
+
+ How to implement the quota
+
+ The best way to do that is to modify your mail server to implement the
+ protocol defined by this document. Not everyone, of course, has this
+ ability. Therefore, an alternate approach is available.
+
+ This package creates a very short utility called "deliverquota". It
+ will NOT be installed anywhere by default, unless this maildir quota
+ implementation is a part of a larger package, in which case the parent
+ package may install this utility somewhere. If you obtained the
+ maildir package separately, you will need to compile it by running the
+ configure script, then by running make.
+
+ deliverquota takes two arguments. deliverquota reads the message from
+ standard input, then delivers it to the maildir specified by the first
+ argument to deliverquota. The second argument specifies the actual
+ quota for this maildir, as defined elsewhere in this document.
+ deliverquota will deliver the message to the maildir, making a best
+ effort not to exceed the stated quota. If the maildir is over quota,
+ deliverquota terminates with exit code 77. Otherwise, it delivers the
+ message, updates the quota, and terminates with exit code 0.
+
+ Therefore, proceed as follows:
+ * Copy deliverquota to some convenient location, say /usr/local/bin.
+ * Configure your mail server to use deliverquota. For example, if
+ you use Qmail and your maildirs are all located in $HOME/Maildir,
+ replace the './Maildir/' argument to qmail-start with the
+ following:
+'| /usr/local/bin/deliverquota ./Maildir 1000000S'
+
+
+
+
+ This sets a one million byte limit on all Maildirs. As I
+ mentioned, this is meaningless if login access is available,
+ because the individual account owner can create his own
+ $HOME/.qmail file, and ignore deliverquota. Note that in this
+ case, you MUST use apostrophes on the qmail-start command line, in
+ order to quote this as one argument.
+
+ If you would like to use different quotas for different users, you
+ will have to put together a separate process or a script that looks up
+ the appropriate quota for the recipient, and runs deliverquota
+ specifying the quota. If no login access to the mail server is
+ available, you can simply create a separate $HOME/.qmail for every
+ recipient.
+
+ That's pretty much it. If you handle a moderate amount of mail, I have
+ one more suggestion. For the first couple of weeks, run deliverquota
+ setting the second argument to an empty string. This disables quota
+ enforcement, however it still activates certain optimizations that
+ permit very fast quota recalculation. Messages delivered by
+ deliverquota have their message size encoded in their filename; this
+ makes it possible to avoid stat-ing the message in the Maildir, when
+ recalculating the quota. Then, after most messages in your maildirs
+ have been delivered by deliverquota, activate the quotas!!!
+
+ maildirquota-enhanced applications
+
+ This is a list of applications that have been enhanced to support the
+ maildirquota extension:
+ * maildrop - mail delivery agent/mail filter.
+ * SqWebmail - webmail CGI binary.
+
+ These applications fall into two classes:
+ * Mail delivery agents. These applications read some externally
+ defined table of mail recipients and their maildir quota.
+ * Mail clients. These applications read maildir quota information
+ that has been defined by the mail delivery agent.
+
+ Mail clients generally do not need any additional setup in order to
+ use the maildirquota extension. They will automatically read and
+ implement any quota specification set by the mail delivery agent.
+
+ On the other hand, mail delivery agents will require some kind of
+ configuration in order to activate the maildirquota extension for some
+ or all recipients. The instructions for doing that depends upon the
+ mail delivery agent. The documentation for the mail delivery agent
+ should be consulted for additional information.
+ _________________________________________________________________
+
+Mission statement
+
+ Maildir++ is a mail storage structure that's based on the Maildir
+ structure, first used in the Qmail mail server. Actually, Maildir++ is
+ just a minor extension to the standard Maildir structure.
+
+ For more information, see http://www.qmail.org/man/man5/maildir.html.
+ I am not going to include the definition of a Maildir in this
+ document. Consider it included right here. This document only
+ describes the differences.
+
+ Maildir++ adds a couple of things to a standard Maildir: folders and
+ quotas.
+
+ Quotas enforce a maximum allowable size of a Maildir. In many
+ situations, using the quota mechanism of the underlying filesystem
+ won't work very well. If a filesystem quota mechanism is used, then
+ when a Maildir goes over quota, Qmail does not bounce additional mail,
+ but keeps it queued, changing one bad situation into another bad
+ situation. Not only know you have an account that's backed up, but now
+ your queue starts to back up too.
+
+Definitions, and goals
+
+ Maildir++ and Maildir shall be completely interchangeable. A Maildir++
+ client will be able to use a standard Maildir, automatically
+ "upgrading" it in the process. A Maildir client will be able to use a
+ Maildir++ just like a regular Maildir. Of course, a plain Maildir
+ client won't be able to enforce a quota, and won't be able to access
+ messages stored in folders.
+
+ Folders are created as subdirectories under the main Maildir. The name
+ of the subdirectory always starts with a period. For example, a folder
+ named "Important" will be a subdirectory called ".Important". You
+ can't have subdirectories that start with two periods.
+
+ A Maildir++ client ignores anything in the main Maildir that starts
+ with a period, but is not a subdirectory.
+
+ Each subdirectory is a fully-fledged Maildir of its own, that is you
+ have .Important/tmp, .Important/new, and .Important/cur. Everything
+ that applies to the main Maildir applies equally well to the
+ subdirectory, including automatically cleaning up old files in tmp. A
+ Maildir++ enhancement is that a message can be moved between folders
+ and/or the main Maildir simply by moving/renaming the file (into the
+ cur subdirectory of the destination folder). Therefore, the entire
+ Maildir++ must reside on the same filesystem.
+
+ Within each subdirectory there's an empty file, maildirfolder. Its
+ existence tells the mail delivery agent that this Maildir is a really
+ a folder underneath a parent Maildir++.
+
+ Only one special folder is reserved: Trash (subdirectory .Trash).
+ Instead of marking deleted messages with the D flag, Maildir++ clients
+ move the message into the Trash folder. Maildir++ readers are
+ responsible for expunging messages from Trash after a system-defined
+ retention interval.
+
+ When a Maildir++ reader sees a message marked with a D flag it may at
+ its option: remove the message immediately, move it into Trash, or
+ ignore it.
+
+ Can folders have subfolders, defined in a recursive fashion? The
+ answer is no. If you want to have a client with a hierarchy of
+ folders, emulate it. Pick a hierarchy separator character, say ":".
+ Then, folder foo/bar is subdirectory .foo:bar.
+
+ This is all that there's to say about folders. The rest of this
+ document deals with quotas.
+
+ The purpose of quotas is to temporarily disable a Maildir, if it goes
+ over the quota. There is one and only major goal that this quota
+ implementation tries to achieve:
+ * Place as little overhead as possible on the mail system that's
+ delivering to the Maildir++
+
+ That's it. To achieve that goal, certain compromises are made:
+ * Mail delivery will stop as soon as possible after Maildir++'s size
+ goes over quota. Certain race conditions may happen with Maildir++
+ going a lot over quota, in rare circumstances. That is taken into
+ account, and the situation will eventually resolve itself, but you
+ should not simply take your systemwide quota, multiply it by the
+ number of mail accounts, and allocate that much disk space. Always
+ leave room to spare.
+ * How well the quota mechanism will work will depend on whether or
+ not everything that accesses the Maildir++ is a Maildir++ client.
+ You can have a transition period where some of your mail clients
+ are just Maildir clients, and things should run more or less well.
+ There will be some additional load because the size of the Maildir
+ will be recalculated more often, but the additional load shouldn't
+ be noticeable.
+
+ This won't be a perfect solution, but it will hopefully be good
+ enough. Maildirs are simply designed to rely on the filesystem to
+ enforce individual quotas. If a filesystem-based quota works for you,
+ use it.
+
+ A Maildir++ may contain the following additional file: maildirsize.
+
+Contents of maildirsize
+
+ maildirsize contains two or more lines terminated by newline
+ characters.
+
+ The first line contains a copy of the quota definition as used by the
+ system's mail server. Each application that uses the maildir must know
+ what it's quota is. Instead of configuring each application with the
+ quota logic, and making sure that every application's quota definition
+ for the same maildir is exactly the same, the quota specification used
+ by the system mail server is saved as the first line of the
+ maildirsize file. All other application that enforce the maildir quota
+ simply read the first line of maildirsize.
+
+ The quota definition is a list, separate by commas. Each member of the
+ list consists of an integer followed by a letter, specifying the
+ nature of the quota. Currently defined quota types are 'S' - total
+ size of all messages, and 'C' - the maximum count of messages in the
+ maildir. For example, 10000000S,1000C specifies a quota of 10,000,000
+ bytes or 1,000 messages, whichever comes first.
+
+ All remaining lines all contain two integers separated by a single
+ space. The first integer is interpreted as a byte count. The second
+ integer is interpreted as a file count. A Maildir++ writer can add up
+ all byte counts and file counts from maildirsize and enforce a quota
+ based either on number of messages or the total size of all the
+ messages.
+
+Calculating maildirsize
+
+ In most cases, changes to maildirsize are recorded by appending an
+ additional line. Under some conditions maildirsize has to be
+ recalculated from scratch. These conditions are defined later. This is
+ the procedure that's used to recalculate maildirsize:
+ 1. If we find a maildirfolder within the directory, we're delivering
+ to a folder, so back up to the parent directory, and start again.
+ 2. Read the contents of the new and cur subdirectories. Also, read
+ the contents of the new and cur subdirectories in each Maildir++
+ folder, except Trash. Before reading each subdirectory, stat() the
+ subdirectory itself, and keep track of the latest timestamp you
+ get.
+ 3. If the filename of each message is of the form xxxxx,S=nnnnn or
+ xxxxx,S=nnnnn:xxxxx where "xxxxx" represents arbitrary text, then
+ use nnnnn as the size of the file (which will be conveniently
+ recorded in the filename by a Maildir++ writer, within the
+ conventions of filename naming in a Maildir). If the message was
+ not written by a Maildir++ writer, stat() it to obtain the message
+ size. If stat() fails, a race condition removed the file, so just
+ ignore it and move on to the next one.
+ 4. When done, you have the grand total of the number of messages and
+ their total size. Create a new maildirsize by: creating the file
+ in the tmp subdirectory, observing the conventions for writing to
+ a Maildir. Then rename the file as maildirsize.Afterwards, stat
+ all new and cur subdirectories again. If you find a timestamp
+ later than the saved timestamp, REMOVE maildirsize.
+ 5. Before running this calculation procedure, the Maildir++ user
+ wanted to know the size of the Maildir++, so return the calculated
+ values. This is done even if maildirsize was removed.
+
+Calculating the quota for a Maildir++
+
+ This is the procedure for reading the contents of maildirsize for the
+ purpose of determine if the Maildir++ is over quota.
+ 1. If maildirsize does not exist, or if its size is at least 5120
+ bytes, recalculate it using the procedure defined above, and use
+ the recalculated numbers. Otherwise, read the contents of
+ maildirsize, and add up the totals.
+ 2. The most efficient way of doing this is to: open maildirsize, then
+ start reading it into a 5120 byte buffer (some broken NFS
+ implementations may return less than 5120 bytes read even before
+ reaching the end of the file). If we fill it, which, in most
+ cases, will happen with one read, close it, and run the
+ recalculation procedure.
+ 3. In many cases the quota calculation is for the purpose of adding
+ or removing messages from a Maildir++, so keep the file descriptor
+ to maildirsize open. A file descriptor will not be available if
+ quota recalculation ended up removing maildirsize due to a race
+ condition, so the caller may or may not get a file descriptor
+ together with the Maildir++ size.
+ 4. If the numbers we got indicated that the Maidlir++ is over quota,
+ some additional logic is in order: if we did not recalculate
+ maildirsize, if the numbers in maildirsize indicated that we are
+ over quota, then if maildirsize was more than one line long, or if
+ the timestamp on maildirsize indicated that it's at least 15
+ minutes old, throw out the totals, and recalculate maildirsize
+ from scratch.
+
+ Eventually the 5120 byte limitation will always cause maildirsize to
+ be recalculated, which will compensate for any race conditions which
+ previously threw off the totals. Each time a message is delivered or
+ removed from a Maildir++, one line is added to maildirsize (this is
+ described below in greater detail). Most messages are less than 10K
+ long, so each line appended to maildirsize will be either between
+ seven and nine bytes long (four bytes for message count, space, digit
+ 1, newline, optional minus sign in front of both counts if the message
+ was removed). This results in about 640 Maildir++ operations before a
+ recalculation is forced. Since most messages are added once and
+ removed once from a Maildir, expect recalculation to happen
+ approximately every 320 messages, keeping the overhead of a
+ recalculation to a minimum. Even if most messages include large
+ attachments, most attachments are less than 100K long, which brings
+ down the average recalculation frequency to about 150 messages.
+
+ Also, the effect of having non-Maildir++ clients accessing the
+ Maildir++ is reduced by forcing a recalculation when we're potentially
+ over quota. Even if non-Maildir++ clients are used to remove messages
+ from the Maildir, the fact that the Maildir++ is still over quota will
+ be verified every 15 minutes.
+
+Delivering to a Maildir++
+
+ Delivering to a Maildir++ is like delivering to a Maildir, with the
+ following exceptions:
+ 1. Follow the usual Maildir conventions for naming the filename used
+ to store the message, except that append ,S=nnnnn to the name of
+ the file, where nnnnn is the size of the file. This eliminates the
+ need to stat() most messages when calculating the quota. If the
+ size of the message is not known at the beginning, append ,S=nnnnn
+ when renaming the message from tmp to new.
+ 2. As soon as the size of the message is known (hopefully before it
+ is written into tmp), calculate Maildir++'s quota, using the
+ procedure defined previously. If the message is over quota, back
+ out, cleaning up anything that was created in tmp.
+ 3. If a file descriptor to maildirsize was opened for us, after
+ moving the file from tmp to new append a line to the file
+ containing the message size, and "1".
+
+Reading from a Maildir++
+
+ Maildir++ readers should mind the following additional tasks:
+ 1. Make sure to create the maildirfolder file in any new folders
+ created within the Maildir++.
+ 2. When moving a message to the Trash folder, append a line to
+ maildirsize, containing a negative message size and a '-1'.
+ 3. When moving a message from the Trash folder, follow the steps
+ described in "Delivering to Maildir++", as far as quota logic
+ goes. That is, refuse to move messages out of Trash if the
+ Maildir++ is over quota.
+ 4. Moving a message between other folders carries no additional
+ requirements.
+
diff --git a/doc/doc-misc/Ext-mbx-locking b/doc/doc-misc/Ext-mbx-locking
new file mode 100644
index 000000000..f1b0523f6
--- /dev/null
+++ b/doc/doc-misc/Ext-mbx-locking
@@ -0,0 +1,400 @@
+ UNIX Advisory File Locking Implications on c-client
+ Mark Crispin, 28 November 1995
+
+
+ THIS DOCUMENT HAS BEEN UPDATED TO REFLECT THE CODE IN THE
+ IMAP-4 TOOLKIT AS OF NOVEMBER 28, 1995. SOME STATEMENTS
+ IN THIS DOCUMENT DO NOT APPLY TO EARLIER VERSIONS OF THE
+ IMAP TOOLKIT.
+
+INTRODUCTION
+
+ Advisory locking is a mechanism by which cooperating processes
+can signal to each other their usage of a resource and whether or not
+that usage is critical. It is not a mechanism to protect against
+processes which do not cooperate in the locking.
+
+ The most basic form of locking involves a counter. This counter
+is -1 when the resource is available. If a process wants the lock, it
+executes an atomic increment-and-test-if-zero. If the value is zero,
+the process has the lock and can execute the critical code that needs
+exclusive usage of a resource. When it is finished, it sets the lock
+back to -1. In C terms:
+
+ while (++lock) /* try to get lock */
+ invoke_other_threads (); /* failed, try again */
+ .
+ . /* critical code here */
+ .
+ lock = -1; /* release lock */
+
+ This particular form of locking appears most commonly in
+multi-threaded applications such as operating system kernels. It
+makes several presumptions:
+ (1) it is alright to keep testing the lock (no overflow)
+ (2) the critical resource is single-access only
+ (3) there is shared writeable memory between the two threads
+ (4) the threads can be trusted to release the lock when finished
+
+ In applications programming on multi-user systems, most commonly
+the other threads are in an entirely different process, which may even
+be logged in as a different user. Few operating systems offer shared
+writeable memory between such processes.
+
+ A means of communicating this is by use of a file with a mutually
+agreed upon name. A binary semaphore can be passed by means of the
+existance or non-existance of that file, provided that there is an
+atomic means to create a file if and only if that file does not exist.
+In C terms:
+
+ /* try to get lock */
+ while ((fd = open ("lockfile",O_WRONLY|O_CREAT|O_EXCL,0666)) < 0)
+ sleep (1); /* failed, try again */
+ close (fd); /* got the lock */
+ .
+ . /* critical code here */
+ .
+ unlink ("lockfile"); /* release lock */
+
+ This form of locking makes fewer presumptions, but it still is
+guilty of presumptions (2) and (4) above. Presumption (2) limits the
+ability to have processes sharing a resource in a non-conflicting
+fashion (e.g. reading from a file). Presumption (4) leads to
+deadlocks should the process crash while it has a resource locked.
+
+ Most modern operating systems provide a resource locking system
+call that has none of these presumptions. In particular, a mechanism
+is provided for identifying shared locks as opposed to exclusive
+locks. A shared lock permits other processes to obtain a shared lock,
+but denies exclusive locks. In other words:
+
+ current state want shared want exclusive
+ ------------- ----------- --------------
+ unlocked YES YES
+ locked shared YES NO
+ locked exclusive NO NO
+
+ Furthermore, the operating system automatically relinquishes all
+locks held by that process when it terminates.
+
+ A useful operation is the ability to upgrade a shared lock to
+exclusive (provided there are no other shared users of the lock) and
+to downgrade an exclusive lock to shared. It is important that at no
+time is the lock ever removed; a process upgrading to exclusive must
+not relenquish its shared lock.
+
+ Most commonly, the resources being locked are files. Shared
+locks are particularly important with files; multiple simultaneous
+processes can read from a file, but only one can safely write at a
+time. Some writes may be safer than others; an append to the end of
+the file is safer than changing existing file data. In turn, changing
+a file record in place is safer than rewriting the file with an
+entirely different structure.
+
+
+FILE LOCKING ON UNIX
+
+ In the oldest versions of UNIX, the use of a semaphore lockfile
+was the only available form of locking. Advisory locking system calls
+were not added to UNIX until after the BSD vs. System V split. Both
+of these system calls deal with file resources only.
+
+ Most systems only have one or the other form of locking. AIX
+emulates the BSD form of locking as a jacket into the System V form.
+Ultrix and OSF/1 implement both forms.
+
+BSD
+
+ BSD added the flock() system call. It offers capabilities to
+acquire shared lock, acquire exclusive lock, and unlock. Optionally,
+the process can request an immediate error return instead of blocking
+when the lock is unavailable.
+
+
+FLOCK() BUGS
+
+ flock() advertises that it permits upgrading of shared locks to
+exclusive and downgrading of exclusive locks to shared, but it does so
+by releasing the former lock and then trying to acquire the new lock.
+This creates a window of vulnerability in which another process can
+grab the exclusive lock. Therefore, this capability is not useful,
+although many programmers have been deluded by incautious reading of
+the flock() man page to believe otherwise. This problem can be
+programmed around, once the programmer is aware of it.
+
+ flock() always returns as if it succeeded on NFS files, when in
+fact it is a no-op. There is no way around this.
+
+ Leaving aside these two problems, flock() works remarkably well,
+and has shown itself to be robust and trustworthy.
+
+SYSTEM V/POSIX
+
+ System V added new functions to the fnctl() system call, and a
+simple interface through the lockf() subroutine. This was
+subsequently included in POSIX. Both offer the facility to apply the
+lock to a particular region of the file instead of to the entire file.
+lockf() only supports exclusive locks, and calls fcntl() internally;
+hence it won't be discussed further.
+
+ Functionally, fcntl() locking is a superset of flock(); it is
+possible to implement a flock() emulator using fcntl(), with one minor
+exception: it is not possible to acquire an exclusive lock if the file
+is not open for write.
+
+ The fcntl() locking functions are: query lock station of a file
+region, lock/unlock a region, and lock/unlock a region and block until
+have the lock. The locks may be shared or exclusive. By means of the
+statd and lockd daemons, fcntl() locking is available on NFS files.
+
+ When statd is started at system boot, it reads its /etc/state
+file (which contains the number of times it has been invoked) and
+/etc/sm directory (which contains a list of all remote sites which are
+client or server locking with this site), and notifies the statd on
+each of these systems that it has been restarted. Each statd then
+notifies the local lockd of the restart of that system.
+
+ lockd receives fcntl() requests for NFS files. It communicates
+with the lockd at the server and requests it to apply the lock, and
+with the statd to request it for notification when the server goes
+down. It blocks until all these requests are completed.
+
+ There is quite a mythos about fcntl() locking.
+
+ One religion holds that fcntl() locking is the best thing since
+sliced bread, and that programs which use flock() should be converted
+to fcntl() so that NFS locking will work. However, as noted above,
+very few systems support both calls, so such an exercise is pointless
+except on Ultrix and OSF/1.
+
+ Another religion, which I adhere to, has the opposite viewpoint.
+
+
+FCNTL() BUGS
+
+ For all of the hairy code to do individual section locking of a
+file, it's clear that the designers of fcntl() locking never
+considered some very basic locking operations. It's as if all they
+knew about locking they got out of some CS textbook with not
+investigation of real-world needs.
+
+ It is not possible to acquire an exclusive lock unless the file
+is open for write. You could have append with shared read, and thus
+you could have a case in which a read-only access may need to go
+exclusive. This problem can be programmed around once the programmer
+is aware of it.
+
+ If the file is opened on another file designator in the same
+process, the file is unlocked even if no attempt is made to do any
+form of locking on the second designator. This is a very bad bug. It
+means that an application must keep track of all the files that it has
+opened and locked.
+
+ If there is no statd/lockd on the NFS server, fcntl() will hang
+forever waiting for them to appear. This is a bad bug. It means that
+any attempt to lock on a server that doesn't run these daemons will
+hang. There is no way for an application to request flock() style
+``try to lock, but no-op if the mechanism ain't there''.
+
+ There is a rumor to the effect that fcntl() will hang forever on
+local files too if there is no local statd/lockd. These daemons are
+running on mailer.u, although they appear not to have much CPU time.
+A useful experiment would be to kill them and see if imapd is affected
+in any way, but I decline to do so without an OK from UCS! ;-) If
+killing statd/lockd can be done without breaking fcntl() on local
+files, this would become one of the primary means of dealing with this
+problem.
+
+ The statd and lockd daemons have quite a reputation for extreme
+fragility. There have been numerous reports about the locking
+mechanism being wedged on a systemwide or even clusterwide basis,
+requiring a reboot to clear. It is rumored that this wedge, once it
+happens, also blocks local locking. Presumably killing and restarting
+statd would suffice to clear the wedge, but I haven't verified this.
+
+ There appears to be a limit to how many locks may be in use at a
+time on the system, although the documentation only mentions it in
+passing. On some of their systems, UCS has increased lockd's ``size
+of the socket buffer'', whatever that means.
+
+C-CLIENT USAGE
+
+ c-client uses flock(). On System V systems, flock() is simulated
+by an emulator that calls fcntl(). This emulator is provided by some
+systems (e.g. AIX), or uses c-client's flock.c module.
+
+
+BEZERK AND MMDF
+
+ Locking in the traditional UNIX formats was largely dictated by
+the status quo in other applications; however, additional protection
+is added against inadvertantly running multiple instances of a
+c-client application on the same mail file.
+
+ (1) c-client attempts to create a .lock file (mail file name with
+``.lock'' appended) whenever it reads from, or writes to, the mail
+file. This is an exclusive lock, and is held only for short periods
+of time while c-client is actually doing the I/O. There is a 5-minute
+timeout for this lock, after which it is broken on the presumption
+that it is a stale lock. If it can not create the .lock file due to
+an EACCES (protection failure) error, it once silently proceeded
+without this lock; this was for systems which protect /usr/spool/mail
+from unprivileged processes creating files. Today, c-client reports
+an error unless it is built otherwise. The purpose of this lock is to
+prevent against unfavorable interactions with mail delivery.
+
+ (2) c-client applies a shared flock() to the mail file whenever
+it reads from the mail file, and an exclusive flock() whenever it
+writes to the mail file. This lock is freed as soon as it finishes
+reading. The purpose of this lock is to prevent against unfavorable
+interactions with mail delivery.
+
+ (3) c-client applies an exclusive flock() to a file on /tmp
+(whose name represents the device and inode number of the file) when
+it opens the mail file. This lock is maintained throughout the
+session, although c-client has a feature (called ``kiss of death'')
+which permits c-client to forcibly and irreversibly seize the lock
+from a cooperating c-client application that surrenders the lock on
+demand. The purpose of this lock is to prevent against unfavorable
+interactions with other instances of c-client (rewriting the mail
+file).
+
+ Mail delivery daemons use lock (1), (2), or both. Lock (1) works
+over NFS; lock (2) is the only one that works on sites that protect
+/usr/spool/mail against unprivileged file creation. Prudent mail
+delivery daemons use both forms of locking, and of course so does
+c-client.
+
+ If only lock (2) is used, then multiple processes can read from
+the mail file simultaneously, although in real life this doesn't
+really change things. The normal state of locks (1) and (2) is
+unlocked except for very brief periods.
+
+
+TENEX AND MTX
+
+ The design of the locking mechanism of these formats was
+motivated by a design to enable multiple simultaneous read/write
+access. It is almost the reverse of how locking works with
+bezerk/mmdf.
+
+ (1) c-client applies a shared flock() to the mail file when it
+opens the mail file. It upgrades this lock to exclusive whenever it
+tries to expunge the mail file. Because of the flock() bug that
+upgrading a lock actually releases it, it will not do so until it has
+acquired an exclusive lock (2) first. The purpose of this lock is to
+prevent against expunge taking place while some other c-client has the
+mail file open (and thus knows where all the messages are).
+
+ (2) c-client applies a shared flock() to a file on /tmp (whose
+name represents the device and inode number of the file) when it
+parses the mail file. It applies an exclusive flock() to this file
+when it appends new mail to the mail file, as well as before it
+attempts to upgrade lock (1) to exclusive. The purpose of this lock
+is to prevent against data being appended while some other c-client is
+parsing mail in the file (to prevent reading of incomplete messages).
+It also protects against the lock-releasing timing race on lock (1).
+
+OBSERVATIONS
+
+ In a perfect world, locking works. You are protected against
+unfavorable interactions with the mailer and against your own mistake
+by running more than one instance of your mail reader. In tenex/mtx
+formats, you have the additional benefit that multiple simultaneous
+read/write access works, with the sole restriction being that you
+can't expunge if there are any sharers of the mail file.
+
+ If the mail file is NFS-mounted, then flock() locking is a silent
+no-op. This is the way BSD implements flock(), and c-client's
+emulation of flock() through fcntl() tests for NFS files and
+duplicates this functionality. There is no locking protection for
+tenex/mtx mail files at all, and only protection against the mailer
+for bezerk/mmdf mail files. This has been the accepted state of
+affairs on UNIX for many sad years.
+
+ If you can not create .lock files, it should not affect locking,
+since the flock() locks suffice for all protection. This is, however,
+not true if the mailer does not check for flock() locking, or if the
+the mail file is NFS-mounted.
+
+ What this means is that there is *no* locking protection at all
+in the case of a client using an NFS-mounted /usr/spool/mail that does
+not permit file creation by unprivileged programs. It is impossible,
+under these circumstances, for an unprivileged program to do anything
+about it. Worse, if EACCES errors on .lock file creation are no-op'ed
+, the user won't even know about it. This is arguably a site
+configuration error.
+
+ The problem with not being able to create .lock files exists on
+System V as well, but the failure modes for flock() -- which is
+implemented via fcntl() -- are different.
+
+ On System V, if the mail file is NFS-mounted and either the
+client or the server lacks a functioning statd/lockd pair, then the
+lock attempt would have hung forever if it weren't for the fact that
+c-client tests for NFS and no-ops the flock() emulator in this case.
+Systemwide or clusterwide failures of statd/lockd have been known to
+occur which cause all locks in all processes to hang (including
+local?). Without the special NFS test made by c-client, there would
+be no way to request BSD-style no-op behavior, nor is there any way to
+determine that this is happening other than the system being hung.
+
+ The additional locking introduced by c-client was shown to cause
+much more stress on the System V locking mechanism than has
+traditionally been placed upon it. If it was stressed too far, all
+hell broke loose. Fortunately, this is now past history.
+
+TRADEOFFS
+
+ c-client based applications have a reasonable chance of winning
+as long as you don't use NFS for remote access to mail files. That's
+what IMAP is for, after all. It is, however, very important to
+realize that you can *not* use the lock-upgrade feature by itself
+because it releases the lock as an interim step -- you need to have
+lock-upgrading guarded by another lock.
+
+ If you have the misfortune of using System V, you are likely to
+run into problems sooner or later having to do with statd/lockd. You
+basically end up with one of three unsatisfactory choices:
+ 1) Grit your teeth and live with it.
+ 2) Try to make it work:
+ a) avoid NFS access so as not to stress statd/lockd.
+ b) try to understand the code in statd/lockd and hack it
+ to be more robust.
+ c) hunt out the system limit of locks, if there is one,
+ and increase it. Figure on at least two locks per
+ simultaneous imapd process and four locks per Pine
+ process. Better yet, make the limit be 10 times the
+ maximum number of processes.
+ d) increase the socket buffer (-S switch to lockd) if
+ it is offered. I don't know what this actually does,
+ but giving lockd more resources to do its work can't
+ hurt. Maybe.
+ 3) Decide that it can't possibly work, and turn off the
+ fcntl() calls in your program.
+ 4) If nuking statd/lockd can be done without breaking local
+ locking, then do so. This would make SVR4 have the same
+ limitations as BSD locking, with a couple of additional
+ bugs.
+ 5) Check for NFS, and don't do the fcntl() in the NFS case.
+ This is what c-client does.
+
+ Note that if you are going to use NFS to access files on a server
+which does not have statd/lockd running, your only choice is (3), (4),
+or (5). Here again, IMAP can bail you out.
+
+ These problems aren't unique to c-client applications; they have
+also been reported with Elm, Mediamail, and other email tools.
+
+ Of the other two SVR4 locking bugs:
+
+ Programmer awareness is necessary to deal with the bug that you
+can not get an exclusive lock unless the file is open for write. I
+believe that c-client has fixed all of these cases.
+
+ The problem about opening a second designator smashing any
+current locks on the file has not been addressed satisfactorily yet.
+This is not an easy problem to deal with, especially in c-client which
+really doesn't know what other files/streams may be open by Pine.
+
+ Aren't you so happy that you bought an System V system?
diff --git a/doc/doc-misc/LongTermIssues b/doc/doc-misc/LongTermIssues
new file mode 100644
index 000000000..808ba3f8c
--- /dev/null
+++ b/doc/doc-misc/LongTermIssues
@@ -0,0 +1,200 @@
+$Cambridge: exim/doc/doc-misc/LongTermIssues,v 1.1 2004/10/08 10:38:47 ph10 Exp $
+
+Exim Long Term Issues
+---------------------
+
+I restarted this list from scratch for Exim 4. I amalgamated it with another
+list when creating the CVS repository (October 2004). But it still probably
+needs a substantial spring clean. Some of it is very old now.
+
+
+AUTOCONF
+--------
+
+Somebody once tried to \(autoconf)\ Exim, but found it too big a job. I now
+have some experience with using \(autoconf)\ for PCRE, and I think maybe some
+use could be made of it. I don't, however, believe that \(all)\ Exim build-time
+configuration should be done that way. The reason is that, unlike something
+like PCRE, there is quite a lot of information that is "user choice". Giving it
+all as options to a \(configure)\ command does not seem the best way of doing
+things.
+
+Whenever I build something that needs more than a couple of obvious options to
+\(configure)\, I always save them in a file anyway, so I know what I did for
+next time. Therefore, I think it is sensible to retain the current Local file
+structure for all the user choice configuration.
+
+However, it might be helpful to use \(autoconf)\ to dig out various bits of
+information about the operating system. At present, the \(OS/Makefile-*)\ files
+have hard-wired settings, and maybe this information could be figured out by
+running \(autoconf)\, which would save having to keep maintaining these files.
+
+I would arrange things so that \(configure)\ is run automatically the first
+time that \(make)\ is run, but it would be possible to run it manually first,
+to override defaults. (For example, if you have both \(cc)\ and \(gcc)\
+installed on your system, as I do, you need to be able to specify which to
+use.) I will need to do some experiments to see exactly how this would work.
+
+
+EXIMON and other utilities
+--------------------------
+
+. Consider optionally making it possible to link with something other than
+ Athena widgets - for example, gtk. Or indeed re-write the whole thing!
+
+
+GENERAL
+-------
+
+. Convert os.c into a directory of separate functions, with the macro
+ switches defined elsewhere. Then make it into a library.
+
+. Use a pointer to an address structure for expanding $domain etc, to make it
+ easier to save/restore this collection of variables. But note that $domain
+ and $local_part aren't always in an address. Check out when these are set.
+ Note also the new $address_data possibility.
+
+. Spool_in and spool_out - speed up by using a table?
+
+. Find a more compact way of encoding the options interpretation, and also of
+ checking for incompatible options.
+
+. Find a more compact way of passing an open SMTP channel without having
+ to use options. What about the TLS state information? Could use a pipe to
+ pass more data.
+
+. Some people have suggested separately loadable modules. But do all systems
+ have them? Is this going too far for just a few specialist users? In
+ particular, people want to be able to replace the logging with his own code.
+ Can we arrange this without going for the separately loaded modules? (cf the
+ incoming checking code.)
+
+. SIGHUP the daemon - don't close the sockets; instead pass a list of them
+ somewhere for the new daemon to pick up. Iff started by exim or root, of
+ course. There might be quite a long list of them - argv might not be the best
+ idea. If this were done, then a non-setuid exim daemon could be SIGHUPped.
+
+. Parallel deliveries. Currently dead host information doesn't get propagated
+ between them very well. Is there anyway this could be improved?
+
+. In some environments the use of gethostbyname() seems to cause problems.
+ Check out its use, and see if having a "force DNS" option could be helpful.
+ But people would have to know what they were doing.
+
+. accept_max_per_host is a slow, linear search. If smtp_accept_max is large,
+ this can be very slow. Is there some way we can speed this up? Some kind of
+ index based on the IP address? Remember, this is in the daemon, so it must
+ not consume store.
+
+. Change the names of all the pcre_ stuff to, say, PCRE_ so that Exim can be
+ linked with libraries or whatever that also use an external PCRE library.
+
+. Look at code in pidentd for running Exim in wait mode from inetd and re-using
+ the socket. This would allow it to run more tidily as non-root.
+
+. Think up some scheme for checking for orphan files in the spool directories.
+ Perhaps -bp should always do it, but it would be nice to have it done
+ automatically now and again. Maybe we just leave this for a cron job? Perhaps
+ a new -bx, e.g. -bpck or something. Better, perhaps, is a separate Perl
+ script. Orphan = a file that is over 24h old (or 1s when test harness) and
+ either doesn't end in -D or -H, or is a -D without a matching -H (or vice
+ versa).
+
+. Make set_process_info buffer bigger, and put the overflowed message at the
+ end, thereby leaving the start.
+
+. Swamping with delays in checking for reserved hosts - the connections are
+ counted in the total allowed. Can we improve on this somehow? Maybe shared
+ memory can help here. Think about different states and different limits.
+
+. Lists that must use colons: can we check for other cases, and fix them up
+ before passing them on? Is it worth it?
+
+. Linux for S/390 - create configuration?
+
+. Process receiving error message fails - can we get more info, such as the
+ stdout/stderr?
+
+. dbmbuild - if renaming one of .dir/.pag fails, reinstate the other. Should
+ there be a lock?
+
+. Write a script to check for format problems in the source - formats that are
+ not fixed strings and are built from outside code.
+
+. freeze_tell: Don't if message is a bounce message containing From: the local
+ machine - even if the bounce comes from another host.
+
+. Add additional data into the "frozen" log message at end of delivery, e.g. if
+ remote host was the local host or whatever. At least some cross referencing.
+
+. Someone had a requirement to install the Exim binary in a different place to
+ the utilities, etc. Also, for different builds on the same host and
+ architecture.
+
+. Include (part of?) the ppid in the message id? Or a random number?
+
+. Re-implement the code in readconf that reads error names for retry rules.
+ Make it use a table for most of the error types. Then see if we can usefully
+ add any additional error types.
+
+. Should there be "exim -bP acls" etc? It would mean inventing some kind of
+ "hide" facility within the ACL syntax.
+
+. VERY LONG TERM: the message ID is too small now, with the recent changes to
+ cram in the sub-second time. It would be a big project to extend it; Exim
+ would have to recognize both forms for a while, and become stable, before
+ generating the new form. Probably a runtime switch needed. The new form needs
+ at least microsecond time (or more?) and should probably cope with 64-bit
+ pids, just to be safe (or leave expansion space that could be used for that).
+ It should also be able to hold big enough things in base 36.
+
+. Take a look at libexec.
+
+. Sort out the stcncpy/strlcpy issue once and for all. Time things.
+
+. Error in transport filter. See test 407. All 3 processes see errors - which
+ one should be noticed? Transport_filter_temp_errors may be needed.
+
+. Think about 5xx thresholds -- too many and you're out. What about 4xx?
+
+. autoreply - should it call /usr/sbin/sendmail? Provide a way of not passing
+ -C and -D when creating the message ('cause it won't be privileged).
+
+. Strings containing \000 - anything we can do?
+
+. OpenSSL - can we pass an opened file for certificate? Repeatedly?
+ Otherwise pre-initialize while root? There do seem to be functions for
+ manipulating certificates, but documentation is scarce. Can we just load the
+ certificate in as root in the server?
+
+. Consider using poll() to close unwanted fds. Is this efficient? Perhaps it
+ doesn't matter for the daemon.
+
+. On a 64-bit system there are some cast warnings for casting addresses to
+ ints. Either we must find a way of not warning, or we'll have to use unions
+ to get round it.
+
+. Run splint on the source?
+
+. It has been suggested that rejection because not authenticated should use
+ 530 and not 550, but this is hard to detect because of the way ACLs work.
+
+. When there is a sender verify failure, $acl_verify_message contains "sender
+ verify failed", not the details of the failure. Should this change? Some of
+ the waffly details are added later in smtp_in.c. In the ACL that text is in
+ sender_verified_failed->user_message.
+
+. An empty string for a transport filter currently causes an error. Should it
+ ignore? Tricky because of special expansion rules for commands.
+
+. GFDL for documentation (www.gnu.org/licenses/fdl.html)? The 1.2 version of
+ this licence is still quite new (it is dated November 2002) so I think
+ waiting for reaction/opinion is the best plan. There are Debian concerns
+ about this licence. At very least, no Invariant Sections and no Cover Texts
+ can be used.
+
+. Allow $recipients in other places. Not clear what this value should be if,
+ say, the system filter has overridden them. Default would be envelope
+ recipients, as now.
+
+End
diff --git a/doc/doc-misc/RFC.conform b/doc/doc-misc/RFC.conform
new file mode 100644
index 000000000..2fc57cdf2
--- /dev/null
+++ b/doc/doc-misc/RFC.conform
@@ -0,0 +1,401 @@
+$Cambridge: exim/doc/doc-misc/RFC.conform,v 1.1 2004/10/08 10:38:47 ph10 Exp $
+
+Conformance with RFCs
+---------------------
+
+Exim is written to follow the rules laid down in the RFCs. However, there are
+some circumstances where it either extends what is specified, or chooses not to
+follow them strictly, for various reasons. Sometimes variations are controlled
+by an option, which may default on or off. This document lists the variations
+from the latest email RFCs, and discusses their background and implications.
+
+Last Updated: 25 January 1999
+
+
+1. RFC 822
+----------
+
+The original specification of the format of Internet mail messages is RFC 822,
+later clarified and modified by RFC 1123. At the time of writing (January 1999)
+a new RFC (currently known as draft-ietf-drums-msg-fmt-07) which updates and
+consolidates all the material related to the message format is at a late stage
+of drafting, and is expected to become an Internet Standard in due course.
+
+The following is (I hope) a complete list of major variations from the draft
+RFC. References in square brackets are to the -07 draft.
+
+
+1.1 Line termination [2.1, 2.3]
+-------------------------------
+
+[Lines are terminated by CRLF; isolated CR and LF are not permitted.]
+
+The CRLF requirement has to be interpreted carefully, because the RFC also says
+that it does not cover the internal format "used by sites". Exim keeps messages
+on its spool in Unix format, using only LF as the line terminator, and also
+does local deliveries using only LF. I believe this is compliant with the RFC,
+as these are both "internal formats".
+
+Messages sent out by SMTP have CRLF line terminators. However, isolated CR
+characters are treated as any other data characters, because Exim is eight-bit
+clean (see 1.2 below).
+
+See 2.1 below for a discussion of line terminators in incoming messages.
+
+
+1.2 Eight-bit characters [2.1]
+------------------------------
+
+[Messages consist of 7-bit characters.]
+
+Exim is eight-bit clean. It does not do any processing of the characters in the
+body of a message.
+
+
+1.3 Maximum line length [2.1, 2.3]
+----------------------------------
+
+[The maximum length of a line is 998 characters.]
+
+Exim does not enforce any limit on line length.
+
+
+1.4 The "phrase" part of an address [3.4]
+-----------------------------------------
+
+[The phrase is a sequence of "words"; a word is an "atom" or a quoted string.]
+
+The characters that can be used in an "atom" do not include the full stop
+(dot, period). Thus a header line such as
+
+ To: John Q. Public <jqp@anywhere.org>
+
+is syntactically invalid under a strict interpretation of the RFC because the
+dot in the phrase part is not quoted. However, many MTAs do not enforce this
+restriction, so Exim was changed to be relaxed about it as well. In fact, the
+draft RFC is moving towards allowing this. In section [4.1], which is defining
+"obsolete" syntax that programs must accept (but not generate), it says this:
+
+ The period character is added to obs-phrase.
+
+ Note: The period character in obs-phrase is not a form that was allowed
+ in earlier versions of this or any other standard. Period (nor any other
+ character from specials) was not allowed in phrase because it introduced
+ a parsing difficulty distinguishing between phrases and portions of an
+ addr-spec (see section 4.4). It appears here because the period
+ character is currently used in many messages in the display-name portion
+ of addresses, especially for initials in names, and therefore must be
+ interpreted properly. In the future, period may appear in the regular
+ syntax of phrase.
+
+
+1.5 Source routed addresses [4.4]
+---------------------------------
+
+[Source routed addresses are always enclosed in <>.]
+
+Source routed addresses are declared obsolete in the draft RFC, but MTAs are
+still required to handle them. Strictly, a source-routed address must be
+enclosed in <> characters, so a header such as
+
+ From: @a,@b:c@d
+
+is syntactally invalid. Exim does not enforce this restriction.
+
+
+1.6 Local parts [3.4.1]
+-----------------------
+
+[Dots in unquoted local parts may not be consecutive or at either end.]
+
+Exim allows unquoted local parts to begin or end with a dot (period, full
+stop), and it also permits two consecutive dots in a local part.
+
+
+
+2. RFC 821
+----------
+
+The original specification of SMTP is RFC 821, later clarified and modified by
+RFC 1123. Domain name system requirements and their implications for mail are
+covered in RFCs 1035 and 974. A scheme for extending the SMTP protocol is
+described in RFC 1869, and there are subsequent RFCs specifying particular
+extensions.
+
+At the time of writing (January 1999) a new RFC (currently known as
+draft-ietf-drums-smtpupd-09) which updates and consolidates all the material
+connected with SMTP message transmission is at a late stage of drafting, and is
+expected to become an Internet Standard in due course.
+
+The new draft is written using the terms MUST, SHOULD, and MAY, which, when
+written in capital letters, have precise meanings. To quote from the draft:
+
+ "MUST" or "MUST NOT" identify absolute requirements for conformance to
+ this specification. Implementations that do not conform to them lie
+ outside the scope of this specification and often will not
+ interoperate properly with SMTP implementations that do conform.
+ Implementations that are fully conforming also adhere to all "SHOULD"
+ and "SHOULD NOT" requirements. Implementations that adhere to all
+ "MUST" ("MUST NOT") but not to all of these are considered to be
+ partially conforming. Such implementations may interoperate properly
+ with fully conforming ones and with each other, but this will
+ typically be the case only if great care is taken. Consequently, an
+ implementation should violate "SHOULD" ("SHOULD NOT") requirements
+ only under exceptional and well-understood circumstances.
+
+The implementation of Exim is intended to conform to the spirit of this
+paragraph. The following is (I hope) a complete list of major variations
+from the draft RFC. In addition to the items listed here, there are other minor
+extensions such as the tolerance of white space in places where it is not
+strictly permitted by the RFC. References in square brackets are to the -09
+draft sections, and brief summaries of the RFC requirement are also given in
+square brackets.
+
+
+2.1 Line termination [2.3.7, 4.1.1.4]
+-------------------------------------
+
+[SMTP lines are terminated by CRLF.]
+
+Exim recognizes LF without CR as a line terminator in all forms of input. For
+SMTP input, any preceding CR is discarded. An early version of Exim followed
+the RFC strictly, and did not recognize LF without CR in SMTP input. However,
+it seems that sites on the net send out messages with just LF terminators,
+despite the warnings in the RFCs, and other MTAs handle this, so Exim was
+changed. However, there is a compile time macro called STRICT_CRLF which can be
+set to restore the strict behaviour, though this is undocumented.
+
+
+2.2 Eight-bit characters [2.4.1]
+--------------------------------
+
+[SMTP transmits only 7-bit characters.]
+
+Exim is eight-bit clean, and makes no attempt to modify the data in a message
+in any way. In particular, for messages containing characters with the top bit
+set, it neither tries to negotiate 8-bit transmission, nor converts such
+characters into an encoded form. In other words, it adopts the "just send 8"
+strategy. It can be configured to send out 8BITMIME in its response to EHLO
+(which it does not do by default), and it recognizes the 8BITMIME keyword on
+incoming messages, but neither of these affect its handling of message data.
+"Just send 8" is the strategy of a number of MTAs; it is argued that it
+achieves what the user wants more often than other strategies.
+
+
+2.3 Use of EHLO/HELO [3.2]
+--------------------------
+
+[Client MTAs should always start with EHLO, not HELO.]
+
+Exim sends EHLO only when it finds the string "ESMTP" in an SMTP greeting
+message. If EHLO is refused with a 5xx return code, it then reverts to HELO as
+required, but it does not contain logic for converting to HELO on other errors
+such as loss of connection or timeout after EHLO. That is one reason why it
+doesn't always send EHLO; there are reported to be ancient SMTP servers out
+there which collapse on receiving EHLO. (There is also at least one server
+whose banner reads "<host name> ignores ESMTP", but it is RFC 821 compliant in
+that it responds with 5O0 to EHLO, so Exim successfully reverts to HELO.)
+
+
+2.4 Closing the connection [4.1.1.10]
+-------------------------------------
+
+[Client must wait for response to QUIT before closing the connection.]
+
+Exim closes the connection immediately after sending QUIT, without waiting for
+the reply. There was a lot of discussion about this on one of the mailing
+lists. The conclusion was that this behaviour is fine on Unix systems, which
+have TCP/IP implementations that close down the underlying channel tidily even
+when the associated process has terminated. Indeed, not waiting may be
+beneficial, as it moves the TIME_WAIT state (waiting to ensure there's no more
+data in transit) from the server to the client system. On some other operating
+systems (I understand) it is a disaster to terminate the sending process
+without waiting for the QUIT response, because all the data about the
+connection lives in the client's process space, and is therefore thrown away
+before the response arrives. The subsequent arrival of the response then causes
+bad behaviour.
+
+
+2.5 IPv6 address literals [4.1.2]
+---------------------------------
+
+[IPv6 address literals are introduced by "IPv6".]
+
+Exim recognizes IPv6 literals as just the colon-separated hexadecimal form of
+an IPv6 address, for example 1080:0:0:0:8:800:200C:417A, without the need for a
+prefix. At present, it does not even recognize the prefix. When IPv6 becomes
+more widespread, Exim will follow whatever the common usage is.
+
+
+2.6 Underscores in domain names [4.1.2]
+---------------------------------------
+
+[Underscores are not legal in domain names.]
+
+RFC 822 allows all characters except specials, space, and controls in domain
+names, but the SMTP RFCs are stricter, allowing only letters, digits, and
+hyphen. Exim is compliant when checking incoming addresses in SMTP commands,
+but it is more relaxed by default when checking domain names that are supplied
+by EHLO or HELO commands, because many client workstations get set up with
+underscores in their names. There is an option that can be set to cause Exim to
+refuse underscores. (There are also options to specify certain hosts from which
+it will accept any old junk after EHLO or HELO. Such is the woeful state of
+some SMTP clients.)
+
+
+2.7 Removal of return-path headers [4.4]
+----------------------------------------
+
+[Relaying MTAs should not remove return-path.]
+
+Exim removes Return-Path: headers from all messages, if return_path_remove is
+set (the default). It does not attempt to determine if it is being a relay or
+not. Indeed, for some messages it might be both a relay and a final destination
+MTA for the same message.
+
+
+2.8 Randomizing the order of addresses of multihomed hosts [5]
+--------------------------------------------------------------
+
+[Multihomed host addresses should not be randomized.]
+
+Exim does randomize a list of several addresses for a single host, because
+caching in resolvers will defeat the round-robinning that many namerservers
+use. (Note: this is not the same as randomizing equal-valued MX records. That
+is required by the RFC.)
+
+
+2.9 Handling "MX points to self" [5]
+------------------------------------
+
+[MX points to self must be treated as an error.]
+
+The RFC doesn't allow for the possibility of special-purpose routing in the
+case when the lowest numbered MX record points to the local host. The default
+Exim configuration is compliant, but it is possible to configure Exim to behave
+differently, and there are several situations where this can be useful.
+
+
+2.10 Source routing [6.1]
+-------------------------
+
+[Source routes should be stripped.]
+
+The new RFC has moved forward in deprecating source-routed email addresses.
+Exim does not strip them down by default, but can be made to do so by setting
+collapse_source_routes. However, even when it is not stripping them down, it
+does not add host routing to reverse-paths when processing a source-routed
+forward-path.
+
+
+2.11 Loop detection [6.2]
+-------------------------
+
+[Loop count for Received: headers should be at least 100.]
+
+Exim's default setting of the received_headers_max option is 30. Most messages
+these days seem to accumulate less than half a dozen Received: headers, and
+even a couple of forwardings don't bring this anywhere near 30.
+
+
+2.12 Addition of missing headers [6.3]
+--------------------------------------
+
+[Missing headers may be added, and domains qualified, only if client is
+identified.]
+
+Exim always adds Message-Id: and Date: headers if these are missing, whatever
+the source of the message, and likewise when it expands non-fully-qualified
+domains, it does so independently of the message's source.
+
+
+2.13 Syntax of MAIL and RCPT commands [4.1.1.2, 4.1.1.3]
+--------------------------------------------------------
+
+Exim is more relaxed than the RFC requires:
+
+(1) Trailing white space is ignored.
+
+(2) It permits white space after the "FROM" and "TO" keywords.
+
+(3) It does not insist on the address being enclosed in <> characters. In fact,
+ it recognizes addresses in RFC 822 format here, except that domain
+ components are restricted to containing only letters, digits, and hyphens.
+
+(4) Local parts are permitted to contain null components, that is, may start or
+ end with an unquoted full stop (period) or contain two consecutive
+ unquoted full stops.
+
+
+2.14 Non-fully-qualified domains [2.3.5]
+----------------------------------------
+
+[All domains must be fully qualified.]
+
+A domain that is not fully qualified has some of its trailing components
+missing, and is normally a local alias of some sort, for example, just a
+single-component host name.
+
+Exim can be configured to "widen" non-fully-qualified domains, either by using
+the facilities of the DNS resolver, or by an explicit list of widening strings.
+When this is done, it applies to addresses received by SMTP from other hosts,
+as well as to locally-originated addresses. Address re-writing could also be
+used for this purpose.
+
+
+2.15 Unqualified addresses [4.1.2]
+----------------------------------
+
+[Addresses in SMTP commands must include domains.]
+
+An unqualified address consists of a local part without a domain. Do not
+confuse "qualified address" and "qualified domain". A qualified address may
+include a non-fully-qualified domain.
+
+There is one exception to the RFC rule: it is required that the unqualified
+address "<postmaster>" always be accepted. Apart from this, Exim rejects
+domainless addresses in SMTP commands by default, but it can be configured with
+a list of hosts and/or networks that are permitted to send addresses without
+domains in SMTP commands. Any such address that is accepted (including
+<postmaster>) is qualified by adding the value of the qualify_domain option.
+
+
+2.16 VRFY and EXPN [3.5.1, 3.5.2, 3.5.3, 7.3]
+---------------------------------------------
+
+[VRFY and EXPN should be supported.]
+
+Exim does not support VRFY and EXPN by default, but a list of hosts and
+networks for which they are permitted can be given.
+
+
+2.17 Checking of EHLO/HELO commands [4.1.4]
+-------------------------------------------
+
+[Client must send EHLO. Server must not refuse message if EHLO/HELO check
+fails.]
+
+Exim, as a client, always sends EHLO or HELO (see 2.3 above). As a server, it
+does not insist on there having been a valid EHLO or HELO command before the
+start of a message transaction. Any EHLO or HELO command that is received is
+rejected only if it contains a syntax error. That is, it is never rejected on
+the basis of any validation checking that may be performed on the data it
+contains.
+
+However, Exim can be configured to insist that (a) there is valid EHLO/HELO
+command before any message transaction and (b) the domain in that command
+matches the domain obtained by looking up the IP address of the sending host.
+It is possible to specify exception lists of hosts and/or networks for which
+this check does not apply.
+
+
+2.18 Format of delivery error messages [3.7]
+--------------------------------------------
+
+[Standard report formats should be used if possible.]
+
+Exim's delivery failure reports do not conform to the format described in RFC
+1894.
+
+
+## End ##
diff --git a/doc/doc-misc/TexiNotes b/doc/doc-misc/TexiNotes
new file mode 100644
index 000000000..9b16a7ae1
--- /dev/null
+++ b/doc/doc-misc/TexiNotes
@@ -0,0 +1,193 @@
+$Cambridge: exim/doc/doc-misc/TexiNotes,v 1.1 2004/10/08 10:38:47 ph10 Exp $
+
+Notes for conversion of sgcal input into Texinfo input
+------------------------------------------------------
+
+(Dated 6 August 1996)
+
+The escape character is @. Only @ and curly brackets are sensitive. Get them in
+by @@ @{ and @} if required.
+
+@: after a dot that is not a sentence end.
+
+@. instead of . if sentence ends with capital letter
+
+@copyright{} for copyright
+
+@minus{} is a slighly longer minus sign
+
+Input file ends with .texinfo usually.
+
+MUST start the file with
+
+ \input texinfo
+ @c %**start of header
+ @setfilename INFO-FILE-NAME
+ @settitle NAME_OF_MANUAL
+ $c %**end of header
+
+Then, typically
+
+ @ifinfo
+ summary and copyright
+ @end ifinfo
+
+Followed by
+
+ @titlepage
+ title and copyright
+ @end titlepage
+
+Then the top node and master menu - for info file only
+
+ @node Top, First Chapter, (dir), (dir)
+ @comment node-name next, previous, up
+ @top
+
+ @menu
+ * First Chapter:: The first chapter is the
+ only chapter in the sample
+ * Concept Index:: An index
+ @end menu
+
+
+Then the body
+
+ @node First Chapter, Concept Index, Top, Top
+ @comment node-name next, previous, up
+ @chapter First Chapter
+ @cindex Sample index entry
+
+ This is the contents of the first chapter
+ @cindex Another sample index
+
+
+Then stuff about indexes and tables of contents
+
+ @node Concept Index, , First Chapter, Top
+ @unnumbered Concept Index
+
+ @printindex cp
+
+ @contents
+
+MUST end the file with
+
+ @bye
+
+
+. NEWLINE AND NO-FILL MODE
+
+ @page for new page
+ @* forces a line break
+
+
+. LINE CENTERING
+
+ @center stuff
+
+
+. ROMAN, ITALIC, BOLD ITALIC, SMALL CAPS
+
+ @code{...} for 'code' => `...' in info
+ @file{...} for file names => `...' in info
+ @samp{...} for sample text => `...' in info
+ @var{...} for variable => caps in info
+ @dfn{...} defining a term => double quotes in info
+ @emph{...} produces italic
+ @strong{...} produces bold
+ @sc{...} small caps but with letters in lower case.
+ @i italic )
+ @b bold ) no effect on info file
+ @r roman )
+
+
+. TABBING
+
+. CHAPTERS & SECTIONS
+
+ @chapter <title>
+ @unnumbered <title> is an unnumbered chapter
+ @section
+
+
+
+. SECTION
+
+. FANCY VS PLAIN
+
+ @iftex ... @end iftex for printed only; likewise @ifinfo ... @end ifinfo
+
+
+. LEAVING BLANK SPACE
+
+ @sp 10
+
+
+. EM & NEM
+
+ no can no
+
+. DISPLAY ASIS
+
+ @example ... @end example
+ @display ... @end display no change of font => rm
+
+
+. COMMENTS
+
+ @comment or @c introduces comment lines
+
+
+. NUMBERED LISTS
+
+ @enumerate
+ @item
+ first item
+
+ @item
+ second
+ @end enumerate
+
+
+
+. BULLETED LISTS
+
+ @itemize @bullet
+ ...
+
+
+
+. CROSS REFERENCES
+
+ @xref start sentence
+ @ref{name}
+ @pxref (parenthesized)
+
+ 5 args: node name (required), cross-ref name, topic description, name of
+ info file, name of printed manual.
+
+
+
+. TABLES
+
+ @table for two-column tables
+ @table @asis
+
+ @item first column
+ second column
+
+ @item ...
+
+
+
+. INDEX
+
+ @cindex concept index
+ @findex function index
+ @vindex variable index
+ @kindex key index
+ @pindex program index
+ @tindex data type index
+
+***
diff --git a/doc/doc-misc/WishList b/doc/doc-misc/WishList
new file mode 100644
index 000000000..bee4f328a
--- /dev/null
+++ b/doc/doc-misc/WishList
@@ -0,0 +1,1727 @@
+$Cambridge: exim/doc/doc-misc/WishList,v 1.1 2004/10/08 10:38:47 ph10 Exp $
+
+EXIM 4 WISH LIST
+----------------
+
+Even when it was first released, Exim 4 had a Wish List because not all the
+things suggested for it were implemented. The list has not stopped growing...
+
+Another reason it is so long is that I have retained some items from the Exim 3
+Wish List that never got implemented, but which seem reasonable possibilities
+for later addition to Exim 4.
+
+I have guessed at the amount of work involved, and categorized the items as
+Tiny, Small, Medium, Large, or Unknown. The guesses are not based on any
+detailed investigation, so must be taken as very rough.
+
+
+------------------------------------------------------------------------------
+------------------------------------------------------------------------------
+----- Retained from the Exim 3 Wish List ------
+------------------------------------------------------------------------------
+------------------------------------------------------------------------------
+
+(10) 13-Jul-98 M more flexibility for pipe returns
+Ben Smithurst
+
+The ability to specify more precisely what happens concerning the return code
+from the pipe and the presence/absence of STDOUT/STDERR is requested. The
+particular configuration that was requested was:
+
+> if the command exited EX_OK, *and* produced nothing on STDOUT or
+> STDERR, it succeeded...
+> if the command exited EX_TEMPFAIL, defer, regardless of
+> STDOUT/STDERR...
+> otherwise freeze the message (this will get my attention by way of
+> freeze_tell_mailmaster)...
+------------------------------------------------------------------------------
+
+(11) 17-Jul-98 G support for DSN
+Andy Mell
+
+It is unclear to me how this should work in the presence of aliases and
+forwarding. Local deliveries would have to explicitly configured as deliveries
+or relaying or whatever. A substantial amount of code is probably needed.
+
+Jeffrey Goldberg
+I have nothing to add except to say that for many of the reasons you've
+stated, I don't think that DSN is coherent enough to be worth the effort
+to implement.
+
+Another comment:
+
+ I thought the RFC was pretty clear on this. In a nutshell, if the
+ delivery rewrites the envelope from address, it's considered a
+ terminal delivery (i.e. delivery to a mailing list exploder), otherwise
+ treat it as a forwarding operation (the /etc/aliases case). I would
+ treat a .forward expansion as a final delivery event (it got to the
+ user as far as the MTA is concerned).
+
+ Yes, we need the DSN syntax. We also require the complete semantics of
+ NOTIFY=SUCCESS,FAILURE for our application to work.
+
+ Electronic Bill Presentment is really going to push the need for
+ DSN support in MTAs. We just don't want to get stuck in a situation
+ where we're faced with a non-DSN-aware MTA when we go to install
+ our bill/statement engine, thus our interest in what the MTA vendors
+ are planning to do about DSN.
+------------------------------------------------------------------------------
+
+(41) 14-Oct-98 M Find a way of modifying header lines
+Oliver Smith
+
+The problem with header_remove followed by header_add is that you can't refer
+to the previous value of the header when adding a replacement. This could be
+solved with a replace_header option.
+------------------------------------------------------------------------------
+
+(43) 15-Oct-98 M Sender rewrite *after* SMTP incoming checks
+Andreas Edler
+
+The anti-relaying check happens after the sender has been rewritten; there are
+times when it would be helpful to do the check on the original sender, not on
+the rewritten one. Quite how to configure this I'm not sure.
+
+A related suggestion (from Steve Sargent) is to retain the original sender
+address and make it accessible somehow.
+------------------------------------------------------------------------------
+
+(46) 20-Oct-98 L SMTP protocol hooks
+Malcolm Ray
+
+"But there are enough broken SMTP implementations to make me wonder whether
+there isn't a case for providing hooks for tweaking the SMTP transport's
+protocol exchange. Something which would allow me to say things like 'if, when
+talking to lame.example.com, you get a 251 response to a MAIL command, rewrite
+the response to 501 before continuing'."
+------------------------------------------------------------------------------
+
+(50) 13-Nov-98 M A "Focus" option for eximon
+Frank Elsner
+
+This is the opposite of "Hide"; it just displays a certain subset. Hmm. Could
+something clever be done with regular expressions?
+------------------------------------------------------------------------------
+
+(61) 22-Dec-98 M Send failed error messages to somebody
+Harald Meland
+
+With sendmail, the failed error message is made into a error message,
+with both envelope sender and recipient set to MAILER-DAEMON. The
+original, bogus-envelope-sender message is then available to whoever
+receives MAILER-DAEMON's mail. A more flexible approach would be to
+specify a specific recipient.
+------------------------------------------------------------------------------
+
+(81) 01-Mar-99 M Addition of Content-MD5 support
+Martin Hamilton
+
+Martin supplied a suggested patch at
+http://www.net.lut.ac.uk/~martin/antispam/exim-hacks/
+------------------------------------------------------------------------------
+
+(85) 15-Mar-99 M ability to rewrite addresses in non-standard headers
+Dave Lewney
+John Holman
+
+Such as "return-receipt-to". See also 41.
+------------------------------------------------------------------------------
+
+(90) 21-Apr-99 M change wild prefix/suffix greediness
+Ben Smithurst
+
+Currently, when prefix or suffix containing * is set on a director, and the
+fixed part occurs more than once in a local part, the length of the prefix or
+suffix is maximized. For example, with suffix = -* and a local part of
+foo-bar-baz the suffix is taken as bar-baz, leaving the local part as foo.
+An option is proposed to invert this rule.
+------------------------------------------------------------------------------
+
+(91) 26-Apr-99 S make queue_run_in_order to newest first
+"Andreas M. Kirchwitz"
+
+The tidiest thing would be to have queue_run_order={random,oldest,newest},
+and make queue_run_in_order obsolete.
+------------------------------------------------------------------------------
+
+(93) 04-May-1999 L fallback_transport
+
+This would be a generic transport option, specifying a different transport to
+be used if the first one failed. Failed hard, or failed soft? Or an option?
+And if failed hard, is a bounce message sent as well, or not? There are uid
+issues. Remote delivery would have to be done always in a subprocess so that
+the main process could retain privilege in case the fallback transport was
+local. That could be conditional. That's why this is labelled "Large". Some of
+the things people want to do with this can be done by variations in the
+routers, e.g. use $message_age to switch routers.
+------------------------------------------------------------------------------
+
+(94) 13-May-1999 M message to go with -Mg
+Dave Holland
+Alan Thew
+
+So the admin can pass back a reason.
+------------------------------------------------------------------------------
+
+(99) 28-May-1999 M header to list failures for syntax_errors_to
+mark david mcCreary
+
+"I use the syntax_errors_to feature to email a copy of the error message.
+It would be helpful to have the X-Failed-Receipients header in there,
+identifying which addreses(s) are the problem, so that I don't have to
+parse the body of the email message to figure out which addresses."
+------------------------------------------------------------------------------
+
+(100) 04-Jun-1999 S admin_users option, like trusted_users
+Paul Mansfield
+------------------------------------------------------------------------------
+
+(102) 21-Jun-1999 M expanded basic variables
+Julian King
+
+Oh, and a wishlist entry, qualify_domain, and preferably other variables
+can be set with a $lookup in the first part of the exim configuration
+file, perhaps by an equivalent to backticks in shell script ("`command`")?
+------------------------------------------------------------------------------
+
+(105) 28-Jun-1999 M MIME-format bounce messages
+Paul Makepeace
+
+"Is there any work going/gone on/planned to enable exim to report delivery
+status notifications using RFC1892 multipart/report MIME messages? It would be
+great to have errors reported in a message/rfc822 attachment."
+
+Jeffrey Goldberg
+"I like plain bounces, so would hope that if you do this, that it be
+configurable. I think that even for those who want it, it shouldn't be very
+high on the wish list priority."
+
+Other suggestions: toggle for bounces/warnings; override max_return for
+certain addresses; use plain text if original not MIME. See Paul's hack
+for background of what to do.
+
+Nigel suggests using a specially named autoreply transport to generate bounces;
+people could then replace this with another transport (e.g. pipe) if they want
+to customize it themselves.
+
+Eli Chen posted an unconditional patch for 3.32 that does some of this work.
+That could form a basis.
+------------------------------------------------------------------------------
+
+(107) 12-Jul-1999 S defer transport at given load level
+Marc Haber
+
+------------------------------------------------------------------------------
+
+(108) 16-Jul-1999 S remote sort by numbers of recipients
+mark david mcCreary
+
+In the absence of remote_sort, sort remote domains by the number of recipients
+in each.
+------------------------------------------------------------------------------
+
+(114) 11-Nov-1999 S List of possible outgoing interfaces
+
+Allow the smtp "interface" option to be a list: try them in turn until one
+is found to work. Also allow masks to specify a range of addresses.
+------------------------------------------------------------------------------
+
+(123) 23-Dec-1999 L Use AUTH + TURN for dial-in hosts
+Andrew Tverdokhleb
+
+The way to do this would be to have Exim deliver messages into per-host
+directories in, say, BSMTP format. Accept TURN if authenticated, and cause it
+to run a helper program that is passed the socket in order to deliver the mail.
+Provide a helper program!
+------------------------------------------------------------------------------
+
+(125) 04-Jan-2000 L Use shared memory segment for queue list
+Theo Schlossnagle
+
+The idea is that a queue-runner that finds no existing shared segment should
+create one (if configured - possibly some fixed size) and all Exim processes
+should maintain a list of messages in it, thereby saving on directory scans
+when there are lots of messages. This needs a lot of careful thought to try to
+eliminate any possibility of data loss. The interlocking could be quite tricky.
+Further posters suggested using a db file to hold the list. See also 127.
+------------------------------------------------------------------------------
+
+(129) 14-Jan-2000 L Dynamically loadable lookup modules
+Steve Haslam
+
+Suggested patch provided.
+------------------------------------------------------------------------------
+
+(131) 17-Jan-2000 T Facility for assuming existence for EACCES
+Peter Radcliffe
+
+The opposite option for "+" in require_files: assume existence if cannot
+peer into the directory (+ assumes non-existence).
+------------------------------------------------------------------------------
+
+(131) 29-Feb-2000 M? Control total number of outgoing SMTP calls
+Brian White
+
+This is for hosts with slow connections. Could some modification of
+serialize_hosts be used for this? Or maybe use a semaphore? They seem to
+be quite widely available.
+------------------------------------------------------------------------------
+
+(132) 01-Mar-2000 S Lookup host name from outgoing interface
+Vadim Vygonets
+
+Instead of primary_hostname, look up the name for the interface that is being
+used for sending. Suggested patch supplied, but this should be an option of the
+smtp transport.
+------------------------------------------------------------------------------
+
+(133) 06-Mar-2000 S Filter option not to log "previously sent"
+Bruce Bowler
+
+This is when using the "log" option of the autoreply driver.
+------------------------------------------------------------------------------
+
+(134) 09-Mar-2000 S Option to remove attachments when bouncing
+------------------------------------------------------------------------------
+
+(136) 13-Mar-2000 S/M Option for aliasfile to suppress "me too"
+
+Could be tricky determining who "me" is.
+------------------------------------------------------------------------------
+
+(143) 08-May-2000 S Make quota_warn_threshold into a list
+David Carter
+
+So several warnings could be generated as the mailbox got bigger and bigger.
+------------------------------------------------------------------------------
+
+(146) 15-May-2000 M Allow SMTP error codes in retry rules
+
+This would allow special handling of certain errors from certain hosts. In
+particular, it would allow failing of certain 4xx codes.
+
+This is now available for 4xx responses to RCPT commands. Is anything more
+needed?
+------------------------------------------------------------------------------
+
+(148) 15-May-2000 S Warn recipient if message rejected for quota excession.
+Heinz Ekker
+
+Maybe not all that small, because the possibility of retrying must be taken
+into account.
+------------------------------------------------------------------------------
+
+(149) 19-May-2000 L Make added headers visible in filters and other places
+Hans Morten Kind
+
+Headers added by directors/routers are not visible in subsequent processing.
+This is a request to make them visible. What about removed headers? This could
+be tricky to specify, hence the L.
+
+A separate but related issue is the effect of headers added by "unseen"
+directors. These are documented in chapter 19 as not being accumulated. Should
+any change be made?
+------------------------------------------------------------------------------
+
+(155) 16-Jun-2000 M Special handling for certain hosts
+mark david mcCreary
+
+A means of changing the transport depending on the host name/IP of the most
+preferred MX record so that all domains that route to certain hosts can be
+handled specially. Maybe this could be a variable that is available in the
+expansion of the "transport" option.
+------------------------------------------------------------------------------
+
+(158) 29-Jun-2000 S Configure "From" in bounces
+Ben Parker
+
+Cf Reply-To.
+------------------------------------------------------------------------------
+
+(159) 07-Jul-2000 M Keep messages for fixed time
+Gary Palmer
+
+An option to keep messages on the queue for a specified time, even if all their
+destination hosts have timed out.
+------------------------------------------------------------------------------
+
+(164) 17-Aug-2000 S sender_unqualified_auth_hosts
+
+To allow authenticated hosts to send unqualified addresses. Presumably it
+needs received_... as well.
+------------------------------------------------------------------------------
+
+(167) 05-Sep-2000 L Support for ODBC
+
+This would allow access to databases that don't have native support built into
+Exim. See http://www.openlinksw.com/info/docs/rel3doc/unix/odbcsdk.htm
+------------------------------------------------------------------------------
+
+(168) 06-Sep-2000 M Deliver messages that alias to nothing to a given address
+Dr ZP Han
+
+If other people are managing alias lists, and one is empty, bounce that
+delivery to a given address rather than freezing the message. Use the errors_to
+address?
+------------------------------------------------------------------------------
+
+(172) 11-Sep-2000 S Allow file/directory in appendfile to override
+"Michael J. Tubby"
+
+When appendfile is called from forward or filter files, it ignores file or
+directory settings. Maybe they should override. The path set by the forward or
+filter is available in $address_file these days, so it could be used to create
+a longer path.
+------------------------------------------------------------------------------
+
+(173) 18-Sep-2000 S A way of doing lsearches with EOL terminated keys
+Jason Robertson
+
+This is for looking up things like subject contents. Probably need an option to
+exim_dbmbuild to make them into DBM files.
+------------------------------------------------------------------------------
+
+(174) 19-Sep-2000 S A way of using a different port for fallback hosts.
+Dean Brooks
+------------------------------------------------------------------------------
+
+(181) 10-Nov-2000 S Compile-time options for ignoring Sendmail options
+
+So that new ones could be accommodated easily.
+------------------------------------------------------------------------------
+
+(183) 04-Dec-2000 L dns_means_nonexist_after
+Dave C.
+
+In other words, wait a bit before giving up. This needs a mechanism for
+remembering, which is not currently available. To be borne in mind for the
+future.
+------------------------------------------------------------------------------
+
+(184) 04-Dec-2000 M Log more details of local caller
+J. Nick Koston
+
+"I was wondering if it was possible for exim to log the parent pid's cwd and
+exe when it is called from a script/invoked by actually running /usr/sbin/exim
+or /usr/sbin/sendmail." Question: is this information actually/easily
+available to Exim? Needs investigation.
+------------------------------------------------------------------------------
+
+(186) 19-Dec-2000 S A simple utility to reset a retry time
+Marc Haber
+
+Basically, to do what exim_fixdb "delete" can do, but straightforwardly. There
+could be an interface from eximon.
+------------------------------------------------------------------------------
+
+(187) 02-Jan-2001 M Wildcarding in headers_remove
+Tamas TEVESZ
+
+What I'd like to see is it to handle globs (or regexps, but i'm not sure this
+latter would worth the hassle), in a way like:
+
+ headers_remove = "X-*:Additional-header"
+------------------------------------------------------------------------------
+
+(188) 02-Jan-2001 S Make pipe timeout a temporary error
+Georg v.Zezschwitz
+
+A way to make a timeout into a temporary error.
+------------------------------------------------------------------------------
+
+(190) 03-Jan-2001 M Multiple message operations in eximon
+------------------------------------------------------------------------------
+
+(195) 19-Mar-2001 T TCP window size
+
+TCP window size for receiving/sending, SMTP client/server.
+------------------------------------------------------------------------------
+------------------------------------------------------------------------------
+
+
+
+------------------------------------------------------------------------------
+------------------------------------------------------------------------------
+----- Things that didn't make it into Exim 4 ------
+------------------------------------------------------------------------------
+------------------------------------------------------------------------------
+
+. An option to send messages to postmaster when ignore_errmsg_errors_after
+times out.
+
+. When an address is being routed, its constituents are in $local_part and
+$domain, but there is currently no variable that contains the whole thing. It
+could be put into $recipient, but that risks confusion with $recipients
+(which is available in system filters). Maybe $address could be used?
+
+. The ability to relay to host X without knowing all the domains that host X
+might have. At ACL time, one would need to verify the recipient, and determine
+that it routed to host X.
+
+. A new lookup library that operates on a specially prepared file of IP
+addresses and masks so that a single "lookup" yields a yes/no answer. This
+should be a freestanding thing - needs a utility to build the file from a list.
+
+. People want to change the wording of messages; can we find an efficient way
+of allowing this? (Maybe put all messages into a separate module?) The problem
+is not in the messages themselves, but in the values that get inserted into
+messages. Would have to invent a new kind of function that used identified
+values rather than positional ones. Use GNU gettext?
+
+. Invent lf_hosts for those that may use LF without CR. Any other RFC
+things we need to worry about?
+
+. A user would really like to see something similar, perhaps with
+"ID=$authenticated_id", similar to "helo=" and "ident=" in the default received
+header. BUT there are security issues. Maybe give it as a commented out option
+in the default configuration?
+
+. Consider expanding further options that take integer values. What about
+smtp_xxx options for different limits at different times of day (for example)?
+What about tls_advertise_hosts (so can look at incoming IP/port)?
+
+. How about a "hold hosts" option (cf hold_domains) to hold delivery to certain
+hosts?
+
+. Allow user filters to use "headers add", but probably not remove. Or maybe
+just implement "allow" options for both of these features.
+
+. Have the return from pipe in a variable, so that (e.g. error_message_file)
+can make use of it.
+
+. Implement randomize for ldap/sql servers.
+
+. Add an option for ETRN that says "wait for the command to finish, and use its
+stdout as the SMTP response."
+
+. -odsomething for "ignore retry when doing immediate delivery".
+
+. Add an option to the smtp transport to make it treat 5xx on connection as if
+it were 4xx. Or possible add a sophisticated "after command X, treat xxx as
+yyy".
+
+. A way of rewriting addresses in non-standard header lines such as
+Mail-Followup-To.
+
+. Global option to enable initgroups() for exim uid. Default off.
+
+. When verifying a sender, should it be rewritten with any T rewrites, because
+it would be so rewritten if it actually was a recipient in a message?
+
+. Sean Witham wants a way of defining macros that are not privileged, and a
+sort of #ifdef structure that allows for different configurations in the same
+file.
+
+. Allow :fail: to specify that 551 be used instead of 550. Maybe allow a code
+at the start, optionally? What about :defer:?
+
+. SMTP timeout in middle of receiving message: log sender address if known, and
+possibly message_id if known.
+
+. Make -brw show rewrites for transports too.
+
+. Have the MTA log destinations that have timed-out on a ident request and
+no longer send rfc1413_queries to them. Add an option for how not to cache
+these entries.
+
+. Options and/or a utility to enable non-privileged users to view the queue
+(e.g. -bpp), manipulate their own messages, etc.
+
+. Specify a port along with a host in a route_list.
+
+. A generalized "From" escaping scheme that also escapes >From so that the
+whole thing can be reversed.
+
+. There was a request for the \dns_again_means_nonexist\ option not to be
+instantaneous, but to operate only after the DNS has been giving "try again"
+for some time. Use the misc hints database.
+
+
+
+------------------------------------------------------------------------------
+------------------------------------------------------------------------------
+----- The Exim 4 Wish List ------
+------------------------------------------------------------------------------
+------------------------------------------------------------------------------
+
+(1) 01-Jan-02 U Use of dynamically loaded libraries.
+
+People want Exim to use dynamically loaded modules for a variety of reasons.
+When I started to create Exim, I never expected anything other than source
+distribution; the RPMs and inclusions in OS distributions caught me by
+surprise. I know very little about the mechanics of dynamic loading, but I'm
+aware that not all operating systems support it. I'm also aware that not all
+people support it!
+
+Furthermore, a way round this might be to supply more hooks along the lines of
+local_scan(). Then people can write their own dynamic loaders if they want.
+------------------------------------------------------------------------------
+
+(3) 01-Jan-02 U Test for over-quota at SMTP time
+
+This is a hard one, because the only way to test for over quota is to try to
+deliver a message, certainly if system quotas are being used. And also, the
+only available size at RCPT time is the SIZE option, though of course the test
+could be run at DATA time. I think maybe we leave this one to an external
+program, and require people to use ${run} to access the data. Let someone else
+figure out how to extract the current mailbox size!
+
+One suggestion is to implement
+
+ ${file_size:/path/to/file}
+ ${directory_size:/path/to/directory}
+
+so that explicit checks can be done. It may be necessary to have four
+operators, two being based on the block count, and two showing the "visible"
+size. Directory scanning is expensive; is there any scope for caching? It would
+seem not (you don't often get two addresses to the same user).
+------------------------------------------------------------------------------
+
+(4) 01-Jan-02 S Option to reject if no From: or Date: header line
+
+Exim, in common with many other MTAs, inserts a From: or Date: header line if
+one is missing. (It also inserts a blank Bcc:, but that is no longer needed by
+RFC 2822 - it was by 822.) The suggestion is an option to give an error
+instead. This could be done by making it possible to detect these insertions in
+the acl_smtp_data ACL.
+------------------------------------------------------------------------------
+
+(6) 01-Jan-02 S Option to disable the use of -t
+Dave C.
+
+Would require work so that Exim itself doesn't use -t.
+------------------------------------------------------------------------------
+
+(7) 01-Jan-02 M Avoid showing LDAP passwords in log lines for LDAP errors
+John W Baxter
+
+May be tricky, because at the higher levels, the format of the query is not
+understood.
+------------------------------------------------------------------------------
+
+(8) 01-Jan-02 S Expand once_repeat in autoreply
+John Jetmore
+------------------------------------------------------------------------------
+
+(9) 01-Jan-02 S Headers as well as body in file for autoreply
+Florian Laws
+------------------------------------------------------------------------------
+
+(10) 01-Jan-02 T Make "true" and "false" valid expansion conditions
+
+This might help with "and" and "or" when one of the sub-conditions is, for
+example, a lookup.
+------------------------------------------------------------------------------
+
+(11) 01-Jan-02 S Allow a filter to include another file.
+------------------------------------------------------------------------------
+
+(12) 01-Jan-02 M Support for different SQL servers per query
+
+In other words, the global mysql_servers etc. is too restrictive.
+------------------------------------------------------------------------------
+
+(14) 01-Jan-02 M? Support for Sendmail milters
+
+This could perhaps be done by extending the local_scan() idea and providing a
+"standard" module which interfaced to milter.
+------------------------------------------------------------------------------
+
+(15) 01-Jan-02 M More hooks like local_scan()
+
+One request has been for a similar hook at logging time. For other SMTP
+interactions, maybe a hook into the ACL? See also 79 and 218.
+------------------------------------------------------------------------------
+
+(17) 11-Jan-02 M The construction of config.h needs refactoring
+
+This has been hacked about substantially since the original implementation.
+Given that there is a program (buildconfig), the messing around with the
+environment could be abolished. Also, the distinction between "yes" and "no"
+isn't always properly made (tests for #ifdef don't care about the value).
+------------------------------------------------------------------------------
+
+(18) 24-Jan-02 S Make $value retain its value after a top-level expansion
+
+This was specifically for use in filter files. Currently it reverts to empty
+as a consequence of save/restore for every lookup. It might be confusing to
+do otherwise, however.
+------------------------------------------------------------------------------
+
+(19) 29-Jan-02 L Use of multiple DBM libraries
+
+The problem is how to handle conflicting function names. Much research is
+needed.
+------------------------------------------------------------------------------
+
+(20) 29-Jan-02 S Make system filter refreeze after manual thaw
+
+Currently, a "freeze" in a system filter doesn't freeze after a manual thaw.
+------------------------------------------------------------------------------
+
+(21) 12-Feb-02 S Expand return_size_limit
+Joachim Wieland
+
+Is this really worth it? A per-transport value is also suggested - that would
+mean remembering the value with each failed address and taking a minimum or
+a maximimum (which?).
+------------------------------------------------------------------------------
+
+(24) 21-Feb-02 ? A way of testing TLS using -bh
+------------------------------------------------------------------------------
+
+(27) 06-Mar-02 M Distinguishing between different temporary callout errors
+
+The request was to distinguish between a 4xx error and a failure to connect.
+Problem is: how to cope when there is more than one host? Maybe only if ALL
+fail to connect. An option like /callout_no_connect_ok.
+------------------------------------------------------------------------------
+
+(30) 12-Mar-02 S Add "recipients" precondition to routers.
+
+This would avoid having to use "condition". (See also requirement for $address
+mentioned above.) However, it would also require adding a caching feature, and
+probably $recipient_data (cf $domain_data).
+------------------------------------------------------------------------------
+
+(31) 21-Mar-02 S Variables that indicate 8-bit message and 8-bit host, and
+ a way of using them to suppress a transport filter
+
+A variable that is set if the message contains 8-bit characters, and another
+that is set during the smtp transport if the host supports 8-bit. Then we also
+need a condition that's expanded in the transport to control whether the filter
+is run or not (e.g. transport_filter_condition).
+------------------------------------------------------------------------------
+
+(32) 22-Mar-02 M More info about callout fails for header sender verify
+
+When there's a callout failure for an envelope address, the error message
+contains details (by default) of the callout commands. This doesn't happen
+for addresses in the header because there may be more than one of them, and
+deciding how to give that information is tricky. Can we do better?
+------------------------------------------------------------------------------
+
+(33) 25-Mar-02 S Option to assume nomatch in dnslist lookups that time out
+
+Currently this causes a DEFER.
+------------------------------------------------------------------------------
+
+(34) 26-Mar-02 S Access to DNS lookup functions via local_scan() API
+
+This would make local_scan() writers lives easier for DNS usage.
+------------------------------------------------------------------------------
+
+(36) 02-Apr-02 ? A way of throttling, but allowing, relaying that would
+ otherwise be denied
+
+This was suggested in connection with anonymizing messages. The "wait" command
+in ACLs goes some way towards this. Is it enough?
+------------------------------------------------------------------------------
+
+(41) 17-Apr-02 T Make config.samples available as a directory for ftp
+
+This is so that people can browse individual samples directly.
+------------------------------------------------------------------------------
+
+(42) 23-Apr-02 T An option not to flatten newlines in $message_body.
+
+Or maybe better to provide $message_body_nl so as to have both.
+------------------------------------------------------------------------------
+
+(43) 23-Apr-02 T An option to treat 4xx as 5xx from STARTTLS
+
+This would make Exim retry in clear unless the host is in hosts_require_tls.
+------------------------------------------------------------------------------
+
+(44) 24-Apr-02 ? Use errors_to for timeouts after redirect syntax errors
+
+A syntax error in redirection data (with skip_syntax_errors false) causes a
+defer. Eventually, the address may time out. This suggestion is that, when it
+does, the bounce is sent to errors_to rather than to the sender.
+------------------------------------------------------------------------------
+
+(45) 13-May-02 T smtp_etrn_serialize_id = ....
+
+The default behaviour would be equivalent to
+
+ smtp_etrn_serialize_id = $smtp_command_argument
+------------------------------------------------------------------------------
+
+(47) 16-May-02 S Access to all addresses in batched local delivery
+Miquel van Smoorenburg
+
+In a batched local delivery with more than one recipient, there's no way to
+access the list of recipients for doing custom things, such as stuffing them
+all into a header. (BSMTP is the only approach; not everybody can use it.)
+Suggested patch supplied.
+------------------------------------------------------------------------------
+
+(48) 21-May-02 M Support for ATRN (server and client)
+Brian Candler
+
+Server: If Exim had the ability to accept an ATRN command and then simply
+invoke an external program, passing the SMTP stream on stdin and stdout and
+the authenticated id as a parameter, that would do the job nicely.
+
+Client: We need a variant of 'exim -bs' which would connect to a specified
+host, send AUTH/ATRN, and then accept incoming messages as usual.
+------------------------------------------------------------------------------
+
+(50) 22-May-02 T Add comment (duplicate address) to Envelope-To:
+
+This is just to minimize the confusion some people have.
+------------------------------------------------------------------------------
+
+(51) 07-Jun-02 S Option to use another address in callout MAIL FROM
+
+This would be an address to try if MAIL FROM:<> failed. Is this actually going
+to be helpful? See also 101.
+------------------------------------------------------------------------------
+
+(53) 11-Jun-02 S Make local_scan() dynamically loadable
+
+David Woodhouse sent a patch. There's a more sophisticated one from Marc
+Merlin. (See also Peter Benie's comments.) But should the base Exim have all
+this in it?
+------------------------------------------------------------------------------
+
+(54) 11-Jun-02 S Ignore -Ac if called as mailq
+
+I am not sure if this makes sense. This flag requests a listing of a different
+mail queue, but Exim doesn't work like that. Is is not better for people to be
+aware of this?
+------------------------------------------------------------------------------
+
+(55) 13-Jun-02 M Rewriting whole header lines
+Dave C.
+
+Current rewriting rules apply to individual addresses in header lines. This
+feature would use a regex to match whole lines and replace them. It could be
+useful for patching up syntactically invalid lines from crappy clients, before
+the syntax check kicks in. (It might also be useful for hiding local host names
+in Received: headers.)
+------------------------------------------------------------------------------
+
+(58) 26-Jun-02 ? Extend PAM support
+
+Apparently PAM can do challenge-response authentication. The Exim interface
+can't handle this. Investigate and think about how to do this.
+------------------------------------------------------------------------------
+
+(59) 26-Jun-02 M A "custom" authenticator
+
+... that is simply a front end to external code. For example, there may be
+an external API that hides the user password and does CRAM-MD5 when passed the
+details of the challenge and response.
+------------------------------------------------------------------------------
+
+(60) 27-Jun-02 S Make trusted_users a local part list
+
+So that it can use lsearch etc.
+------------------------------------------------------------------------------
+
+(62) 28-Jun-02 S Remove headers before DATA ACL
+Patrice Fournier
+
+"I'd like to be able to give Exim a list of headers that must be removed
+from the message at arrival, before data_acl processing (and before the
+rcpt_acl warn headers are added to the message)."
+------------------------------------------------------------------------------
+
+(63) 28-Jun-02 S Access to ACL-added headers in ACLs
+Patrice Fournier
+
+"I'd like also to be able to look at the already added headers by a
+rcpt_acl when still checking rcpt_acl (either later in the acl for the same
+RCPT TO or for another RCPT TO)."
+------------------------------------------------------------------------------
+
+(65) 28-Jun-02 M Expand fallback hosts
+
+See also 174 of the Exim 3 list.
+------------------------------------------------------------------------------
+
+(66) 01-Jul-02 M Use Berkeley DB 4 concurrent access features
+
+This might give better performance on very busy sites by reducing the
+contention for access to hints databases. Rob Butler points out that this could
+also be useful to allow updates of other DB files used by Exim to happen
+concurrently. Another thing to think about with BDB is the possible use of
+B-trees.
+------------------------------------------------------------------------------
+
+(68) 01-Jul-02 S Add sender host to delivery line
+
+"Would it be possible to have a "sending_host_on_delivery" option that
+logs the IP of the sending host in the => line?" Also requested was amount of
+data transmitted for a non-delivery attempt.
+------------------------------------------------------------------------------
+
+(69) 03-Jul-02 T Log selector to log whoson checs
+Matt Bernstein
+
+"I'd quite like a log_selector option which could spot you'd done a whoson
+lookup in your DATA ACL and maybe log it as W=user."
+------------------------------------------------------------------------------
+
+(70) 09-Jul-02 S A way of changing the RCPT address in an accept router
+
+So as to avoid duplication problems when sending multiple addresses in multiple
+copies to the same address.
+------------------------------------------------------------------------------
+
+(73) 17-Jul-02 M Match a list from within a condition
+
+e.g. ${if matchdomain {$domain}{+domainlist} ...
+ ${if matchhost {$sender_host_address}{1.2.3.4/10:2.3.4.5/16}...
+
+Thought needed about how to handle host names. This may be too messy to specify
+cleanly.
+
+22-Apr-04: Implemented for domains, addresses, and local parts. Hosts are
+too messy!
+------------------------------------------------------------------------------
+
+(74) 22-Jul-02 M Extend -bV to do more semantic checking
+
+For example, diagnose "local_hosts" that should probably be "+local_hosts".
+------------------------------------------------------------------------------
+
+(75) 23-Jul-02 S Reference option on command line
+
+The idea here is that a spam scanner that re-injects a message can supply a
+reference on the command line that gets logged with R=.
+------------------------------------------------------------------------------
+
+(78) 30-Jul-02 S Expand queue_only (and/or queue_only_file)
+
+The requirement is to make it possible to queue messages if certain conditions
+are met (e.g. messages from certain local users). See also 93.
+
+This control can now be achieved in the ACL - is this still needed?
+------------------------------------------------------------------------------
+
+(79) 31-Jul-02 S Additional info for log lines
+
+An option to set an expanded string to be added to <= lines. And also for the
+other delivery lines? See also 15.
+------------------------------------------------------------------------------
+
+(84) 09-Aug-02 S Make interfaces available in a variable
+
+Something like $local_interfaces. Maybe limit the max length.
+------------------------------------------------------------------------------
+
+(85) 12-Aug-02 S/M Notice database connection failures
+
+The small version of this just removes a server from the list within a single
+Exim process when a connection to it fails. The bigger project would use the
+retry database - but that has implications for bottlenecking and may not be
+helpful. See also item 109. Another suggestion is to randomize the order in
+which database servers are tried (randomize_database_servers). And another is
+to measure response times and remember which server is fastest.
+------------------------------------------------------------------------------
+
+(87) 12-Aug-02 M Partial lookups for query-style lookups
+
+The suggestion is to allow the lookup to contain a keystring (same syntax as
+single-key lookups) which is then permuted and place in a suitable variable
+each time - $permuted_key or something.
+------------------------------------------------------------------------------
+
+(88) 20-Aug-02 S Allow special retrying for forced defer
+
+See also 146 in Exim 3 wish list above.
+------------------------------------------------------------------------------
+
+(89) 20-Aug-02 S Also allow retry rules on routers and transports
+------------------------------------------------------------------------------
+
+(90) 23-Aug-02 M Macros with arguments, a la C
+
+I don't like this, because of the cost of frequent interpretation.
+------------------------------------------------------------------------------
+
+(93) 27-Aug-02 S queue_only_condition
+Peter A. Savitch
+
+queue_only_condition global option, expanded string. This contain
+condition, which if evaluated to `no' or `false' or `0', behaves like
+queue_only (queue_only_load ?). Don't know what to do is the string
+expansion fails with DEFER (either force queueing or continue with
+immediate delivery). Another option can control Exim behaviour if the
+expansion fails. Don't know how the name for it ;-) See also 78.
+
+This control can now be achieved in the ACL - is the new feature now needed?
+------------------------------------------------------------------------------
+
+(95) 27-Aug-02 S Log all parents as a router option
+
+So that specific addresses can be logged like this. Should there be more log
+selector options per router? Per transport?
+------------------------------------------------------------------------------
+
+(99) 28-Aug-02 L Test pre-conditions in order given
+
+This would get round certain problems with require_files. However, it is
+totally incompatible, and therefore an "Exim 5" wish.
+------------------------------------------------------------------------------
+
+(101) 02-Oct-02 M Callout and <> rejections
+
+Some people don't want to fail the callout if the MAIL FROM:<> command is
+rejected. Think of a way of handling this tidily. See also 51.
+------------------------------------------------------------------------------
+
+(102) 03-Oct-02 M Log option to suppress message-id logging
+
+M because it would involve a change to eximstats.
+------------------------------------------------------------------------------
+
+(106) 09-Oct-02 S Appendfile to create directory not as user
+
+Arrange for the setup entry to appendfile to create the directory under some
+other uid (and with given owners/permissions?)
+------------------------------------------------------------------------------
+
+(109) 15-Oct-02 M Remember when LDAP (etc) servers are down
+
+The idea would be to use some kind of retry rule, just like for hosts.
+See also 85.
+------------------------------------------------------------------------------
+
+(110) 18-Oct-02 M errors_to for pipe command in filter
+
+To work in the same was as errors_to for deliver commands.
+------------------------------------------------------------------------------
+
+(113) 15-Nov-02 M support for XMLRPC
+
+Patch supplied for 4.10 by Joel Vandal.
+------------------------------------------------------------------------------
+
+(114) 04-Dec-02 M local_scan: return message on accept
+
+(This actually dates from earlier.) The problem with this is that the string
+currently passes into $local_scan_data. Thus, an incompatible change of some
+sort would be required. Possibly a global that local_scan can set?
+------------------------------------------------------------------------------
+
+(118) 10-Dec-02 S access to Perl from local_scan
+------------------------------------------------------------------------------
+
+(119) 12-Dec-02 M ability to specify additional headers in an autoreply
+
+This is so that vacation messages etc can have MIME headers that specify, for
+example, the character set.
+------------------------------------------------------------------------------
+
+(125) 02-Jan-03 M Per-host daemon logging
+
+"So what I would like is an option like debug_hosts, that allows to specify
+an hostlist, and if the current incoming/outgoing hosts matches, creates a
+logfile like $hostname_(in|out).log in my logdirectory."
+------------------------------------------------------------------------------
+
+(127) 06-Jan-03 M Different messages for different callout failures
+
+The real requirement here is to detect when a callout "MAIL FROM:<>" failed, so
+that a specific warning about that can be sent, different to the message when a
+callout "RCPT TO:" fails. I think this is in fact now mostly done.
+------------------------------------------------------------------------------
+
+(129) 09-Jan-03 M Keep track of DNSBL timeouts, and refrain from calling
+
+If so configured, keep track of DNSBL timeouts in a hints record, and don't
+retry that DNSBL for a while after (a sufficient number of) timeouts. It is
+effectively disabled for a while. Log enable/disable, of course. Another
+thought is an option not to apply +defer_unknown unless *all* DNSBL lookups in
+a list defer.
+------------------------------------------------------------------------------
+
+(130) 09-Jan-03 M A number of LDAP-related things
+Peter A. Savitch
+
+OpenLDAP 2.1 is going to be more popular (2.1.9 is available with many
+bug fixes). TLS-enabled LDAP is an interesting and usefull thing.
+I can try to implement some things and send the patches, like with
+ldapi.
+
+How do You see:
+
+1) The propagation of TLS options (key, certificate, CA certificate)
+ to the OpenLDAP library.
+
+2) (was dereferencing; done in 4.23).
+------------------------------------------------------------------------------
+
+(131) 09-Jan-03 S Additional variables
+Peter A. Savitch
+
+$root_uid Why?
+
+(Some that were previously here are done)
+
+$smtp_accept_count -- used for acl_smtp_connect
+
+$queue_runners -- children of the listening daemon could use this
+ value for controlling the number of queue runners
+
+I don't like either of these because they cannot be real-time values. They
+would be snapshots of the values at the time the process was forked from the
+daemon, and I fear they would just be confusing. For processes that were not
+forked from the daemon they couldn't be set at all.
+------------------------------------------------------------------------------
+
+(131) 09-Jan-03 S Additional options
+Peter A. Savitch
+
+exim_processes_max
+exim_file_descriptors_max
+queue_run_condition -- to deprecate queue_run_max, better system
+ load control
+
+Given Exim's distributed nature, I'm not at all sure how the first two of these
+can usefully be implemented.
+------------------------------------------------------------------------------
+
+(132) 16-Jan-03 M Option for when a transport filter fails (crashes)
+
+Freezing is one obvious option. Currently, Exim just retries. Another user
+wanted to retry without the filter, but that is much harder.
+------------------------------------------------------------------------------
+
+(136) 24-Jan-03 M Make "personal" available as a condition for use in routers
+------------------------------------------------------------------------------
+
+(138) 28-Jan-03 M A variable containing what was matched in a host list
+
+Or, presumably, other lists. This is so that ACL messages can say things like
+"your host name matches xxxx". Note: not the same as $domain_data. Also, this
+could be tricky with lookups and things that match in files.
+------------------------------------------------------------------------------
+
+(143) 06-Mar-03 L Ability to have multiple authenticators of same type
+
+For example, to have two PLAIN authenticators; if the first fails, try the
+second.
+------------------------------------------------------------------------------
+
+(144) 07-Mar-03 T ACL control = local_scan_skip to skip the local scan
+
+A bigger project would be control = local_scan <xxx> where xxx could select
+different local_scan functions (possibly by dynamic loading).
+
+This can now be simulated using the fact that ACL variables are preserved,
+so it doesn't look as it once did.
+------------------------------------------------------------------------------
+
+(145) 07-Mar-03 T Export string_cat() to local_scan()
+------------------------------------------------------------------------------
+
+(147) 17-Mar-03 T Option to treat 5xx as 4xx if received on initial connection
+
+This issue is controversial. That may be a good reason for not changing
+anything.
+------------------------------------------------------------------------------
+
+(153) 25-Apr-03 S A way of making log_as_local apply to the smtp transport
+
+Either an option on the transport, or log_remote_as_local for the router.
+Messy, either way. Maybe log_local_as_local and log_remote_as_local, and
+deprecate log_as_local?
+------------------------------------------------------------------------------
+
+(154) 01-May-03 M Teergrubing at the CR/LF level
+
+It is believed that the most effective way to teergrube is to insert a delay
+between transmitting CR and LF in the SMTP response. Furthermore, this is also
+the best place to test for bad synchronization (i.e. at the last possible
+time).
+------------------------------------------------------------------------------
+
+(155) 01-May-03 S "control=no_pipelining" for connect and EHLO ACLs
+
+Yet more flexibility! Maybe this should be a more general control for what is
+sent in response to EHLO.
+------------------------------------------------------------------------------
+
+(156) 06-May-02 M Finer-grained synchronisation checking
+
+On operating systems that can be asked whether any sent bytes have not yet been
+ACK'd at the TCP/IP level, a finer-grained check for proper synchronisation can
+be done. All bytes must have been ACK'd if the client has received the previous
+response before sending the next command. See also 293.
+------------------------------------------------------------------------------
+
+(157) 07-May-03 M Newline as a list item separator
+
+This will make life easier for lists obtained form databases where the
+separator is naturally a newline.
+------------------------------------------------------------------------------
+
+(158) 13-May-03 M Ability to add to OK message for SMTP commands
+
+For sending reasons for slow response, etc.
+------------------------------------------------------------------------------
+
+(160) 19-May-03 M Remove headers using wild cards
+------------------------------------------------------------------------------
+
+(162) 28-May-03 M/L Use of real numbers in filters, expansions, and options
+
+The motivation for this is for handling spam scores that are real numbers. The
+questions are (a) how widely should it spread and (b) whether floating point or
+fixed point representations should be used. And what about the eval operator?
+------------------------------------------------------------------------------
+
+(164) 02-Jun-03 S Set variables for interface and port in smtp transport
+
+These could be useful for varying HELO data etc. See also several other
+items about interfaces above.
+------------------------------------------------------------------------------
+
+(166) 18-Jun-03 S CN verification in client TLS code
+
+A tls_verify_cn option is suggested by Sven Geggus.
+------------------------------------------------------------------------------
+
+(168) 19-Jun-03 S Ability to add a header recording envelope rewrites
+
+Current code adds a deleted header with only some information. Maybe what is
+needed is a flag for a rewrite rule.
+------------------------------------------------------------------------------
+
+(169) 19-Jun-03 M A way of detecting timeouts in callout returns
+
+------------------------------------------------------------------------------
+
+(170) 23-Jun-03 S Option to accept rather than defer after local scan timeout
+
+Suggested patch supplied.
+------------------------------------------------------------------------------
+
+(171) 23-Jun-03 S Option to make timeout a soft failure on pipe transport
+
+------------------------------------------------------------------------------
+
+(172) 23-Jun-03 M Option to make SQL query to specific server
+
+------------------------------------------------------------------------------
+
+(175) 04-Jul-03 S show_all_ancestors_in_errmsg for the redirect router
+
+This is the opposite of hide_child_in_errmsg in effect.
+------------------------------------------------------------------------------
+
+(180) 14-Jul-03 M Extend never_users to be more flexible
+
+e.g. never_users = ! mailnull : ! cyrus : !mailman : 0-100
+------------------------------------------------------------------------------
+
+(183) 16-Jul-03 S freeze_tell_text to add custom text to the message
+
+------------------------------------------------------------------------------
+
+(185) 24-Jul-03 S An expansion operator that decodes RFC 2047 strings
+
+------------------------------------------------------------------------------
+
+(188) 13-Aug-03 T batch_max=0 to mean unlimited
+
+------------------------------------------------------------------------------
+
+(189) 22-Aug-03 S Allow filter "logwrite" to write to syslog
+
+I feel this is a dangerous facility, and also of very minority interest, at
+least for user's filters. Allowing a system filter to write to mainlog or
+syslog may be different. However, writing the main log would only be possible
+if the filter runs as root or exim.
+------------------------------------------------------------------------------
+
+(190) 22-Aug-03 S A way of testing "forced delivery" in filter and routers
+------------------------------------------------------------------------------
+
+(191) 26-Aug-03 M Preserve $address_data for a verified recipient
+
+The idea is to preserve it in the recipients data structure so that local_scan
+can have access to it. The value could also be used as the initial value of
+$address_data while routing.
+------------------------------------------------------------------------------
+
+(192) 05-Sep-03 M Better handling of TXT records for dnslists
+
+When multiple lists are accessible via a merged lookup, handling TXT records
+is difficult. An option for doing the TXT lookup in a sub-list has been
+suggested, with syntax such as
+
+ dnslists = list.example.org=127.0.0.2%dialups \
+ ,127.0.0.3%relays \
+ ,127.0.0.5%spews
+------------------------------------------------------------------------------
+
+(194) 10-Sep-03 M $addresslist_data to be like $host_data/$domain_data
+
+------------------------------------------------------------------------------
+
+(195) 29-Sep-03 M A variable containing the error for verify = header_syntax
+
+Maybe there should always be a variable with the error message for all the
+different kinds of verify failure.
+------------------------------------------------------------------------------
+
+(196) 30-Sep-03 S A way of detecting whether it was HELO or EHLO in the ACL
+
+$received_protocol isn't reset until after the command is accepted (which
+seems right), and $smtp_data shows only the arguments. Maybe $smtp_command?
+------------------------------------------------------------------------------
+
+(197) 30-Sep-03 S MACROS_DROP_PRIVS and ALT_CONFIG_DROP_PRIVS
+
+Now that alternative configurations can be restricted to certain directories,
+some more flexibility can be allowed. Not by default, though.
+------------------------------------------------------------------------------
+
+(198) 01-Oct-03 M Accept mail after local_scan() crash instead of defer
+
+This may not be as easy to implement as it sounds; one is never sure of the
+environment after a crash. Is is actually a good idea? The crashing local_scan
+may have wrecked the memory in arbitrary ways; for example, screwing up the
+recipients list...
+------------------------------------------------------------------------------
+
+(199) 01-Oct-03 M ${pipe which will pipe the message to a script ...
+
+... and otherwise behave as ${run. Probably needs to have locking out features
+so that it can be turned off for users .forwards if the sysadmin so desires.
+------------------------------------------------------------------------------
+
+(200) 07-Oct-03 L Alternative ways of storing hints
+
+People want to store hints in databases. Some assert that SQL databases can
+be made to perform satisfactorily. If a general interface could be worked on,
+people could at least try different strategies. See also 66 above, which is
+specifically concerned with Berkeley DB. Another possible option is a switch to
+disable smtp-wait hints - to avoid contention problems.
+------------------------------------------------------------------------------
+
+(201) 07-Oct-03 M A "soft bounce" feature
+
+This is an option that turns all hard bounces into soft bounces. The idea is
+that it can be used as a safety-net while testing configurations. Instead of a
+local bounce, the message stays on the queue; instead of 5xx SMTP responses,
+4xx ones are given.
+
+The ability to do the opposite - turn 4xx into 5xx under certain circumstances
+might also be useful (e.g. after a certain time). This might best be done by
+extending the retry logic to recognize 4xx as a special error. (This is now
+done.)
+------------------------------------------------------------------------------
+
+(202) 10-Oct-03 S -bvsomething to do a callout after the verify
+------------------------------------------------------------------------------
+
+(203) 14-Oct-03 S verify=something to easily check for header presence
+
+This is purely cosmetic; "condition" can already be used.
+------------------------------------------------------------------------------
+
+(204) 27-Oct-03 S an inverted queue_only_file
+
+That is, queue if a file does NOT exist.
+------------------------------------------------------------------------------
+
+(205) 27-Oct-03 S expand smtp_accept_queue_per_connection
+
+------------------------------------------------------------------------------
+
+(206) 27-Oct-03 S appendfile: a variable containing the maildir base name
+
+------------------------------------------------------------------------------
+
+(207) 29-Oct-03 S ability to keep trusted users in a file - expand it.
+------------------------------------------------------------------------------
+
+(208) 31-Oct-03 M cache temporary verification errors and fail after a time
+
+This request was for a way of turning temporary verification failures into
+permanent ones after some fixed time.
+------------------------------------------------------------------------------
+
+(209) 31-Oct-03 S a way of making crashes in pipe commands temporary errors
+
+------------------------------------------------------------------------------
+
+(210) 31-Oct-03 S runtime option to change the daemon name used for tcprwappers
+
+A patch for compile time was supplied, but this seems better as a runtime
+option, for use with multiple Exim daemons.
+------------------------------------------------------------------------------
+
+(211) 31-Oct-03 S ability to disable debugging output from -bh & -bhc
+------------------------------------------------------------------------------
+
+(212) 31-Oct-03 M specify headers lines in HELO ACL to be added to all msgs
+------------------------------------------------------------------------------
+
+(214) 05-Nov-03 S Put the wild part of local part prefix/suffx in variables
+
+Unfortunately, this isn't quite as trivial as it seems.
+------------------------------------------------------------------------------
+
+(215) 14-Nov-03 S A way of turning off message-submission fix-ups
+
+Globally, and perhaps also via an ACL control so that it can be done on a
+per-message basis.
+------------------------------------------------------------------------------
+
+(215) 26-Nov-03 M/L Conversion of IDNA domain names for logging
+
+IDNA (RFCs 3490-3492) converts domains names containing non-ASCII characters
+into ASCII strings of a special form. Exim will of course handle these.
+However, it might be nice to convert them to a local code for logging. This
+might be quite a big project: there's also output from -bp and eximon queue
+display and no doubt other places as well. (Utilities that process the logs,
+e.g. exigrep, eximstats, will be automatically handled if the logs are
+changed.)
+------------------------------------------------------------------------------
+
+(216) 27-Nov-03 S Option to bounce if required TLS doesn't happen
+
+This is for the smtp transport with hosts_require_tls set. Currently, it
+defers. Possibly the best approach is to make the error one that can be seen by
+the retry logic.
+------------------------------------------------------------------------------
+
+(217) 27-Nov-03 M A function to pass back variables from Perl
+
+This is a function that can be called from Perl, to take a name and a value and
+put that value into an Exim variable.
+------------------------------------------------------------------------------
+
+(218) 01-Dec-03 M A local_scan-like hook at system filter time
+
+That is, make a C API available for custom filtering at this point.
+------------------------------------------------------------------------------
+
+(221) 18-Dec-03 U Merge routers and ACLs - or at least make more similar
+
+"It will be very useful to be able to use most of the ACL conditions
+(authenticated, hosts, senders, sender_domains, ... ) in routers and also the
+possibility to have multiple conditions in routers. It will be great to also
+be able to set variables in routers like in acl's." This is effectively a
+radical suggestion for a complete re-design, and is therefore BIG.
+------------------------------------------------------------------------------
+
+(222) 19-Dec-03 S Iterative option for dnsdb
+
+A way of getting a dnsdb lookup to chop off components until something is
+found: e.g. ${lookup dndsb-i{ns=a.b.c.d}} would look for nameservers for
+a.b.c.d, then b.c.d, etc.
+------------------------------------------------------------------------------
+
+(223) 22-Dec-03 S Support SOA lookup in dnsdb lookups
+------------------------------------------------------------------------------
+
+(225) 22-Dec-03 M Add acl= to routers
+
+This would use an ACL to "control access" to a router, opening up a number
+of interesting possibilities. Details of possible limitations need to be
+investigated.
+------------------------------------------------------------------------------
+
+(226) 23-Dec-03 S A way of treating DEFER as fail in dnsdb lookups
+
+(i.e. the dnsdb lookup failed, so accept the message)
+------------------------------------------------------------------------------
+
+(227) 30-Jan-04 M A configuration .if facility
+
+"Second with the .ifdef and such, it would be nice to have a base .if,
+so I could do something like
+.if DEFINED_DATA == xyz
+configuration here
+.elseif DEFINED_DATA == abc
+configuration here
+.else
+configuration here
+.endif
+also this would be nice at least in my case in the system filters, but
+isn't required but you could pass the defined data to the system, in
+variables."
+------------------------------------------------------------------------------
+
+(229) 30-Jan-04 M New expansion mechanism: {list ...}
+
+"Proposed syntax: {list {separator}{item}{item}...}
+This first expands the contents of {separator} and all of the {item}s,
+then constructs a separator-delimited list. The twist is: if an {item}
+generates the empty string, no separator will be generated for it.
+The entire construct will fail is {separator} fails, or all {item}s
+fail. If just some {item}s fail, they will be treated as if they
+generated empty strings.
+Examples:
+ {list {,}{aaaaaa}{bbbbbb}{cccccc}} -> aaaaaa,bbbbbb,cccccc
+ {list {,}{:fail:}{bbbbbb}{cccccc}} -> bbbbbb,cccccc
+ {list {,}{aaaaaa}{:fail:}{cccccc}} -> aaaaaa,cccccc
+ {list { }{aaaaaa}{bbbbbb}{}} -> aaaaaa bbbbbb
+ {list { }{:fail:}{:fail:}{:fail:}} -> :fail:
+ {list {:fail:}{aaaaa}{bbbb}{cccc}} -> :fail:
+See particularly examples 2-4, which handle the case of a missing first
+and last item with ease; doing this using {if ...} would be quite difficult!"
+------------------------------------------------------------------------------
+
+(230) 30-Jan-04 M Find IP addresses of a domain's nameservers
+
+This needs some way of processing a list of things in a similar way, which
+should perhaps be a more general facility.
+------------------------------------------------------------------------------
+
+(231) 30-Jan-04 ? -C has a number of problems when used for real
+
+-C was intended for testing; people are using it for "alternate"
+configurations, and it doesn't work too well. Can a better way of doing this be
+invented?
+------------------------------------------------------------------------------
+
+(232) 02-Feb-04 ? Make parts of the code loadable
+
+The idea being that drivers, etc. could be compiled separately. There are, of
+course, security issues. This is not something I want to go into at present.
+------------------------------------------------------------------------------
+
+(235) 02-Feb-04 T Make smtp_accept_count available as a variable
+
+This is for use in ACLs. Of course, it is a snapshot of the count at the
+start of the receiving process.
+------------------------------------------------------------------------------
+
+(236) 02-Feb-04 S String in local_scan that's added to the binary version string
+------------------------------------------------------------------------------
+
+(237) 02-Feb-04 M Add_header in ACLs because "message" is overloaded
+
+This would be useful for verbs where "message" is an error message.
+------------------------------------------------------------------------------
+
+(238) 05-Feb-04 S ${address to handle multiple addresses
+
+At present, ${address expects to see just one address. An extension would let
+it handle header lines with multiple addresses, just retaining the actual
+addresses. Or perhaps a new operator is needed?
+------------------------------------------------------------------------------
+
+(239) 23-Feb-04 ? Expansion items for encryption/decryption
+
+Perhaps for some kind of cookie handling? This would need an external crypto
+library, because there's no crypto code in Exim itself.
+------------------------------------------------------------------------------
+
+(240) 23-Feb-04 ? Some way to know if a ip is a mx for a given domain
+
+Some kind of iterative operation for dnsdb might be a general way of providing
+this.
+------------------------------------------------------------------------------
+
+(242) 01-Mar-04 ? Run a filter from an expansion condition
+
+This would add a lot of power to ACLs, but its implementation might be tricky
+because of the possibility of recursion.
+------------------------------------------------------------------------------
+
+(243) 01-Mar-04 ? Run an ACL from an expansion condition
+
+The problem here is knowing what data is available at an arbitrary time.
+------------------------------------------------------------------------------
+
+(244) 01-Mar-04 ? Add an on-success event to transports
+
+This could just be an expansion string, whose value is either ignored or
+logged, but it could be used to run SQL updates or run programs etc.
+However, what is "success" when a transport has multiple recipients?
+------------------------------------------------------------------------------
+
+(245) 01-Mar-04 M Add all the string expansion conditions to filters
+
+Some thought would be needed on how to design the syntax for this.
+------------------------------------------------------------------------------
+
+(247) 09-Mar-04 S IP addresses that are never looked up
+
+It would be nice if we could prevent this for certain IP addresses for
+which we _know_ we'll never get a valid PTR record, like 2002::/16.
+So a new option might reasonably default to:
+
+ hosts_never_lookup = <; 2002::/16
+------------------------------------------------------------------------------
+
+(253) 05-Apr-04 M Use ESMTP and TLS for recipient callout verification
+
+The best way to do this would involve quite a bit of refactoring so as to
+abstract some of the code from the smtp transport into subroutines that could
+also be used from the callout code. The tls parameters should probably be
+taken from the transport. That might also require some substantial code
+refactoring. See also 294.
+------------------------------------------------------------------------------
+
+(260) 30-Apr-04 S Respect +tls_cipher +tls_peerdn in rejectlog entries
+
+------------------------------------------------------------------------------
+
+(261) 05-May-04 S Add a "required_version" option
+
+So that configurations can insist on a specific Exim version.
+------------------------------------------------------------------------------
+
+(262) 10-May-04 S Add "scratch" ACL variables
+
+The idea is for variables that are flushed at the start of each ACL. I'm not
+really convinced that these are worth implementing.
+------------------------------------------------------------------------------
+
+(263) 10-May-04 S Add variable $router_name $transport_name
+
+These could be used in debug_print settings, which are output during -bt, and
+thus don't need the privilege to run with -d.
+------------------------------------------------------------------------------
+
+(265) 25-May-04 M An init.d script for exim is needed
+
+The old sendmail script used to "just work" because it just did -bd -q 20m or
+whatever. Newer versions start more than one sendmail daemon, so do not work.
+------------------------------------------------------------------------------
+
+(267) 25-May-04 S tarpitting delay option
+
+A modifier that sets a delay between lines for multiline responses.
+------------------------------------------------------------------------------
+
+(268) 25-May-04 S? Add a PID to every log line
+
+Given that pids are reused non-cyclically these days, is this actually useful?
+------------------------------------------------------------------------------
+
+(269) 26-May-04 U Run both a system and a user filter in test mode
+
+ exim -bF systemfilter -bf userfilter -f sender@dom < message
+
+This would allow testing the way the userfilter handles the system
+variables set by the systemfilter.
+------------------------------------------------------------------------------
+
+(270) 01-Jun-04 M Add headers at top and middle
+
+Various initiatives like SPF and DomainKeys require header lines to be added
+above or in the middle of existing headers. Exim always adds at the bottom.
+When these requirements are more standard and clearer, some way of controlling
+where header lines are added will probably become necessary. Some new syntax
+will be required.
+
+This can now be done fairly generally from local_scan(), and at the start and
+after the Received: block from an ACL. Is anything more needed?
+------------------------------------------------------------------------------
+
+(271) 02-Jun-04 L Callouts at routing time
+
+From a user's message:
+
+> I would like to be able to:
+>[...]
+> 2) Forcing callouts as address verification at router level
+> [ check_callout just like check_local_user ]
+>
+> I would like to redirect messages in some domain to "domain with callout
+> verification" and to "domain without callout verification"
+>
+> e.g.
+> userA@domain.in -> userX@doamin.out-verify (use callout to verify)
+> userB@domain.in -> userY@doamin.out-noverify (do not use callout verify)
+>
+> [both out-* domains delivered via "callout ready" transports]
+
+Other versions of the wish:
+* limiting callouts in acls to specific transport
+ verify = recipient/callout=5s,transport:intranet_smtp
+* adding "select transport" to ACL conditions
+ accept domains = +local_domains
+ transport = cyrus_ltcp
+ verify = recipient/callout=5s
+------------------------------------------------------------------------------
+
+(272) 07-Jun-04 S Expand hosts_randomize
+
+It occurs in manualroute and in smtp.
+------------------------------------------------------------------------------
+
+(278) 21-Jun-04 M quota_warn_message_file option
+
+Similar to the bounce and delivery warn message files.
+------------------------------------------------------------------------------
+
+(280) 23-Jun-04 M A way of adding a header line after callout defer_ok
+
+This would record that, e.g., a sender domain verified, but the callout
+could not be done.
+------------------------------------------------------------------------------
+
+(285) 16-Jul-04 M Separate and independent log_selector for rejectlog
+
+For example: mainlog_selector and rejectlog_selector, with log_selector setting
+both of them.
+------------------------------------------------------------------------------
+
+(286) 21-Jul-04 M Distinguishing a larger number of errors
+
+For instance, detecting "connection reset by peer" (ENETRESET or ECONNRESET)
+might be useful.
+------------------------------------------------------------------------------
+
+(288) 10-Aug-04 M Option for verify to require MX
+
+e.g. verify=sender/require_mx
+I'm not too keen because this is rather special purpose, and of course could
+only apply if the verification happened to hit a dnslookup router.
+------------------------------------------------------------------------------
+
+(289) 10-Aug-04 L Option to treat defers in database lookups as "not found"
+
+This is so that alternatives can be coded for when databases are down. A
+suggested patch has been sent, but it just catches all instances of "defer"
+from a lookup in an expansion string. These can occur for a number of different
+reasons, not just connection failures. I think that we need a specific
+"connection failed" indicator. Also, what about lookups in lists?
+------------------------------------------------------------------------------
+
+(291) 13-Aug-04 M An ACL or "local_scan()" to be run on size excession
+
+The idea is to give something a chance to look at the data so far received when
+more than message_size_limit (or some other limit) has arrived. I am not sure
+how useful this would actually be in practice.
+------------------------------------------------------------------------------
+
+(292) 13-Aug-04 M Overall timeout for message reception
+
+A client could in priciple keep an SMTP connection open for a very long time by
+trickling in data very slowly. Also, after message_size_limit is exceeded, Exim
+continues to swallow the data (though it does not write it to disk) until the
+end is reached. Again, the connection could be held open for a very long time.
+Some kind of overall time limit for an SMTP connection, possibly reset at the
+start of each message, might be helpful in these situations.
+------------------------------------------------------------------------------
+
+(294) 23-Aug-04 L Callouts and AUTH and LMTP
+
+People want to do callouts using LMTP as well as SMTP, and that would also
+include sockets as well as TCP/IP connections. Also, people want to make use of
+AUTH during the callout checking, on all types of connection. I suppose that
+means making TLS available as well. This probably means a rewrite of the code
+that actually does the callout. Should we use the relevant transport in a new
+"callout" mode instead of keeping things separate? See also 253.
+------------------------------------------------------------------------------
+
+(296) 09-Sep-04 S Make deliver_time work for == lines as well as =>
+
+What about ** lines?
+------------------------------------------------------------------------------
+--- HWM 297 ------------------------------------------------------------------
+---------------------------- End of WishList ---------------------------------
diff --git a/doc/doc-scripts/ABOUT b/doc/doc-scripts/ABOUT
new file mode 100644
index 000000000..623e88c2e
--- /dev/null
+++ b/doc/doc-scripts/ABOUT
@@ -0,0 +1,9 @@
+$Cambridge: exim/doc/doc-scripts/ABOUT,v 1.1 2004/10/08 10:38:48 ph10 Exp $
+
+CVS directory exim/exim-doc/doc-scripts
+---------------------------------------
+
+This directory contains various scripts that are used to build the distributed
+documentation from its source files.
+
+End
diff --git a/doc/doc-src/ABOUT b/doc/doc-src/ABOUT
new file mode 100644
index 000000000..9e281e996
--- /dev/null
+++ b/doc/doc-src/ABOUT
@@ -0,0 +1,11 @@
+$Cambridge: exim/doc/doc-src/ABOUT,v 1.1 2004/10/08 10:38:48 ph10 Exp $
+
+CVS directory exim/exim-doc/doc-src
+-----------------------------------
+
+This directory contains documentation files that are processed in some way in
+order to make the documentation files that form part of Exim distributions. A
+non-standard document processor is currently in use (October 2004), but in the
+long term something more standard will have to take over.
+
+End
diff --git a/doc/doc-txt/ABOUT b/doc/doc-txt/ABOUT
new file mode 100644
index 000000000..b9c355a5a
--- /dev/null
+++ b/doc/doc-txt/ABOUT
@@ -0,0 +1,9 @@
+$Cambridge: exim/doc/doc-txt/ABOUT,v 1.1 2004/10/08 10:38:48 ph10 Exp $
+
+CVS directory exim/exim-doc/doc-txt
+-----------------------------------
+
+This directory contains various documentation files that exist only as plain
+text files, and are distributed in that format.
+
+End
diff --git a/src/ABOUT b/src/ABOUT
new file mode 100644
index 000000000..35857bf71
--- /dev/null
+++ b/src/ABOUT
@@ -0,0 +1,11 @@
+$Cambridge: exim/src/ABOUT,v 1.1 2004/10/08 10:38:48 ph10 Exp $
+
+CVS directory exim/exim-src
+---------------------------
+
+This directory contains everything that is included in an Exim distribution
+tarball, with the exception of the doc directory and an empty Local directory.
+You can build Exim from the contents of this directory by adding a Local
+directory that contains appropriate configuration files.
+
+End