Apply Jakob Hirsch's patch for arbitrary ACL variable names, tidied up

just a little bit.

24 files changed, 335 insertions, 170 deletions

diff --git a/doc/doc-txt/ChangeLog b/doc/doc-txt/ChangeLog
index cf5f158a7..7dc6095d7 100644
--- a/doc/doc-txt/ChangeLog
+++ b/doc/doc-txt/ChangeLog
@@ -1,4 +1,4 @@
-$Cambridge: exim/doc/doc-txt/ChangeLog,v 1.392 2006/09/18 14:49:23 ph10 Exp $
+$Cambridge: exim/doc/doc-txt/ChangeLog,v 1.393 2006/09/19 11:28:44 ph10 Exp $
 
 Change log file for Exim from version 4.21
 -------------------------------------------
@@ -42,6 +42,9 @@ PH/07 There was no check for overflow in expansions such as ${if >{1}{4096M}}.
 
 PH/08 An error is now given if message_size_limit is specified negative.
 
+PH/09 Applied and tidied up Jakob Hirsch's patch for allowing ACL variables
+      to be given arbitrary names.
+
 
 Exim version 4.63
 -----------------
diff --git a/doc/doc-txt/NewStuff b/doc/doc-txt/NewStuff
index 7f00ee5cd..5be3e69bf 100644
--- a/doc/doc-txt/NewStuff
+++ b/doc/doc-txt/NewStuff
@@ -1,52 +1,58 @@
-$Cambridge: exim/doc/doc-txt/NewStuff,v 1.108 2006/07/31 14:19:31 ph10 Exp $
+$Cambridge: exim/doc/doc-txt/NewStuff,v 1.109 2006/09/19 11:28:44 ph10 Exp $
 
 New Features in Exim
 --------------------
 
-This file contains descriptions of new features that have been added to Exim,
-but have not yet made it into the main manual (which is most conveniently
-updated when there is a relatively large batch of changes). The doc/ChangeLog
-file contains a listing of all changes, including bug fixes.
+This file contains descriptions of new features that have been added to Exim.
+Before a formal release, there may be quite a lot of detail so that people can
+test from the snapshots or the CVS before the documentation is updated. Once
+the documentation is updated, this file is reduced to a short list.
+
+
+Version 4.64
+------------
+
+1. ACL variables can now be given arbitrary names, as long as they start with
+   "acl_c" or "acl_m" (for connection variables and message variables) and are
+   at least 6 characters long. However, only alphanumeric characters and
+   underscores are allowed. This is a compatible change because the old set of
+   variables such as acl_m12 are a subset of the allowed names. There may now
+   be any number of ACL variables. For example:
+
+     set acl_c_foo = something
+     set acl_ccc   = thingsome
+     set acl_m13   = value for original ACL variable
+     set acl_m13b  = whatever
+
+   What happens if an undefined ACL variable is referenced depends on the
+   setting of the strict_acl_vars option. If it is false (the default), an
+   empty string is substituted; if it is true, an error is generated. This
+   affects all ACL variables, including the "old" ones such as acl_c4.
+   (Previously there wasn't the concept of an undefined ACL variable.)
+
+   The implementation has been done in such a way that spool files containing
+   ACL variable settings written by previous releases of Exim are compatible
+   and can be read by the new release. If only the original numeric names are
+   used, spool files written by the new release can be read by earlier
+   releases.
+
 
 Version 4.63
 ------------
 
 1. There is a new Boolean option called filter_prepend_home for the redirect
-   router. It defaults true, for backward compatibility. If a "save" command in
-   an Exim filter has a relative path for its argument, and $home is defined,
-   it is automatically prepended to the relative path. This action can now be
-   prevented by setting filter_prepend_home false.
+   router.
 
 2. There is a new acl, set by acl_not_smtp_start, which is run right at the
    start of receiving a non-SMTP message, before any of the message has been
-   read. (This is the analogue of the acl_smtp_predata ACL for SMTP input.) The
-   result of this ACL is ignored; it cannot be used to reject a message. If
-   you really need to, you could set a value in an ACL variable here and reject
-   based on that in the acl_not_smtp ACL. However, this ACL can be used to set
-   controls, and in particular, it can be used to set control=suppress_local_
-   fixups, which cannot be used in the acl_not_smtp ACL because by the time
-   that ACL is run, it is too late. When the acl_not_smtp_start ACL is run, the
-   sender and recipients are known, so the "senders" and "sender_domains"
-   conditions and $sender_address and $recipients variables can be used.
-   Variables such as $authenticated_ sender are also available. It is possible
-   to specify added header lines in this ACL.
+   read.
 
 3. When an SMTP error message is specified in a "message" modifier in an ACL,
    or in a :fail: or :defer: message in a redirect router, Exim now checks the
-   start of the message for an SMTP error code. This consists of three digits
-   followed by a space, optionally followed by an extended code of the form
-   n.n.n, also followed by a space. If this is the case and the very first
-   digit is the same as the default error code, the code from the message is
-   used instead. If the very first digit is incorrect, a panic error is logged,
-   and the default code is used. This is an incompatible change, but it is not
-   expected to affect many (if any) configurations. It is possible to suppress
-   the use of the supplied code in a redirect router by setting the
-   forbid_smtp_code option true. In this case, any SMTP code is quietly
-   ignored.
+   start of the message for an SMTP error code.
 
 4. There is a new parameter for LDAP lookups called "referrals", which takes
-   one of the settings "follow" (the default) or "nofollow". The latter stops
-   the LDAP library from trying to follow referrals issued by the LDAP server.
+   one of the settings "follow" (the default) or "nofollow".
 
 5. Version 20070721.2 of exipick now included, offering these new options:
     --reverse
diff --git a/doc/doc-txt/OptionLists.txt b/doc/doc-txt/OptionLists.txt
index 518cce2c1..26546e1ba 100644
--- a/doc/doc-txt/OptionLists.txt
+++ b/doc/doc-txt/OptionLists.txt
@@ -1,4 +1,4 @@
-$Cambridge: exim/doc/doc-txt/OptionLists.txt,v 1.22 2006/07/31 14:19:31 ph10 Exp $
+$Cambridge: exim/doc/doc-txt/OptionLists.txt,v 1.23 2006/09/19 11:28:44 ph10 Exp $
 
 LISTS OF EXIM OPTIONS
 ---------------------
@@ -11,7 +11,7 @@ This file contains complete lists of four kinds of Exim option:
   4. Those that can appear in the build time configuration for the Exim monitor
      (Local/eximon.conf).
 
-This file was last updated for Exim release 4.63.
+This file was last updated for Exim release 4.64.
 
 
 1. RUN TIME OPTIONS
@@ -484,6 +484,7 @@ spamd_address                        string          +             main
 split_spool_directory                boolean         false         main              1.70
 spool_directory                      string          ++            main
 sqlite_lock_timeout                  time            5s            main              4.53
+strict_acl_vars                      boolean         false         main              4.64
 srv_fail_domains                     domain list     unset         dnslookup         4.43
 strip_excess_angle_brackets          boolean         false         main
 strip_trailing_dot                   boolean         false         main
@@ -740,8 +741,6 @@ to stop you overriding it in your Local/Makefile.
 Option                       Type         Description
 ------------------------------------------------------------------------------
 
-ACL_CVARS=20                 optional*    number of acl_c variables
-ACL_MVARS=20                 optional*    number of acl_m variables
 ALT_CONFIG_PREFIX            optional     restricts location of -C files
 ALT_CONFIG_ROOT_ONLY         optional*    privileged -C needs root (not exim)
 APPENDFILE_MODE              optional*
diff --git a/src/ACKNOWLEDGMENTS b/src/ACKNOWLEDGMENTS
index c04f4018c..43b171e00 100644
--- a/src/ACKNOWLEDGMENTS
+++ b/src/ACKNOWLEDGMENTS
@@ -1,4 +1,4 @@
-$Cambridge: exim/src/ACKNOWLEDGMENTS,v 1.54 2006/07/26 14:39:13 ph10 Exp $
+$Cambridge: exim/src/ACKNOWLEDGMENTS,v 1.55 2006/09/19 11:28:44 ph10 Exp $
 
 EXIM ACKNOWLEDGEMENTS
 
@@ -20,7 +20,7 @@ relatively small patches.
 Philip Hazel
 
 Lists created: 20 November 2002
-Last updated:  25 July 2006
+Last updated:  19 September 2006
 
 
 THE OLD LIST
@@ -158,6 +158,7 @@ Steve Haslam              Lots of stuff, including
 Sheldon Hearn             Suggested patch for smtp_accept_max_nonmail_hosts
 Bryan Henderson           Patch to use RM_COMMAND everywhere during building
 Jakob Hirsch              Patch for % operator
+                          Patch for arbitrarily named ACL variables
 Magnus Holmgren           Patch for filter_prepend_home
                           Patch for "h" flag in Domain Keys
 Kjetil Torgrim Homme      Patch for require_files problem on NFS file systems
diff --git a/src/exim_monitor/em_globals.c b/src/exim_monitor/em_globals.c
index 621d8b207..361ac4e78 100644
--- a/src/exim_monitor/em_globals.c
+++ b/src/exim_monitor/em_globals.c
@@ -1,4 +1,4 @@
-/* $Cambridge: exim/src/exim_monitor/em_globals.c,v 1.9 2006/02/07 11:18:59 ph10 Exp $ */
+/* $Cambridge: exim/src/exim_monitor/em_globals.c,v 1.10 2006/09/19 11:28:45 ph10 Exp $ */
 
 /*************************************************
 *                Exim Monitor                    *
@@ -117,9 +117,8 @@ that fires up the monitor fishes the value out by using -bP anyway. */
 #define SPOOL_DIRECTORY ""
 #endif
 
-
-uschar *acl_var[ACL_CVARS + ACL_MVARS];
-
+tree_node *acl_var_c           = NULL;
+tree_node *acl_var_m           = NULL;
 uschar *active_hostname        = NULL;
 BOOL    allow_unqualified_recipient = FALSE;
 BOOL    allow_unqualified_sender = FALSE;
diff --git a/src/exim_monitor/em_queue.c b/src/exim_monitor/em_queue.c
index ec54116a0..212b15136 100644
--- a/src/exim_monitor/em_queue.c
+++ b/src/exim_monitor/em_queue.c
@@ -1,4 +1,4 @@
-/* $Cambridge: exim/src/exim_monitor/em_queue.c,v 1.4 2006/02/14 14:26:15 ph10 Exp $ */
+/* $Cambridge: exim/src/exim_monitor/em_queue.c,v 1.5 2006/09/19 11:28:45 ph10 Exp $ */
 
 /*************************************************
 *                 Exim Monitor                   *
@@ -125,6 +125,29 @@ store_free(p);
 
 
 /*************************************************
+*         Set up an ACL variable                 *
+*************************************************/
+
+/* The spool_read_header() function calls acl_var_create() when it reads in an
+ACL variable. We know that in this case, the variable will be new, not re-used,
+so this is a cut-down version, to save including the whole acl.c module (which
+would need conditional compilation to cut most of it out). */
+
+tree_node *
+acl_var_create(uschar *name)
+{
+tree_node *node, **root;
+root = (name[0] == 'c')? &acl_var_c : &acl_var_m;
+node = store_get(sizeof(tree_node) + Ustrlen(name));
+Ustrcpy(node->name, name);
+node->data.ptr = NULL;
+(void)tree_insertnode(root, node);
+return node;
+}
+
+
+
+/*************************************************
 *             Set up new queue item              *
 *************************************************/
 
diff --git a/src/src/acl.c b/src/src/acl.c
index 6c06e466c..8c2ab699a 100644
--- a/src/src/acl.c
+++ b/src/src/acl.c
@@ -1,4 +1,4 @@
-/* $Cambridge: exim/src/src/acl.c,v 1.63 2006/09/05 14:05:43 ph10 Exp $ */
+/* $Cambridge: exim/src/src/acl.c,v 1.64 2006/09/19 11:28:45 ph10 Exp $ */
 
 /*************************************************
 *     Exim - an Internet mail transport agent    *
@@ -805,36 +805,44 @@ while ((s = (*func)()) != NULL)
 
   /* The "set" modifier is different in that its argument is "name=value"
   rather than just a value, and we can check the validity of the name, which
-  gives us a variable number to insert into the data block. */
+  gives us a variable name to insert into the data block. The original ACL
+  variable names were acl_c0 ... acl_c9 and acl_m0 ... acl_m9. This was
+  extended to 20 of each type, but after that people successfully argued for
+  arbitrary names. For compatibility, however, the names must still start with
+  acl_c or acl_m. After that, we allow alphanumerics and underscores. */
 
   if (c == ACLC_SET)
     {
-    int offset, max, n;
     uschar *endptr;
 
-    if (Ustrncmp(s, "acl_", 4) != 0) goto BAD_ACL_VAR;
-    if (s[4] == 'c')
+    if (Ustrncmp(s, "acl_c", 5) != 0 &&
+        Ustrncmp(s, "acl_m", 5) != 0)
       {
-      offset = 0;
-      max = ACL_CVARS;
+      *error = string_sprintf("invalid variable name after \"set\" in ACL "
+        "modifier \"set %s\" (must start \"acl_c\" or \"acl_m\")", s);
+      return NULL;
       }
-    else if (s[4] == 'm')
+
+    endptr = s + 5;
+    while (*endptr != 0 && *endptr != '=' && !isspace(*endptr))
       {
-      offset = ACL_CVARS;
-      max = ACL_MVARS;
+      if (!isalnum(*endptr) && *endptr != '_')
+        {
+        *error = string_sprintf("invalid character \"%c\" in variable name "
+          "in ACL modifier \"set %s\"", *endptr, s);
+        return NULL;
+        }
+      endptr++;
       }
-    else goto BAD_ACL_VAR;
 
-    n = Ustrtoul(s + 5, &endptr, 10);
-    if ((*endptr != 0 && *endptr != '=' && !isspace(*endptr)) || n >= max)
+    if (endptr - s < 6)
       {
-      BAD_ACL_VAR:
-      *error = string_sprintf("syntax error or unrecognized name after "
-        "\"set\" in ACL modifier \"set %s\"", s);
+      *error = string_sprintf("invalid variable name after \"set\" in ACL "
+        "modifier \"set %s\" (must be at least 6 characters)", s);
       return NULL;
       }
 
-    cond->u.varnumber = n + offset;
+    cond->u.varname = string_copyn(s + 4, endptr - s - 4);
     s = endptr;
     while (isspace(*s)) s++;
     }
@@ -2426,11 +2434,8 @@ for (; cb != NULL; cb = cb->next)
 
     if (cb->type == ACLC_SET)
       {
-      int n = cb->u.varnumber;
-      int t = (n < ACL_CVARS)? 'c' : 'm';
-      if (n >= ACL_CVARS) n -= ACL_CVARS;
-      debug_printf("acl_%c%d ", t, n);
-      lhswidth += 7;
+      debug_printf("acl_%s ", cb->u.varname);
+      lhswidth += 5 + Ustrlen(cb->u.varname);
       }
 
     debug_printf("= %s\n", cb->arg);
@@ -2926,8 +2931,8 @@ for (; cb != NULL; cb = cb->next)
     case ACLC_SET:
       {
       int old_pool = store_pool;
-      if (cb->u.varnumber < ACL_CVARS) store_pool = POOL_PERM;
-      acl_var[cb->u.varnumber] = string_copy(arg);
+      if (cb->u.varname[0] == 'c') store_pool = POOL_PERM;
+      acl_var_create(cb->u.varname)->data.ptr = string_copy(arg);
       store_pool = old_pool;
       }
     break;
@@ -3597,4 +3602,64 @@ if (rc != OK && *user_msgptr != NULL && Ustrlen(*user_msgptr) > 75)
 return rc;
 }
 
+
+
+/*************************************************
+*             Create ACL variable                *
+*************************************************/
+
+/* Create an ACL variable or reuse an existing one. ACL variables are in a
+binary tree (see tree.c) with acl_var_c and acl_var_m as root nodes.
+
+Argument:
+  name    pointer to the variable's name, starting with c or m
+
+Returns   the pointer to variable's tree node
+*/
+
+tree_node *
+acl_var_create(uschar *name)
+{
+tree_node *node, **root;
+root = (name[0] == 'c')? &acl_var_c : &acl_var_m;
+node = tree_search(*root, name);
+if (node == NULL)
+  {
+  node = store_get(sizeof(tree_node) + Ustrlen(name));
+  Ustrcpy(node->name, name);
+  (void)tree_insertnode(root, node);
+  }
+node->data.ptr = NULL;
+return node;
+}
+
+
+
+/*************************************************
+*       Write an ACL variable in spool format    *
+*************************************************/
+
+/* This function is used as a callback for tree_walk when writing variables to
+the spool file. To retain spool file compatibility, what is written is -aclc or
+-aclm followed by the rest of the name and the data length, space separated,
+then the value itself, starting on a new line, and terminated by an additional
+newline. When we had only numbered ACL variables, the first line might look
+like this: "-aclc 5 20". Now it might be "-aclc foo 20" for the variable called
+acl_cfoo.
+
+Arguments:
+  name    of the variable
+  value   of the variable
+  ctx     FILE pointer (as a void pointer)
+
+Returns:  nothing
+*/
+
+void
+acl_var_write(uschar *name, uschar *value, void *ctx)
+{
+FILE *f = (FILE *)ctx;
+fprintf(f, "-acl%c %s %d\n%s\n", name[0], name+1, Ustrlen(value), value);
+}
+
 /* End of acl.c */
diff --git a/src/src/config.h.defaults b/src/src/config.h.defaults
index 92f894e4c..e754e39e4 100644
--- a/src/src/config.h.defaults
+++ b/src/src/config.h.defaults
@@ -1,4 +1,4 @@
-/* $Cambridge: exim/src/src/config.h.defaults,v 1.10 2006/02/10 14:25:43 ph10 Exp $ */
+/* $Cambridge: exim/src/src/config.h.defaults,v 1.11 2006/09/19 11:28:45 ph10 Exp $ */
 
 /*************************************************
 *     Exim - an Internet mail transport agent    *
@@ -12,9 +12,6 @@ any data just defines the existence of the variable; it won't get included
 in config.h unless some value is defined in Local/Makefile. If there is data,
 it's a default value. */
 
-#define ACL_CVARS                    20
-#define ACL_MVARS                    20
-
 #define ALT_CONFIG_PREFIX
 #define ALT_CONFIG_ROOT_ONLY
 
diff --git a/src/src/expand.c b/src/src/expand.c
index 1517c0625..2e39f2638 100644
--- a/src/src/expand.c
+++ b/src/src/expand.c
@@ -1,4 +1,4 @@
-/* $Cambridge: exim/src/src/expand.c,v 1.60 2006/09/18 14:49:23 ph10 Exp $ */
+/* $Cambridge: exim/src/src/expand.c,v 1.61 2006/09/19 11:28:45 ph10 Exp $ */
 
 /*************************************************
 *     Exim - an Internet mail transport agent    *
@@ -300,6 +300,8 @@ enum {
 /* This table must be kept in alphabetical order. */
 
 static var_entry var_table[] = {
+  /* WARNING: Do not invent variables whose names start acl_c or acl_m because
+     they will be confused with user-creatable ACL variables. */
   { "acl_verify_message",  vtype_stringptr,   &acl_verify_message },
   { "address_data",        vtype_stringptr,   &deliver_address_data },
   { "address_file",        vtype_stringptr,   &address_file },
@@ -1231,37 +1233,25 @@ find_variable(uschar *name, BOOL exists_only, BOOL skipping, int *newsize)
 int first = 0;
 int last = var_table_size;
 
-/* Handle ACL variables, which are not in the table because their number may
-vary depending on a build-time setting. If the variable's name is not of the
-form acl_mddd or acl_cddd, where the d's are digits, fall through to look for
-other names that start with acl_. */
+/* Handle ACL variables, whose names are of the form acl_cxxx or acl_mxxx.
+Originally, xxx had to be a number in the range 0-9 (later 0-19), but from
+release 4.64 onwards arbitrary names are permitted, as long as the first 5
+characters are acl_c or acl_m (this gave backwards compatibility at the
+changeover). There may be built-in variables whose names start acl_ but they
+should never start acl_c or acl_m. This slightly messy specification is a
+consequence of the history, needless to say.
 
-if (Ustrncmp(name, "acl_", 4) == 0)
-  {
-  uschar *endptr;
-  int offset = -1;
-  int max = 0;
-
-  if (name[4] == 'm')
-    {
-    offset = ACL_CVARS;
-    max = ACL_MVARS;
-    }
-  else if (name[4] == 'c')
-    {
-    offset = 0;
-    max = ACL_CVARS;
-    }
+If an ACL variable does not exist, treat it as empty, unless strict_acl_vars is
+set, in which case give an error. */
 
-  if (offset >= 0)
-    {
-    int n = Ustrtoul(name + 5, &endptr, 10);
-    if (*endptr == 0 && n < max)
-      return (acl_var[offset + n] == NULL)? US"" : acl_var[offset + n];
-    }
+if (Ustrncmp(name, "acl_c", 5) == 0 || Ustrncmp(name, "acl_m", 5) == 0)
+  {
+  tree_node *node =
+    tree_search((name[4] == 'c')? acl_var_c : acl_var_m, name + 4);
+  return (node == NULL)? (strict_acl_vars? NULL : US"") : node->data.ptr;
   }
 
-/* Similarly for $auth<n> variables. */
+/* Handle $auth<n> variables. */
 
 if (Ustrncmp(name, "auth", 4) == 0)
   {
@@ -1681,6 +1671,13 @@ switch(cond_type)
       expand_string_message = (name[0] == 0)?
         string_sprintf("variable name omitted after \"def:\"") :
         string_sprintf("unknown variable \"%s\" after \"def:\"", name);
+
+      if (strict_acl_vars &&
+          Ustrncmp(name, "acl_", 4) == 0 &&
+          (name[4] == 'c' || name[4] == 'm'))
+        expand_string_message = string_sprintf("%s (strict_acl_vars is set)",
+          expand_string_message);
+
       return NULL;
       }
     if (yield != NULL) *yield = (value[0] != 0) == testfor;
@@ -2959,6 +2956,13 @@ while (*s != 0)
         {
         expand_string_message =
           string_sprintf("unknown variable name \"%s\"", name);
+
+        if (strict_acl_vars &&
+            Ustrncmp(name, "acl_", 4) == 0 &&
+            (name[4] == 'c' || name[4] == 'm'))
+          expand_string_message = string_sprintf("%s (strict_acl_vars is set)",
+            expand_string_message);
+
         goto EXPAND_FAILED;
         }
       }
@@ -5118,6 +5122,13 @@ while (*s != 0)
       {
       expand_string_message =
         string_sprintf("unknown variable in \"${%s}\"", name);
+
+      if (strict_acl_vars &&
+          Ustrncmp(name, "acl_", 4) == 0 &&
+          (name[4] == 'c' || name[4] == 'm'))
+        expand_string_message = string_sprintf("%s (strict_acl_vars is set)",
+          expand_string_message);
+
       goto EXPAND_FAILED;
       }
     len = Ustrlen(value);
diff --git a/src/src/functions.h b/src/src/functions.h
index eb1a7a5b4..85b48ee37 100644
--- a/src/src/functions.h
+++ b/src/src/functions.h
@@ -1,4 +1,4 @@
-/* $Cambridge: exim/src/src/functions.h,v 1.26 2006/09/18 14:49:23 ph10 Exp $ */
+/* $Cambridge: exim/src/src/functions.h,v 1.27 2006/09/19 11:28:45 ph10 Exp $ */
 
 /*************************************************
 *     Exim - an Internet mail transport agent    *
@@ -40,6 +40,8 @@ extern int     tls_write(const uschar *, size_t);
 
 extern acl_block *acl_read(uschar *(*)(void), uschar **);
 extern int     acl_check(int, uschar *, uschar *, uschar **, uschar **);
+extern tree_node *acl_var_create(uschar *);
+extern void    acl_var_write(uschar *, uschar *, void *);
 extern uschar *auth_b64encode(uschar *, int);
 extern int     auth_b64decode(uschar *, uschar **);
 extern int     auth_call_pam(uschar *, uschar **);
@@ -327,6 +329,7 @@ extern void    tree_add_unusable(host_item *);
 extern int     tree_insertnode(tree_node **, tree_node *);
 extern tree_node *tree_search(tree_node *, uschar *);
 extern void    tree_write(tree_node *, FILE *);
+extern void    tree_walk(tree_node *, void (*)(uschar*, uschar*, void*), void *);
 
 #ifdef WITH_CONTENT_SCAN
 extern void    unspool_mbox(void);
diff --git a/src/src/globals.c b/src/src/globals.c
index 84d564ab8..a694053c5 100644
--- a/src/src/globals.c
+++ b/src/src/globals.c
@@ -1,4 +1,4 @@
-/* $Cambridge: exim/src/src/globals.c,v 1.57 2006/07/28 11:39:11 ph10 Exp $ */
+/* $Cambridge: exim/src/src/globals.c,v 1.58 2006/09/19 11:28:45 ph10 Exp $ */
 
 /*************************************************
 *     Exim - an Internet mail transport agent    *
@@ -186,7 +186,8 @@ uschar *acl_smtp_rcpt          = NULL;
 uschar *acl_smtp_starttls      = NULL;
 uschar *acl_smtp_vrfy          = NULL;
 BOOL    acl_temp_details       = FALSE;
-uschar *acl_var[ACL_CVARS + ACL_MVARS];
+tree_node *acl_var_c           = NULL;
+tree_node *acl_var_m           = NULL;
 uschar *acl_verify_message     = NULL;
 string_item *acl_warn_logged   = NULL;
 
@@ -1102,6 +1103,7 @@ uschar *srs_status             = NULL;
 BOOL    srs_usehash            = TRUE;
 BOOL    srs_usetimestamp       = TRUE;
 #endif
+BOOL    strict_acl_vars        = FALSE;
 int     string_datestamp_offset= -1;
 BOOL    strip_excess_angle_brackets = FALSE;
 BOOL    strip_trailing_dot     = FALSE;
diff --git a/src/src/globals.h b/src/src/globals.h
index a235869c1..cd5fb4e36 100644
--- a/src/src/globals.h
+++ b/src/src/globals.h
@@ -1,4 +1,4 @@
-/* $Cambridge: exim/src/src/globals.h,v 1.39 2006/07/13 13:53:33 ph10 Exp $ */
+/* $Cambridge: exim/src/src/globals.h,v 1.40 2006/09/19 11:28:45 ph10 Exp $ */
 
 /*************************************************
 *     Exim - an Internet mail transport agent    *
@@ -128,7 +128,8 @@ extern uschar *acl_smtp_rcpt;          /* ACL run for RCPT */
 extern uschar *acl_smtp_starttls;      /* ACL run for STARTTLS */
 extern uschar *acl_smtp_vrfy;          /* ACL run for VRFY */
 extern BOOL    acl_temp_details;       /* TRUE to give details for 4xx error */
-extern uschar *acl_var[ACL_CVARS+ACL_MVARS]; /* User ACL variables */
+extern tree_node *acl_var_c;           /* ACL connection variables */
+extern tree_node *acl_var_m;           /* ACL messsage variables */
 extern uschar *acl_verify_message;     /* User message for verify failure */
 extern string_item *acl_warn_logged;   /* Logged lines */
 extern uschar *acl_wherecodes[];       /* Response codes for ACL fails */
@@ -688,6 +689,7 @@ extern uschar *srs_status;             /* SRS staus */
 extern BOOL    srs_usehash;            /* SRS use hash flag */
 extern BOOL    srs_usetimestamp;       /* SRS use timestamp flag */
 #endif
+extern BOOL    strict_acl_vars;        /* ACL variables have to be set before being used */
 extern int     string_datestamp_offset;/* After insertion by string_format */
 extern BOOL    strip_excess_angle_brackets; /* Surrounding route-addrs */
 extern BOOL    strip_trailing_dot;     /* Remove dots at ends of domains */
diff --git a/src/src/readconf.c b/src/src/readconf.c
index 75d444ef1..fd04ab328 100644
--- a/src/src/readconf.c
+++ b/src/src/readconf.c
@@ -1,4 +1,4 @@
-/* $Cambridge: exim/src/src/readconf.c,v 1.23 2006/06/28 16:00:24 ph10 Exp $ */
+/* $Cambridge: exim/src/src/readconf.c,v 1.24 2006/09/19 11:28:45 ph10 Exp $ */
 
 /*************************************************
 *     Exim - an Internet mail transport agent    *
@@ -354,6 +354,7 @@ static optionlist optionlist_config[] = {
   { "srs_usehash",              opt_bool,        &srs_usehash },
   { "srs_usetimestamp",         opt_bool,        &srs_usetimestamp },
 #endif
+  { "strict_acl_vars",          opt_bool,        &strict_acl_vars },
   { "strip_excess_angle_brackets", opt_bool,     &strip_excess_angle_brackets },
   { "strip_trailing_dot",       opt_bool,        &strip_trailing_dot },
   { "syslog_duplication",       opt_bool,        &syslog_duplication },
diff --git a/src/src/smtp_in.c b/src/src/smtp_in.c
index 6c27c15de..614a3ffe0 100644
--- a/src/src/smtp_in.c
+++ b/src/src/smtp_in.c
@@ -1,4 +1,4 @@
-/* $Cambridge: exim/src/src/smtp_in.c,v 1.42 2006/09/18 14:49:23 ph10 Exp $ */
+/* $Cambridge: exim/src/src/smtp_in.c,v 1.43 2006/09/19 11:28:45 ph10 Exp $ */
 
 /*************************************************
 *     Exim - an Internet mail transport agent    *
@@ -793,8 +793,6 @@ return TRUE;
 
 
 
-
-
 /*************************************************
 *         Reset for new message                  *
 *************************************************/
@@ -809,7 +807,6 @@ Returns:    nothing
 static void
 smtp_reset(void *reset_point)
 {
-int i;
 store_reset(reset_point);
 recipients_list = NULL;
 rcpt_count = rcpt_defer_count = rcpt_fail_count =
@@ -855,9 +852,9 @@ sender_rate = sender_rate_limit = sender_rate_period = NULL;
 ratelimiters_mail = NULL;           /* Updated by ratelimit ACL condition */
                    /* Note that ratelimiters_conn persists across resets. */
 
-/* The message variables follow the connection variables. */
+/* Reset message ACL variables */
 
-for (i = 0; i < ACL_MVARS; i++) acl_var[ACL_CVARS + i] = NULL;
+acl_var_m = NULL;
 
 /* The message body variables use malloc store. They may be set if this is
 not the first message in an SMTP session and the previous message caused them
@@ -1144,7 +1141,7 @@ BOOL
 smtp_start_session(void)
 {
 int size = 256;
-int i, ptr;
+int ptr;
 uschar *p, *s, *ss;
 
 /* Default values for certain variables */
@@ -1172,7 +1169,7 @@ tls_advertised = FALSE;
 
 /* Reset ACL connection variables */
 
-for (i = 0; i < ACL_CVARS; i++) acl_var[i] = NULL;
+acl_var_c = NULL;
 
 /* Allow for trailing 0 in the command buffer. */
 
diff --git a/src/src/spool_in.c b/src/src/spool_in.c
index 03379f092..c45914058 100644
--- a/src/src/spool_in.c
+++ b/src/src/spool_in.c
@@ -1,4 +1,4 @@
-/* $Cambridge: exim/src/src/spool_in.c,v 1.15 2006/02/07 11:19:00 ph10 Exp $ */
+/* $Cambridge: exim/src/src/spool_in.c,v 1.16 2006/09/19 11:28:45 ph10 Exp $ */
 
 /*************************************************
 *     Exim - an Internet mail transport agent    *
@@ -236,8 +236,7 @@ uschar *p;
 one exception. DO NOT change the default value of dont_deliver, because it may
 be forced by an external setting. */
 
-for (n = 0; n < ACL_CVARS + ACL_MVARS; n++) acl_var[n] = NULL;
-
+acl_var_c = acl_var_m = NULL;
 authenticated_id = NULL;
 authenticated_sender = NULL;
 allow_unqualified_recipient = FALSE;
@@ -378,48 +377,52 @@ for (;;)
   if (big_buffer[0] != '-') break;
   big_buffer[Ustrlen(big_buffer) - 1] = 0;
 
-  /* For backward compatibility, we recognize "-acl", which was used before the
-  number of ACL variables changed. Its variable number is 0-9 for connection
-  variables, and 10-19 for message variables. */
+  /* For long-term backward compatibility, we recognize "-acl", which was used
+  before the number of ACL variables changed from 10 to 20. This was before the
+  subsequent change to an arbitrary number of named variables. This code is
+  retained so that upgrades from very old versions can still handle old-format
+  spool files. The value given after "-acl" is a number that is 0-9 for
+  connection variables, and 10-19 for message variables. */
 
   if (Ustrncmp(big_buffer, "-acl ", 5) == 0)
     {
     int index, count;
+    uschar name[4];
+    tree_node *node;
+
     if (sscanf(CS big_buffer + 5, "%d %d", &index, &count) != 2)
       goto SPOOL_FORMAT_ERROR;
-    acl_var[index] = store_get(count + 1);
-    if (fread(acl_var[index], 1, count+1, f) < count) goto SPOOL_READ_ERROR;
-    acl_var[index][count] = 0;
-    }
 
-  /* Nowadays we use "-aclc" and "-aclm" for the different types of ACL
-  variable, because Exim may be built with different numbers of them. */
+    (void) string_format(name, 4, "%c%d", (index < 10 ? 'c' : 'm'), index);
+    node = acl_var_create(name);
+    node->data.ptr = store_get(count + 1);
 
-  else if (Ustrncmp(big_buffer, "-aclc ", 6) == 0)
-    {
-    int index, count;
-    if (sscanf(CS big_buffer + 6, "%d %d", &index, &count) != 2)
-      goto SPOOL_FORMAT_ERROR;
-    if (index < ACL_CVARS)
-      {
-      acl_var[index] = store_get(count + 1);
-      if (fread(acl_var[index], 1, count+1, f) < count) goto SPOOL_READ_ERROR;
-      acl_var[index][count] = 0;
-      }
+    if (fread(node->data.ptr, 1, count+1, f) < count) goto SPOOL_READ_ERROR;
+    ((uschar*)node->data.ptr)[count] = 0;
     }
 
-  else if (Ustrncmp(big_buffer, "-aclm ", 6) == 0)
+  /* Nowadays we use "-aclc" and "-aclm" for the different types of ACL
+  variable, because Exim allows any number of them, with arbitrary names.
+  The line in the spool file is "-acl[cm] <name> <length>". The name excludes
+  the c or m. */
+
+  else if (Ustrncmp(big_buffer, "-aclc ", 6) == 0 ||
+           Ustrncmp(big_buffer, "-aclm ", 6) == 0)
     {
-    int index, count;
-    if (sscanf(CS big_buffer + 6, "%d %d", &index, &count) != 2)
-      goto SPOOL_FORMAT_ERROR;
-    if (index < ACL_MVARS)
-      {
-      index += ACL_CVARS;
-      acl_var[index] = store_get(count + 1);
-      if (fread(acl_var[index], 1, count+1, f) < count) goto SPOOL_READ_ERROR;
-      acl_var[index][count] = 0;
-      }
+    uschar *name, *endptr;
+    int count;
+    tree_node *node;
+
+    endptr = Ustrchr(big_buffer + 6, ' ');
+    if (endptr == NULL) goto SPOOL_FORMAT_ERROR;
+    name = string_sprintf("%c%.*s", big_buffer[4], endptr - big_buffer - 6,
+      big_buffer + 6);
+    if (sscanf(CS endptr, " %d", &count) != 1) goto SPOOL_FORMAT_ERROR;
+
+    node = acl_var_create(name);
+    node->data.ptr = store_get(count + 1);
+    if (fread(node->data.ptr, 1, count+1, f) < count) goto SPOOL_READ_ERROR;
+    ((uschar*)node->data.ptr)[count] = 0;
     }
 
   /* Other values */
diff --git a/src/src/spool_out.c b/src/src/spool_out.c
index bbfd785d2..0a17b7a13 100644
--- a/src/src/spool_out.c
+++ b/src/src/spool_out.c
@@ -1,4 +1,4 @@
-/* $Cambridge: exim/src/src/spool_out.c,v 1.10 2006/02/07 11:19:00 ph10 Exp $ */
+/* $Cambridge: exim/src/src/spool_out.c,v 1.11 2006/09/19 11:28:45 ph10 Exp $ */
 
 /*************************************************
 *     Exim - an Internet mail transport agent    *
@@ -190,21 +190,10 @@ if (sender_ident != NULL) fprintf(f, "-ident %s\n", sender_ident);
 if (received_protocol != NULL)
   fprintf(f, "-received_protocol %s\n", received_protocol);
 
-/* Preserve any ACL variables that are set. Because the values may contain
-newlines, we use an explicit length. */
+/* Preserve any ACL variables that are set. */
 
-for (i = 0; i < ACL_CVARS; i++)
-  {
-  if (acl_var[i] != NULL)
-    fprintf(f, "-aclc %d %d\n%s\n", i, Ustrlen(acl_var[i]), acl_var[i]);
-  }
-
-for (i = 0; i < ACL_MVARS; i++)
-  {
-  int j = i + ACL_CVARS;
-  if (acl_var[j] != NULL)
-    fprintf(f, "-aclm %d %d\n%s\n", i, Ustrlen(acl_var[j]), acl_var[j]);
-  }
+tree_walk(acl_var_c, &acl_var_write, f);
+tree_walk(acl_var_m, &acl_var_write, f);
 
 /* Now any other data that needs to be remembered. */
 
diff --git a/src/src/structs.h b/src/src/structs.h
index c0fa960d3..890867d0a 100644
--- a/src/src/structs.h
+++ b/src/src/structs.h
@@ -1,4 +1,4 @@
-/* $Cambridge: exim/src/src/structs.h,v 1.10 2006/02/21 16:24:19 ph10 Exp $ */
+/* $Cambridge: exim/src/src/structs.h,v 1.11 2006/09/19 11:28:45 ph10 Exp $ */
 
 /*************************************************
 *     Exim - an Internet mail transport agent    *
@@ -779,7 +779,7 @@ typedef struct acl_condition_block {
   int type;
   union {
     BOOL negated;
-    int varnumber;
+    uschar *varname;
   } u;
 } acl_condition_block;
 
diff --git a/src/src/tree.c b/src/src/tree.c
index 17216052b..219a53580 100644
--- a/src/src/tree.c
+++ b/src/src/tree.c
@@ -1,4 +1,4 @@
-/* $Cambridge: exim/src/src/tree.c,v 1.3 2006/02/07 11:19:00 ph10 Exp $ */
+/* $Cambridge: exim/src/src/tree.c,v 1.4 2006/09/19 11:28:45 ph10 Exp $ */
 
 /*************************************************
 *     Exim - an Internet mail transport agent    *
@@ -342,4 +342,26 @@ return NULL;
 }
 
 
+
+/*************************************************
+*   Walk tree recursively and execute function   *
+*************************************************/
+
+/*
+Arguments:
+  p       root of the tree
+  f       function to execute for each name-value-pair
+  ctx     context data for f
+*/
+
+void
+tree_walk(tree_node *p, void (*f)(uschar*, uschar*, void*), void *ctx)
+{
+if (p == NULL) return;
+f(p->name, p->data.ptr, ctx);
+if (p->left != NULL) tree_walk(p->left, f, ctx);
+if (p->right != NULL) tree_walk(p->right, f, ctx);
+}
+
+
 /* End of tree.c */
diff --git a/test/aux-var-src/0372.F b/test/aux-var-src/0372.F
index a5c6f6780..c24183786 100644
--- a/test/aux-var-src/0372.F
+++ b/test/aux-var-src/0372.F
@@ -9,7 +9,9 @@ logwrite "  acl_c0=\"$acl_c0\"\n  acl_c1=\"$acl_c1\"\n  acl_c2=\"$acl_c2\"\n  \
             acl_c9=\"$acl_c9\"\n  acl_m0=\"$acl_m0\"\n  acl_m1=\"$acl_m1\"\n  \
             acl_m2=\"$acl_m2\"\n  acl_m3=\"$acl_m3\"\n  acl_m4=\"$acl_m4\"\n  \
             acl_m5=\"$acl_m5\"\n  acl_m6=\"$acl_m6\"\n  acl_m7=\"$acl_m7\"\n  \
-            acl_m8=\"$acl_m8\"\n  acl_m9=\"$acl_m9\""  
+            acl_m8=\"$acl_m8\"\n  acl_m9=\"$acl_m9\"\n  \
+            acl_mfoo=\"$acl_mfoo\"\n  acl_mbar=\"$acl_mbar\"\n  \
+            acl_c_foo=\"$acl_c_foo\"\n  acl_c_bar=\"$acl_c_bar\"\n"  
 logwrite "-------------------------"
 
 # End
diff --git a/test/confs/0372 b/test/confs/0372
index 34fc1aa24..682eb91a8 100644
--- a/test/confs/0372
+++ b/test/confs/0372
@@ -1,5 +1,7 @@
 # Exim test configuration 0372
 
+STRICT=
+
 exim_path = EXIM_PATH
 host_lookup_order = bydns
 primary_hostname = myhost.test.ex
@@ -17,7 +19,7 @@ acl_smtp_connect = a2
 
 system_filter = DIR/aux-var/TESTNUM.F
 trusted_users = CALLER
-
+STRICT
 
 # ----- ACL -----
 
@@ -28,6 +30,8 @@ a1:
          set acl_m0  = value for m0 is local_part: $local_part
          set acl_m5  = value for m5 contains newline\nthere!
          set acl_m9  = value for m9 is domain: $domain
+         set acl_mfoo = value for mfoo is local_part: $local_part
+         set acl_mbar = value for mbar is domain: $domain
 
   accept local_parts = a
          set acl_m0  = value for m0 is local_part: $local_part
@@ -37,6 +41,8 @@ a2:
   accept set acl_c0  = value for c0 is ip: $sender_host_address
          set acl_c5  = c5 has a newline\nthere!
          set acl_c9  = value for c9 is name: $sender_host_name
+         set acl_c_foo = value for c_foo is ip: $sender_host_address
+         set acl_c_bar = value for c_bar is name: $sender_host_name
 
 
 # ----- Routers -----
@@ -51,7 +57,9 @@ r1:
                 acl_c9="$acl_c9"\nacl_m0="$acl_m0"\nacl_m1="$acl_m1"\n\
                 acl_m2="$acl_m2"\nacl_m3="$acl_m3"\nacl_m4="$acl_m4"\n\
                 acl_m5="$acl_m5"\nacl_m6="$acl_m6"\nacl_m7="$acl_m7"\n\
-                acl_m8="$acl_m8"\nacl_m9="$acl_m9"
+                acl_m8="$acl_m8"\nacl_m9="$acl_m9"\n\
+                acl_mfoo="$acl_mfoo"\nacl_mbar="$acl_mbar"\n\
+                acl_c_foo="$acl_c_foo"\nacl_c_bar="$acl_c_bar"
   transport = t1
 
 
@@ -64,5 +72,4 @@ t1:
   file = /dev/null
   user = CALLER
 
-
 # End
diff --git a/test/log/0372 b/test/log/0372
index d4be63b9c..e3c6b8a5c 100644
--- a/test/log/0372
+++ b/test/log/0372
@@ -4,3 +4,4 @@
 1999-03-02 09:44:33 10HmaY-0005vi-00 <= <> H=host.name [1.2.3.4] U=CALLER P=smtp S=sss
 1999-03-02 09:44:33 10HmaY-0005vi-00 => a <a@b> R=r1 T=t1
 1999-03-02 09:44:33 10HmaY-0005vi-00 Completed
+1999-03-02 09:44:33 10HmaZ-0005vi-00 <= <> H=host2.name [4.3.2.1] U=CALLER P=smtp S=sss
diff --git a/test/scripts/0000-Basic/0372 b/test/scripts/0000-Basic/0372
index 4e670167e..f64899e31 100644
--- a/test/scripts/0000-Basic/0372
+++ b/test/scripts/0000-Basic/0372
@@ -10,3 +10,11 @@ data
 .
 quit
 ****
+exim -DSTRICT=strict_acl_vars -bs -oMa 4.3.2.1 -oMs host2.name
+mail from:<>
+rcpt to:<a@b>
+data
+.
+quit
+****
+no_msglog_check
diff --git a/test/stderr/0372 b/test/stderr/0372
index 86eee6d68..b0c01f4a4 100644
--- a/test/stderr/0372
+++ b/test/stderr/0372
@@ -26,6 +26,10 @@ there!"
   acl_m7=""
   acl_m8=""
   acl_m9="value for m9 is domain: y"
+  acl_mfoo="value for mfoo is local_part: x"
+  acl_mbar="value for mbar is domain: y"
+  acl_c_foo="value for c_foo is ip: 1.2.3.4"
+  acl_c_bar="value for c_bar is name: host.name"
 -------------------------
 acl_c0="value for c0 is ip: 1.2.3.4"
 acl_c1=""
@@ -49,6 +53,10 @@ acl_m6=""
 acl_m7=""
 acl_m8=""
 acl_m9="value for m9 is domain: y"
+acl_mfoo="value for mfoo is local_part: x"
+acl_mbar="value for mbar is domain: y"
+acl_c_foo="value for c_foo is ip: 1.2.3.4"
+acl_c_bar="value for c_bar is name: host.name"
 LOG: MAIN
   => x <x@y> R=r1 T=t1
 LOG: MAIN
@@ -78,6 +86,10 @@ there!"
   acl_m7=""
   acl_m8=""
   acl_m9="value for m9 is domain: b"
+  acl_mfoo=""
+  acl_mbar=""
+  acl_c_foo="value for c_foo is ip: 1.2.3.4"
+  acl_c_bar="value for c_bar is name: host.name"
 -------------------------
 acl_c0="value for c0 is ip: 1.2.3.4"
 acl_c1=""
@@ -100,9 +112,15 @@ acl_m6=""
 acl_m7=""
 acl_m8=""
 acl_m9="value for m9 is domain: b"
+acl_mfoo=""
+acl_mbar=""
+acl_c_foo="value for c_foo is ip: 1.2.3.4"
+acl_c_bar="value for c_bar is name: host.name"
 LOG: MAIN
   => a <a@b> R=r1 T=t1
 LOG: MAIN
   Completed
 LOG: smtp_connection MAIN
   SMTP connection from CALLER closed by QUIT
+----- System filter -----
+1999-03-02 09:44:33 10HmaZ-0005vi-00 Error in system filter: failed to expand "  acl_c0="$acl_c0"\n  acl_c1="$acl_c1"\n  acl_c2="$acl_c2"\n  acl_c3="$acl_c3"\n  acl_c4="$acl_c4"\n  acl_c5="$acl_c5"\n  acl_c6="$acl_c6"\n  acl_c7="$acl_c7"\n  acl_c8="$acl_c8"\n  acl_c9="$acl_c9"\n  acl_m0="$acl_m0"\n  acl_m1="$acl_m1"\n  acl_m2="$acl_m2"\n  acl_m3="$acl_m3"\n  acl_m4="$acl_m4"\n  acl_m5="$acl_m5"\n  acl_m6="$acl_m6"\n  acl_m7="$acl_m7"\n  acl_m8="$acl_m8"\n  acl_m9="$acl_m9"\n  acl_mfoo="$acl_mfoo"\n  acl_mbar="$acl_mbar"\n  acl_c_foo="$acl_c_foo"\n  acl_c_bar="$acl_c_bar"\n" in logwrite command: unknown variable name "acl_c1" (strict_acl_vars is set)
diff --git a/test/stdout/0372 b/test/stdout/0372
index 4d1ea2bd8..b4331b65c 100644
--- a/test/stdout/0372
+++ b/test/stdout/0372
@@ -8,3 +8,9 @@
 354 Enter message, ending with "." on a line by itself

 250 OK id=10HmaY-0005vi-00

 221 myhost.test.ex closing connection

+220 myhost.test.ex ESMTP Exim x.yz Tue, 2 Mar 1999 09:44:33 +0000

+250 OK

+250 Accepted

+354 Enter message, ending with "." on a line by itself

+250 OK id=10HmaZ-0005vi-00

+221 myhost.test.ex closing connection