Additions to dnsdb lookups: (a) list of domains (b) change output

separator.

4 files changed, 204 insertions, 99 deletions

diff --git a/doc/doc-misc/WishList b/doc/doc-misc/WishList
index d3ce50b98..aa9fb7b2b 100644
--- a/doc/doc-misc/WishList
+++ b/doc/doc-misc/WishList
@@ -1,4 +1,4 @@
-$Cambridge: exim/doc/doc-misc/WishList,v 1.6 2004/11/09 09:32:58 ph10 Exp $
+$Cambridge: exim/doc/doc-misc/WishList,v 1.7 2004/11/19 15:18:57 ph10 Exp $
 
 EXIM 4 WISH LIST
 ----------------
@@ -1413,13 +1413,6 @@ be able to set variables in routers like in acl's." This is effectively a
 radical suggestion for a complete re-design, and is therefore BIG.
 ------------------------------------------------------------------------------
 
-(222) 19-Dec-03 S  Iterative option for dnsdb
-
-A way of getting a dnsdb lookup to chop off components until something is
-found: e.g. ${lookup dndsb-i{ns=a.b.c.d}} would look for nameservers for
-a.b.c.d, then b.c.d, etc.
-------------------------------------------------------------------------------
-
 (223) 22-Dec-03 S  Support SOA lookup in dnsdb lookups
 ------------------------------------------------------------------------------
 
diff --git a/doc/doc-txt/ChangeLog b/doc/doc-txt/ChangeLog
index 5e213d772..1def41364 100644
--- a/doc/doc-txt/ChangeLog
+++ b/doc/doc-txt/ChangeLog
@@ -1,4 +1,4 @@
-$Cambridge: exim/doc/doc-txt/ChangeLog,v 1.34 2004/11/19 09:45:54 ph10 Exp $
+$Cambridge: exim/doc/doc-txt/ChangeLog,v 1.35 2004/11/19 15:18:57 ph10 Exp $
 
 Change log file for Exim from version 4.21
 -------------------------------------------
@@ -149,8 +149,16 @@ Exim version 4.44
     per-process caching. But the chance of anyone hitting this buglet seems
     very small.
 
-37. The dnsdb lookup has a new type, "zns", which walks up the domain tree
-    until it finds some nameserver records. It should be used with care.
+37. The dnsdb lookup has been extended in a number of ways.
+
+    (1) There is a new type, "zns", which walks up the domain tree until it
+        finds some nameserver records. It should be used with care.
+
+    (2) It is now possible to give a list of domains (or IP addresses) to be
+        looked up.
+
+    (3) It is now possible to specify the separator character for use when
+        multiple records are returned.
 
 
 Exim version 4.43
diff --git a/doc/doc-txt/NewStuff b/doc/doc-txt/NewStuff
index ee9f55c31..810607a5b 100644
--- a/doc/doc-txt/NewStuff
+++ b/doc/doc-txt/NewStuff
@@ -1,4 +1,4 @@
-$Cambridge: exim/doc/doc-txt/NewStuff,v 1.12 2004/11/19 09:45:54 ph10 Exp $
+$Cambridge: exim/doc/doc-txt/NewStuff,v 1.13 2004/11/19 15:18:57 ph10 Exp $
 
 New Features in Exim
 --------------------
@@ -129,6 +129,43 @@ Version 4.44
     the name servers for the high-level domains such as .com or .co.uk are not
     going to be on such a list.
 
+13. It is now possible to specify a list of domains or IP addresses to be
+    looked up in a dnsdb lookup. The list is specified in the normal Exim way,
+    with colon as the default separator, but with the ability to change this.
+    For example:
+
+      ${lookup dnsdb{one.domain.com:two.domain.com}}
+      ${lookup dnsdb{a=one.host.com:two.host.com}}
+      ${lookup dnsdb{ptr = <; 1.2.3.4 ; 4.5.6.8}}
+
+    In order to retain backwards compatibility, there is one special case: if
+    the lookup type is PTR and no change of separator is specified, Exim looks
+    to see if the rest of the string is precisely one IPv6 address. In this
+    case, it does not treat it as a list.
+
+    The data from each lookup is concatenated, with newline separators (by
+    default - see 14 below), in the same way that multiple DNS records for a
+    single item are handled.
+
+    The lookup fails only if all the DNS lookups fail. As long as at least one
+    of them yields some data, the lookup succeeds. However, if there is a
+    temporary DNS error for any of them, the lookup defers.
+
+14. It is now possible to specify the character to be used as a separator when
+    a dnsdb lookup returns data from more than one DNS record. The default is a
+    newline. To specify a different character, put '>' followed by the new
+    character at the start of the query. For example:
+
+      ${lookup dnsdb{>: a=h1.test.ex:h2.test.ex}}
+      ${lookup dnsdb{>| mx=<;m1.test.ex;m2.test.ex}}
+
+    It is permitted to specify a space as the separator character. Note that
+    more than one DNS record can be found for a single lookup item; this
+    feature is relevant even when you do not specify a list.
+
+    The same effect could be achieved by wrapping the lookup in ${tr...}; this
+    feature is just a syntactic simplification.
+
 
 Version 4.43
 ------------
diff --git a/src/src/lookups/dnsdb.c b/src/src/lookups/dnsdb.c
index 492e53574..22a9de4f4 100644
--- a/src/src/lookups/dnsdb.c
+++ b/src/src/lookups/dnsdb.c
@@ -1,4 +1,4 @@
-/* $Cambridge: exim/src/src/lookups/dnsdb.c,v 1.2 2004/11/19 09:45:54 ph10 Exp $ */
+/* $Cambridge: exim/src/src/lookups/dnsdb.c,v 1.3 2004/11/19 15:18:57 ph10 Exp $ */
 
 /*************************************************
 *     Exim - an Internet mail transport agent    *
@@ -77,7 +77,20 @@ return (void *)(-1);   /* Any non-0 value */
 *           Find entry point for dnsdb           *
 *************************************************/
 
-/* See local README for interface description. */
+/* See local README for interface description. The query in the "keystring" may
+consist of a number of parts.
+
+(a) If the first significant character is '>' then the next character is the 
+separator character that is used when multiple records are found. The default 
+separator is newline.
+
+(b) If the next sequence of characters is a sequence of letters and digits 
+followed by '=', it is interpreted as the name of the DNS record type. The 
+default is "A".
+
+(c) Then there follows list of domain names. This is a generalized Exim list, 
+which may start with '<' in order to set a specific separator. The default 
+separator, as always, is colon. */
 
 int
 dnsdb_find(void *handle, uschar *filename, uschar *keystring, int length,
@@ -86,9 +99,10 @@ dnsdb_find(void *handle, uschar *filename, uschar *keystring, int length,
 int rc;
 int size = 256;
 int ptr = 0;
+int sep = 0;
 int type = T_TXT;
-uschar *orig_keystring = keystring;
-uschar *equals = Ustrchr(keystring, '=');
+uschar *outsep = US"\n";
+uschar *equals, *domain;
 uschar buffer[256];
 
 /* Because we're the working in the search pool, we try to reclaim as much
@@ -105,12 +119,26 @@ filename = filename;
 length = length;
 do_cache = do_cache;
 
-/* If the keystring contains an = this is preceded by a type name. */
+/* If the string starts with '>' we change the output separator */
 
-if (equals != NULL)
+while (isspace(*keystring)) keystring++;
+if (*keystring == '>')
   {
-  int i;
-  int len = equals - keystring;
+  outsep = keystring + 1;
+  keystring += 2; 
+  while (isspace(*keystring)) keystring++;
+  } 
+
+/* If the keystring contains an = this must be preceded by a valid type name. */
+
+if ((equals = Ustrchr(keystring, '=')) != NULL)
+  {
+  int i, len;
+  uschar *tend = equals;
+   
+  while (tend > keystring && isspace(tend[-1])) tend--; 
+  len = tend - keystring; 
+ 
   for (i = 0; i < sizeof(type_names)/sizeof(uschar *); i++)
     {
     if (len == Ustrlen(type_names[i]) &&
@@ -120,111 +148,150 @@ if (equals != NULL)
       break;
       }
     }
+     
   if (i >= sizeof(type_names)/sizeof(uschar *))
     {
     *errmsg = US"unsupported DNS record type";
     return DEFER;
     }
-  keystring += len + 1;
+     
+  keystring = equals + 1;
+  while (isspace(*keystring)) keystring++;
   }
-
-/* If the type is PTR, we have to construct the relevant magic lookup
-key. This code is now in a separate function. */
-
-if (type == T_PTR)
-  {
-  dns_build_reverse(keystring, buffer);
-  keystring = buffer;
-  }
-
-DEBUG(D_lookup) debug_printf("dnsdb key: %s\n", keystring);
-
-/* Initialize the resolver, in case this is the first time it is used
-in this run. Then do the lookup and sort out the result. */
+  
+/* Initialize the resolver in case this is the first time it has been used. */
 
 dns_init(FALSE, FALSE);
-rc = dns_special_lookup(&dnsa, keystring, type, NULL);
 
-if (rc == DNS_NOMATCH || rc == DNS_NODATA) return FAIL;
-if (rc != DNS_SUCCEED) return DEFER;
+/* The remainder of the string must be a list of domains. As long as the lookup 
+for at least one of them succeeds, we return success. Failure means that none 
+of them were found. 
 
-/* If the lookup was a pseudo-type, change it to the correct type for searching 
-the returned records; then search for them. */
+The original implementation did not support a list of domains. Adding the list 
+feature is compatible, except in one case: when PTR records are being looked up 
+for a single IPv6 address. Fortunately, we can hack in a compatibility feature 
+here: If the type is PTR and no list separator is specified, and the entire 
+remaining string is valid as an IP address, set an impossible separator so that 
+it is treated as one item. */
 
-if (type == T_ZNS) type = T_NS;
-for (rr = dns_next_rr(&dnsa, &dnss, RESET_ANSWERS);
-     rr != NULL;
-     rr = dns_next_rr(&dnsa, &dnss, RESET_NEXT))
-  {
-  if (rr->type != type) continue;
+if (type == T_PTR && keystring[0] != '<' &&
+    string_is_ip_address(keystring, NULL) > 0) 
+  sep = -1;
 
-  /* There may be several addresses from an A6 record. Put newlines between
-  them, just as for between several records. */
+/* Now scan the list and do a lookup for each item */
 
-  if (type == T_A ||
-      #ifdef SUPPORT_A6
-      type == T_A6 ||
-      #endif
-      type == T_AAAA)
-    {
-    dns_address *da;
-    for (da = dns_address_from_rr(&dnsa, rr); da != NULL; da = da->next)
-      {
-      if (ptr != 0) yield = string_cat(yield, &size, &ptr, US"\n", 1);
-      yield = string_cat(yield, &size, &ptr, da->address, Ustrlen(da->address));
-      }
-    continue;
-    }
-
-  /* Other kinds of record just have one piece of data each. */
+while ((domain = string_nextinlist(&keystring, &sep, buffer, sizeof(buffer))) 
+        != NULL)
+  {       
+  uschar rbuffer[256];
 
-  if (ptr != 0) yield = string_cat(yield, &size, &ptr, US"\n", 1);
-
-  if (type == T_TXT)
+  /* If the type is PTR, we have to construct the relevant magic lookup
+  key. This code is now in a separate function. */
+  
+  if (type == T_PTR)
     {
-    yield = string_cat(yield, &size, &ptr, (uschar *)(rr->data+1),
-      (rr->data)[0]);
+    dns_build_reverse(domain, rbuffer);
+    domain = rbuffer;
     }
-  else   /* T_CNAME, T_MX, T_NS, T_PTR */
+  
+  DEBUG(D_lookup) debug_printf("dnsdb key: %s\n", domain);
+  
+  /* Do the lookup and sort out the result. We use the special 
+  lookup function that knows about pseudo types like "zns". If the lookup 
+  fails, continue with the next domain. */
+  
+  rc = dns_special_lookup(&dnsa, domain, type, NULL);
+  
+  if (rc == DNS_NOMATCH || rc == DNS_NODATA) continue;
+  if (rc != DNS_SUCCEED) return DEFER;
+  
+  /* If the lookup was a pseudo-type, change it to the correct type for
+  searching the returned records; then search for them. */
+  
+  if (type == T_ZNS) type = T_NS;
+  for (rr = dns_next_rr(&dnsa, &dnss, RESET_ANSWERS);
+       rr != NULL;
+       rr = dns_next_rr(&dnsa, &dnss, RESET_NEXT))
     {
-    uschar s[264];
-    uschar *p = (uschar *)(rr->data);
-    if (type == T_MX)
+    if (rr->type != type) continue;
+  
+    /* There may be several addresses from an A6 record. Put the configured 
+    separator between them, just as for between several records. However, A6 
+    support is not normally configured these days. */
+  
+    if (type == T_A ||
+        #ifdef SUPPORT_A6
+        type == T_A6 ||
+        #endif
+        type == T_AAAA)
       {
-      int num;
-      GETSHORT(num, p);            /* pointer is advanced */
-      sprintf(CS s, "%d ", num);
-      yield = string_cat(yield, &size, &ptr, s, Ustrlen(s));
+      dns_address *da;
+      for (da = dns_address_from_rr(&dnsa, rr); da != NULL; da = da->next)
+        {
+        if (ptr != 0) yield = string_cat(yield, &size, &ptr, outsep, 1);
+        yield = string_cat(yield, &size, &ptr, da->address, 
+          Ustrlen(da->address));
+        }
+      continue;
       }
-    else if (type == T_SRV)
+  
+    /* Other kinds of record just have one piece of data each, but there may be 
+    several of them, of course. */
+  
+    if (ptr != 0) yield = string_cat(yield, &size, &ptr, outsep, 1);
+  
+    if (type == T_TXT)
       {
-      int num, weight, port;
-      GETSHORT(num, p);            /* pointer is advanced */
-      GETSHORT(weight, p);
-      GETSHORT(port, p);
-      sprintf(CS s, "%d %d %d ", num, weight, port);
-      yield = string_cat(yield, &size, &ptr, s, Ustrlen(s));
+      yield = string_cat(yield, &size, &ptr, (uschar *)(rr->data+1),
+        (rr->data)[0]);
       }
-    rc = dn_expand(dnsa.answer, dnsa.answer + dnsa.answerlen, p,
-      (DN_EXPAND_ARG4_TYPE)(s), sizeof(s));
-
-    /* If an overlong response was received, the data will have been
-    truncated and dn_expand may fail. */
-
-    if (rc < 0)
+    else   /* T_CNAME, T_MX, T_NS, T_SRV, T_PTR */
       {
-      log_write(0, LOG_MAIN, "host name alias list truncated for %s",
-        orig_keystring);
-      break;
+      uschar s[264];
+      uschar *p = (uschar *)(rr->data);
+      if (type == T_MX)
+        {
+        int num;
+        GETSHORT(num, p);            /* pointer is advanced */
+        sprintf(CS s, "%d ", num);
+        yield = string_cat(yield, &size, &ptr, s, Ustrlen(s));
+        }
+      else if (type == T_SRV)
+        {
+        int num, weight, port;
+        GETSHORT(num, p);            /* pointer is advanced */
+        GETSHORT(weight, p);
+        GETSHORT(port, p);
+        sprintf(CS s, "%d %d %d ", num, weight, port);
+        yield = string_cat(yield, &size, &ptr, s, Ustrlen(s));
+        }
+      rc = dn_expand(dnsa.answer, dnsa.answer + dnsa.answerlen, p,
+        (DN_EXPAND_ARG4_TYPE)(s), sizeof(s));
+  
+      /* If an overlong response was received, the data will have been
+      truncated and dn_expand may fail. */
+  
+      if (rc < 0)
+        {
+        log_write(0, LOG_MAIN, "host name alias list truncated: type=%s "
+          "domain=%s", dns_text_type(type), domain);
+        break;
+        }
+      else yield = string_cat(yield, &size, &ptr, s, Ustrlen(s));
       }
-    else yield = string_cat(yield, &size, &ptr, s, Ustrlen(s));
-    }
-  }
+    }    /* Loop for list of returned records */
+  }      /* Loop for list of domains */
+
+/* Reclaim unused memory */
 
+store_reset(yield + ptr + 1);
+
+/* If ptr == 0 we have not found anything. Otherwise, insert the terminating 
+zero and return the result. */
+
+if (ptr == 0) return FAIL;
 yield[ptr] = 0;
-store_reset(yield + ptr + 1);    /* Reclaim unused */
 *result = yield;
-
 return OK;
 }