Solving the "Directly access these servers or domains" issue in ISA Server 2004 SP2
ISA Server 2004 SP2 makes some changes to the way that destinations specified for direct access are handled. This is documented in KB903746 Changes that are made to the Cache Array Routing Protocol (CARP) in ISA Server 2004 Service Pack 2, particular in the section Clients in a branch office cannot route requests for internal destinations through the local ISA Server computer. These changes have been made to facilitate some typical branch office scenarios. However, by doing that the ISA team broke the way people did use the “Directly access these servers or domains” list to solve some common web proxy and authentication problems due to badly written web client software.
Back in Februari 2006, I logged a case with Microsoft PSS to fix that problem. Very soon I got a private fix that solves the problem without losing the new branch office features introduced by ISA 2004 SP2. However, that private fix was broken with KB916106 Update for HTTP issues in Internet Security and Acceleration Server 2004 Service Pack 2 because the private fix didn’t make it through the usual Microsoft approval process at that time. On July 17, 2006 Microsoft finally released the Knowledge Base article KB920715 Web Proxy clients do not directly access a Web site that you enter in the “Directly access these servers or domains” list in ISA Server 2004 SP2 with a reference to a hotfix described in the article KB920716 Description of the ISA Server 2004 hotfix package: June 6, 2006 to solve that problem.
I have requested that hotfix, installed and tested it in my lab and it effectively solved the “Directly access these servers or domains” issue mentioned above. The key is that in order to get the pre-SP2 behavior, you should specify the destinations as a URL instead of as an FQDN. The generic format for a URL is <Protocol>://<FQDN>:<Port>/<Path>. Therefore, as explained in the article KB920715, you should specify *.domain.tld/* for a whole domain. For a single server the syntax */host.domain.tld/* should do the trick.
For those interested in the bits and bytes, here is what has changed so far in ISA 2004 concerning direct access. It might be useful to read first the article Proxy Client Autoconfig File Format in order to better understand the predefined functions used in the wpad.dat file or the routing script.
If you carefully look at the content of the wpad.dat file or the routing script, you will see three major functions used in the decision process for direct access:
- function MakeIPs(): defines the IP ranges which possible should not be redirected to the web proxy server.
- function MakeNames(): defines the FQDN’s which possible should not be redirected to the web proxy server.
- function FindProxyForURL(url, host): contains the script that IE uses to determine if a destination should be redirected to the web proxy server or not.
With pre-SP2, if direct names and direct IPs are configured in the exception list, we first check if the requested site belongs to the direct names list and if YES then the script returns a DIRECT route immediately. If the requested site was an IP address, then the direct IPs list was used instead. Take note that there was no dependency whatsoever between the direct names and IPs lists. Here is an extract of the function FindProxyForURL(url, host) obtained with SP1:
It was common practice to specify standard the internal domain as *.domain.tld, the internal network ID and the network ID 127/8 in the “Directly access these servers or domains” list (http://www.isaserver.org/IsaNews/copylattowebproxy.html). In that way, any destination that must be considered local was accessed directly, regardless if the request was made by FQDN or IP address. Of course, some other sites such as the Hotmail (*.hotmail.com), the MSN (*.msn.com) and the Passport (*.passport.net and *.passport.com) domains are very popular members of the list too.
With SP2, the above behavior has changed considerable. If direct names and direct IPs are used, we still check if the requested site belongs to the direct names list. If YES we are getting a DIRECT route immediately ONLY if there is NO direct IPs configured. If there are direct IPs, then we try to resolve the requested site name to an IP. Then we are returning a DIRECT route ONLY if this IP belongs to the direct IPs list. Take note that the behavior was not changed if the requested site was an IP address. Here is a code snippet of the function FindProxyForURL(url, host) obtained with SP2:
The problem with the SP2 behavior is that it is impossible for an ISA administrator to enumerate all possible IP addresses belonging to all domains he has configured in the “Directly access these servers or domains” list. Therefore, there was nothing else we could do but removing all IP addresses from the list. Moreover, if we have to add selected IP addresses such as 127.0.0.1 to the list we have to do it as FQDNs, not as IPs. Obviously this workaround can’t be considered a long term solution.
Now with the KB920716 hotfix, there is a clever way to regain the pre-SP2 behavior without losing the new branch office features introduced by ISA 2004 SP2, although you should specify the destinations as a URL instead of as an FQDN (e.g. *.hotmail.com/* instead of *.hotmail.com). In my opinion, that’s a small price to pay. Here is a code snippet of the function FindProxyForURL(url, host) obtained with SP2 + KB920716:
The fix is the new and second if statement if(shExpMatch(url, DirectNames[i])) following the old and first one if(shExpMatch(host, DirectNames[i])). If you specify a destination as a URL then the first if statement will not catch the new format. In other words, we will never have a match for a URL on the first if statement and therefore the second one will evaluate the entry instead. If a match is found on the second if statement then the script returns a DIRECT route immediately, just as it was before SP2. As a result, we can apply again our common practice to specify standard the internal domain as *.domain.tld and the internal network ID in the “Directly access these servers or domains” list. Take note that from SP2 onwards, if IP addresses are added to the list than the address range of 127.0.0.0-127.255.255.255 (127/8) is automatically added to the list.
HTH,
Stefaan
