I ran into an interesting ISA Firewall performance issue that I thought would be worth sharing, since it highlights one of the most important reasons for poor ISA Firewall performance — DNS problems.

First, some background. A couple of weeks ago a lightening storm must have fried some portion of the RAM in my ISA Firewall, since after the storm every time I tried to save a change to the ISA firewall configuration or try to RDP into the ISA Firewall, the machine would reboot. Not good.

However, around the same time, I changed ISPs.

After the ISA firewall RAM got partially fired, and at the same time changing ISPs, it seemed like Web proxy and firewall client performance was significant lower. It look about five to six seconds to bring up a Web page. This is on a FiOS 15/15 connection, so there’s no way that it should have taken that long to bring up a Web page.

My first thought was that maybe it was related to the partially fried ISA firewall, so I replaced it with a new one. I backed up the configuration using the Export command on the old ISA firewall. I then setup a new ISA Firewall with the same name and same configuration as the old one. Deleted the old machine’s account in the AD, took the old ISA firewall offline, and then connected the new ISA Firewall to the network and to the domain (I always join the ISA Firewall to the domain so as to get the highest level of security).

I then imported the old ISA firewall’s configuration into the new ISA firewall and everything worked fine. I could RDP into the machine and make changes to the ISA firewall configuration without having the machine reboot. I was confident that the performance issues would go away.

I was wrong. It still took 5-6 seconds to bring up a Web page. This was getting quite frustrating as I’m paying for a very fast fiberoptic connection and I wasn’t getting fibre level speeds.

Then I thought about the delay, which was about 5 seconds. It was a consistent delay. What configuration in my infrastructure could be related to 5 seconds? Then it occurred to me — a DNS timeout.

I have two DNS servers on my network that I use for external and internal name resolution. For external name resolution, I configure my DNS servers to use my ISP’s DNS servers as forwarders. The configuration on the DNS server is to timeout the query after 5 seconds and move to the next DNS server on the list of forwarders.

Verizon gave me the IP addresses:

68.238.96.12 Primary DNS

68.238.112.12 Secondary DNS

So I entered those DNS server addresses in that order for my forwarders.

I decided to check if these DNS servers were actually online. I used the nslookup command and then the server 68.238.96.12 command to set the DNS server to that address. The DNS queries failed when I used that server. I then pinged that server and found that it didn’t respond to pings.

I then set the server 68.238.112.12 in nslookup. I did some DNS queries to that DNS server and found that it was answering queries quickly. I also pinged that server and found that it was responding in about 30ms.

I used Google to check for Verizon DNS servers and found another one at the address 68.238.64.12 and decided that I would use that one as my secondary and the 68.238.112.12 as my primary. After making those changes on my DNS servers so that those two DNS servers were my forwarders, my Internet connection “popped”! Web pages came up almost instantly and there was less than a second wait time on bringing up Web pages.

It would be nice if I could inform Verizon about their downed DNS server, but the company is so large that you have to spend well over an hour to connect to the right person. I hope that someone at Verizon will someday figure out that their primary DNS server for the Dallas/Ft.Worth area is down, but who knows how long that will take? As for the present time, things are working great.

Moral of the story is that if you’re finding that your ISA firewall performance is slower than it should be, check your DNS configuration. If your DNS configuration is correct, then check all your DNS server and the forwarders that your DNS servers use. Or, don’t use forwarders at all and allow your DNS servers to perform recursion themselves. That’s usually the best option if your ISP isn’t very good at maintaining their DNS servers and the cache on the ISP’s DNS server isn’t much larger than yours (typically the case when you’re on a large corporate network).

HTH,

Tom

Thomas W Shinder, M.D.
Site: http://www.isaserver.org/
Blog: http://blogs.isaserver.org/shinder/
GET THE NEW BOOK! Go to http://tinyurl.com/2gpoo8
Email: tshinder@isaserver.org
MVP — Microsoft Firewalls (ISA)