When you have to support the Secure FTP protocol (aka FTPS or FTP over SSL/TLS) with ISA Server 2000 you have to take some tough decisions, especially if you have to allow Explicit Security. In that case, the Secure FTP protocol uses the default FTP control connection (TCP port 21). Because the FTP Application Filter can’t and will never understand the Secure FTP protocol, and this is by design, you have to unbind the FTP Application Filter from the FTP protocol in order to support Secure FTP in Explicit Security mode. Of course this breaks the normal FTP support. For more information, check out my FTP article How the FTP protocol Challenges Firewall Security, section ‘5. What about Secure FTP’.  

With ISA Server 2004 and 2006 we can solve that Secure FTP dilemma by applying what is explained in the recent ISA Server Product Team Blog article Why do I need a deny rule to make an allow rule for a custom protocol work correctly?. As with HTTP, the binding of the FTP Application Filter to the FTP protocol is a global setting. Moreover, the default FTP protocol definition only specifies the FTP Control connection (primary connection) because the FTP Application Filter handles the Data connections (secondary connections). In other words, unbinding the FTP Application Filter will only allow the FTP Control connection but not the Data connections. So, in order to solve the Secure FTP dilemma we will have to do some more work.

First of all we have to create a custom protocol definition for the Secure FTP protocol (FTPS) as shown in the figure below:

Some important characteristics of the FTPS protocol definition are:

• The FTPS Control connection (Primary Connections) uses TCP port 21 for the Explicit Security mode and TCP port 990 for the Implicit Security mode.
• The FTPS Data connection (Secondary Connections) should be defined as Outbound because only FTPS passive mode can work with a NAT relationship. It is also recommended that the Port Range is specified as exactly as possible. If you don’t know on which ports the Secure FTP server will listen for the data connection, you can specify all unprivileged ports > 1023 (1024 - 65534).
• With the above protocol definition, only Firewall clients will be able to connect to the Secure FTP server due to the secondary connections. If you have to support SecureNAT clients too, you need to adjust the above protocol definition by moving the FTPS Data connection from the Secondary Connections to the Primary Connections section. However, be aware of the security risk associated with specifying such a large port range in the  Primary Connections section.

Next, to allow the FTPS traffic, you need to create two access rules:

• An access rule that uses the custom FTPS protocol and allows traffic from the source network to the computer objects representing the Secure FTP servers.
• An access rule that uses the predefined FTP protocol and denies traffic from the source network to the computer objects representing the Secure FTP servers.

Finally, the new allow rule must come before your original rule that allows the normal FTP traffic from the same source network in the ordered list of policy rules, and the new deny rule should be placed immediately after the new allow rule as shown in the figure above.

HTH,
Stefaan