How To Test An IPS: Radware
How To Test An IPS: Radware
Renaud Bidou
RADWARE
[email protected]
Abstract.
Intrusion Prevention Systems are new technology-based products that intend to
detect and block attacks before they reach the target network. As they are to be
deployed inline into the internal network they must be tested. However the lack of
standard methodology often lower the accuracy of tests, and several issues may be
raised once the device is put in production.
Therefore, this document intends to provide methodology and publicly available
tools that will help security experts to test IPS in the right context.
2 Renaud Bidou
Pre-requisites
Before performing tests, it is necessary to understand the purpose of IPS and their
mission into the global security of the infrastructure.
Understanding IPS
Mission of an IPS
An IPS is not an IDS. IPS are designed to block attacks and make sure that the part
of the infrastructure they protect remain safe. Therefore reporting is still necessary but
report’s accuracy may be less important than for IDS, as long as blocking is justified.
As an example a tool like [snot] is designed to flood IDS with packets containing
attack signatures. Those packets are out of any TCP session and will either be blocked
by the stateful of signature engine of the IPS, depending on its setting and design.
Then, whatever is reported, the IPS blocked illegitimate packets.
As a full part of its mission, an IPS must be capable to help enforcing specific
security policy aspects, such as forbidding root loggings on insecure and unencrypted
channels. However, this must not be set as a default behavior. Every organization has
a different security policy and security devices must help enforcing it. They are not
meant to define it! It is then necessary that the IPS is flexible enough to match
specific cases of the security policy.
reduce the capacity of the IPS to accomplish its missions, usually turning it into an
IDS.
There are some rules to be followed in order to properly test an IPS, with the
objective to evaluate its capacity to efficiently protect the network from attacks.
Common pitfalls
The most common pitfall in IPS testing is to use IDS testing methodology and
tools. IPS are new, and as there is a lot of misunderstanding in their very mission
most methodology in testing IPS are just old IDS testing ones moved to IPS. Such
misconception of tests usually allows testing of IPS in “non-blocking” mode. This is
definitely non-sens. Reporting was critical for IDS as it was supposed to provide
information about possible compromise of a resource and lead any manual reaction.
Mission of an IPS is to block, and this is this capability that must be tested first.
Another usual trick is either testing out of the targeted context and / or not testing
attacks that will have an impact on the context. IPS is designed to block real attacks,
not theoretical ones. It is then mandatory to make sure that attacks will have an
impact on the targeted system before the IPS is put in-line. This is particularly true for
DoS attacks and worms propagation. For intrusion attempts it is useless to test too old
attacks, such as the classical phf vulnerability [ CVE-1999-0067] which appeared in
1996. Also the use of vulnerability scanners such as [nessus] is not appropriate, as
they do not launch real attacks but probes to evaluate the potential exposure to some
vulnerabilities.
Last, there is only one good configuration for a given context. Once the
configuration has been set up it should not change during the test. It is common sense,
configuration will not be changed on the production environment according to the
attack we expect to see next.
How to test an IPS 5
Scans mitigation
Scans are usually the first operation performed before an intrusion. Therefore they can
be considered as a threat in some conditions.
Types of scans
Two main types of scans are to be distinguished: identification scans and
vulnerability assessment scans.
The first kind of scans is done in order to provide information about the nature of
the target. Information gathered are the list of hosts on the network, their operating
system and applications they are running. They are usually the first operation
performed when an intruder attempts to get into the network.
The second type of scan, the vulnerability assessment, is aimed at providing
target’s exposure to known attacks. Some tools perform large scans over thousands of
potential vulnerabilities. Another type of vulnerability assessment tools only tests for
one specific vulnerability. These tools are usually developed to quickly evaluate
exposure to a new and critical threat.
such, will definitely need to be coupled with an external log analysis and correlation
tool.
Identification scans
To detect hosts three techniques are available.
1. Layer 2 identification (only on local network), performed by ARP requests. It
can be tested with a tool such as [thcrut];
2. Layer 3 identification, by sending ICMP ECHO REQUEST on all possible IP
addresses, with a tool such as [fping].
3. Layer 4 identification, which relies on TCP session establishment and can be
performed by any port scanner such as [nmap].
Then identifying running application is usually done by a port scanner. Many
techniques can be used. Most of them are already implemented in [nmap]. The most
interesting point is to test for different scan parameters, as there is always a way to
fool an IPS. Most common options to be tested with [nmap] are :
• -P0 : disable the nmap ping which can easily be detected. It is not necessary as
hosts running have been detected in the preceding stage;
• --scan_delay <delay> : tell the scanner to wait a specific amount of milliseconds
between each port to be scanned. This operation will usually bypass detection
based on threshold.
Usually, scans will be detected but rarely blocked, as explained before.
Qualification scans
The first kind of qualification scan is aimed at detecting OS. Three main
techniques are used to achieve this operation. The first one is based on TCP
exceptions and initial values (such as the TCP window size), and is implemented in
[nmap]. The second one relies on handling of specific ICMP packets, and can be
performed with [xprobe]. Last, the analysis of the retransmission delay of SYN/ACK
packets can reveal the OS of the server. This technique is made available as a patch
for nmap [nmap-cronos].
Then applications versions can be identified in two main ways:
1. Grabbing banners and testing commands, which is a basic technique but still
quite efficient and implemented in [amap];
2. Testing for contents of answers to specific requests, as provided by [httprint].
How to test an IPS 7
Testing evasion
There are numerous evasion techniques, organized in four main categories:
• Fragmentation: splitting the attack in different packets
• Substitution: formatting part of the attack in such a way that the server
understands it but may confuse the IPS
• Insertion: inserting fake or non standard packets may also confuse an IPS while
the server will simply drop them
• Confusion: the use of encryption, especially with SSL, and tunneling
technologies will bypass most IPS which are not able to handle such traffic.
Fragmentation
Fragmenting attacks can be performed at two levels. The first one is at layer 3, by
splitting one packet in smaller ones. The second one is at layer 4 and is performed by
splitting a command or a data in several packets. On top of those basic techniques it is
possible to confuse IPS by adding out of session packets or messing established
connections. Such techniques are implemented in [fragrouter] which behaves like a
router and performs most fragmentation techniques discussed above.
However, it is mandatory to make sure that exploits sent through the fragrouter are
effective against the target. Specificities of IP stacks may prevent some attacks from
working properly and there will be no use for an IPS to block them.
Substitution
Substitution techniques are mainly used to have HTTP attacks bypass prevention
systems. They use several encoding and formatting of malicious URLs to confuse
IPS. An interesting point is that some systems are able to handle properly each
technique but get confused when they are cumulated. A tool like [nikto] will perform
atomic tests for such techniques. [http-mutate] will test them in different
combinations.
Insertion
The purpose of insertion techniques is to add fake data in the middle of a data
stream and have the IPS handle them while they are discarded by the server. They are
usually used in combination with fragmentation and may be designed in different
forms. The most common one used to bypass IPS is at the application level. It simply
uses overlapping sequence numbers and bad TCP checksums, rarely checked by IPS
for performance saving issues. They are implemented and can be tested with [http-
insert].
How to test an IPS 9
Confusion
Most simple way to test for confusion is to launch an attack on a web server
through port 443 (e.g. HTTPS). Specific settings and / or hardware is usually needed
to give IPS the possibility to analyze such traffic.
However, depending on the target architecture other confusing techniques may be
used such as the use of GRE tunnels or IPv6/IPv4 encapsulation. There is no mean to
test for such protocol is they are not implemented into the target infrastructure. On the
other hand it becomes mandatory to validate the capacity of analysis if such protocols
can be found on the network, especially if some gateways (that could be use by an
attacker to terminate its tunnels) can be found on the internal network.
10 Renaud Bidou
[webdevil] will test session pending while [http-flood] will continuously establish
multiple connections on the home page.
Traffic policing
Simulating worms
Effective worms propagation simulation implies that two main aspects are taken in
account. The first one is the vector. This is the exploit used by the worm to
compromise the target. It is a basic intrusion test to be performed according to
techniques described above. The second effect is the load on the network generated
by the propagation itself.
The capability to reduce such traffic must be tested on the IPS to evaluate its
capacity to protect internal network from flooding. Based on [worms], 4 types of
propagation are to be tested:
− TCP based : attempting to establish as much TCP connections as possible, up to
the maximum available bandwidth. [witty] was one of the most lethal one.
− UDP based : same as TCP based but just sending out UDP datagrams. The most
famous is [slammer].
− Mass mailing worms : propagating through SMTP. Resolution of MX entries
require a huge number of DNS requests. Load is then a mix of TCP (SMTP) and
UDP (DNS). [sobig] is a famous one.
− ICMP generators : In order to optimize their propagation, some worms such as
[welchia] do ping IP addresses before attempting connection. At a large scale the
ICMP traffic quickly floods the network.
Specificity in traffic policing tests is that real exploits or worms are not to be
launched, it is just a matter of generating traffic and checking for appropriate
bandwidth reservation of predefined business critical streams. A tool such as [hping]
with the --flood option will help in generating such traffic while performance testing
tools mentioned above will be used to evaluate the results.
How to test an IPS 13
Tunnels Investigation
Typically, this type of channel is created between a client (located in the protected
network) and a server (outside the protected network). Testing non standards
compliant tunnels can be done for any application above UDP or TCP thanks to
[netcat]. Once the server is launches (-l option) client must try to connect and send
data entered on STDIN. If the server gets the data, the tunnel made its way through.
Otherwise it has been blocked.
The same mechanism is to be used in order to test for application compliant
tunnels, except that specific tools are to be found for each and every protocol. As
mentioned above [itunnel] and [httptunnel] are appropriate for ICMP and HTTP
testing.
Another important point to notice is that in many cases a system that should be the
destination of a tunnel can be turned into a gateway to reach systems belonging to the
internal network on any application port. This is particularly true with SSH and SSL
servers. Then it is important either to test protection against encrypted malicious
traffic and tunneling through encrypted traffic, as it could be the second step of an
intrusion.
As an example a web server that can be reached through SSH, could have been
compromised because of a vulnerability in the HTTP server. Then the SSH daemon
could have been restarted with appropriate options so that connection to SSH service
on the web server redirect the traffic to the (supposedly) protected SQL port of a
database server.
Testing methodology
As this kind of tunnel is only another vector for transferring data, all the tests
realized without tunnels have to be done once again with tunneled data. This is
particularly important as tunnel analysis is usually performed by different piece of
software (or even hardware for performance issues). Therefore there is no guarantee
that DoS protection or anti-evasion techniques behave the same way (or even are
implemented) when tunnel investigation is set up.
In terms of tools, the best way is to create tunnels with the same devices or
software than those used in the production platform. It is very important as depending
on nature and configuration of the tunnel the quality of investigation may differ.
16 Renaud Bidou
Addendum
Resources
Documents
[covert] Covert channels – http://www.iv2-technologies/~rbidou/covert_channels.pdf
[CVE 1999-0067] phf remote command execution vulnerability -
http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-1999-0067
[dos] Denial of Service – Understanding Attack Techniques -
http://www.radware.com/content/document.asp?_v=about&document=5613
[sobig] A study of mass mailing worms -
http://www.ece.cmu.edu/~jmmccune/papers/worm04.pdf
[testing] Open source performance testing tools -
http://opensourcetesting.org/performance.php
[welchia] Virus Analysis 1 - http://www.peterszor.com/welchia.pdf
[witty] The spread of the Witty worm - http://www.caida.org/analysis/security/witty/
[worms] Worms Propagation – Traffic typology
Tools
[amap] Application identification tool – http://www.thc.org/thc-amap
[eeye] eEye Tools - Multiple attack specific vulnerability scanners -
http://www.eeye.com/html/Research/Tools/
[fragrouter] Fragrouter 1.6 – IP fragmentation and insertion tool -
http://packetstormsecurity.org/UNIX/IDS/fragrouter-1.6.tar.gz
[foundstone] Foundstone Scanning Tools – Multiple attack specific vulnerability
scanners - http://www.foundstone.com/resources/scanning.htm
[fping] Ping sweeper – www.fping.com
[honeyd] Virtual Honeypot – http://www.honeyd.org
[hping] hping3 – Multi-purpose packet crafter – http://www.hping.org
[http-insert] HTTP insertion technique – http://www.iv2-
technologies.com/~rbidou/http-insert.tar.gz
[http mutate] HTTP confusion tool – http://www.iv2-technologies.com/~rbidou/http-
mutate.tar.gz
[httprint] Web server fingerprint based identification tool - http://net-
square.com/httprint/
[httptunnel] Full HTTP compliant tunnel -
http://www.nocrew.org/software/httptunnel.html
[itunnel] ICMP tunneling tool -
http://packetstormsecurity.org/UNIX/penetration/itunnel-1_2.tar.gz
[metasploit] Exploitation Framework – http://www.metasploit.org
[netcat] GNU netcat 0.7.1 – The network Swiss army knife -
http://netcat.sourceforge.net/
18 Renaud Bidou