CHAPTER-7

INTRUSION
DETECTION AND PREVENTION SYSTEMS,
AND OTHER SECURITY TOOLS
 INTRUSION DETECTION AND PREVENTION
SYSTEMS

• An intrusion occurs when an attacker attempts to gain entry into an organization’s information
systems or disrupt their normal operations.
• Even when such attacks are self-propagating, as with viruses and distributed denial-of-service
attacks, they are almost always instigated by someone whose purpose is to harm an organization.
• Often, the differences among intrusion types lie with the attacker—some intruders don’t care
which organizations they harm and prefer to remain anonymous, while others crave notoriety.
 INTRUSION DETECTION AND PREVENTION
SYSTEMS

• Intrusion prevention consists of activities that deter an intrusion.

• Some important intrusion prevention activities are:
• Writing and implementing good enterprise information security policy
• Planning and executing effective information security programs
• Installing and testing technology-based information security countermeasures, such as firewalls and
intrusion detection and prevention systems
• Conducting and measuring the effectiveness of employee training and awareness activities.
 INTRUSION DETECTION AND PREVENTION
SYSTEMS

• Intrusion detection consists of procedures and systems that identify system intrusions.
• Intrusion reaction encompasses the actions an organization takes when an intrusion is
detected.
• These actions seek to limit the loss from an intrusion and return operations to a normal state as
rapidly as possible.
• Intrusion correction activities complete the restoration of operations to a normal state and
seek to identify the source and method of the intrusion to ensure that the same type of attack
cannot occur again—thus reinitiating intrusion prevention.
 INTRUSION DETECTION SYSTEM (IDS)

• Information security intrusion detection systems (IDSs) became commercially available in the late 1990s.
• An IDS works like a burglar alarm in that it detects a violation and activates an alarm. This alarm can be a sound, a light or
other visual signal, or a silent warning, such as an e-mail message or pager alert.
• With almost all IDSs, system administrators can choose the configuration of various alerts and the alarm levels associated
with each type of alert.
• The systems can also be configured—again like a burglar alarm—to notify an external security service of a “break-in.”
• A current extension of IDS technology is the intrusion prevention system (IPS), which can prevent an intrusion from
successfully attacking the organization by means of an active response.
• Because you seldom find an IPS that does not also have detection capabilities, the term intrusion detection and prevention
system (IDPS) is commonly used.
 IPS TECHNOLOGIES

• An IPS can interdict the attack by itself, without human intervention. This could be accomplished by:
• Terminating the user session or network connection over which the attack is being conducted
• Blocking access to the target system or systems from the source of the attack
• Blocking all access to the targeted information asset

• The IPS can modify its environment by changing the configuration of other security controls to disrupt an attack.
• This could include modifying a firewall’s rule set or configuring another network device to shut down the
communications channel to filter the offending packets.
• Some IPSs can change an attack’s components by replacing malicious content with benign material or by
quarantining a network packet’s contents.1
 IDPS TERMINOLOGY
• Alarm clustering and compaction: A process of grouping almost identical alarms that occur
nearly at the same time into a single higher-level alarm. This consolidation reduces the number of
alarms, which reduces administrative overhead and identifies a relationship among multiple
alarms.
• Alarm filtering: The process of classifying IDPS alerts so they can be more effectively managed.
An IDPS administrator can set up alarm filtering by running the system for a while to track the
types of false positives it generates and then adjusting the alarm classifications.
• Alert or alarm: An indication that a system has just been attacked or is under attack. IDPS
alerts and alarms take the form of audible signals, e-mail messages, pager notifications, or pop-up
windows.
 IDPS TERMINOLOGY

• Confidence value: The measure of an IDPS’s ability to correctly detect and identify certain types of
attacks. The confidence value an organization places in the IDPS is based on experience and past
performance measurements.
• Evasion: The process by which attackers change the format and/or timing of their activities to avoid
being detected by an IDPS.
• False attack stimulus: An event that triggers an alarm when no actual attack is in progress. Scenarios
that test the configuration of IDPSs may use false attack stimuli to determine if the IDPSs can distinguish
between these stimuli and real attacks.
• False negative: The failure of an IDPS to react to an actual attack event. This is the most grievous IDPS
failure, given that its purpose is to detect and respond to attacks.
 IDPS TERMINOLOGY

• False positive: An alert or alarm that occurs in the absence of an actual attack. A false positive
can sometimes be produced when an IDPS mistakes normal system activity for an attack. False
positives tend to make users insensitive to alarms and thus reduce their reactions to actual
intrusion events.
• Noise: Alarm events that are accurate and noteworthy but do not pose significant threats to
information security. Unsuccessful attacks are the most common source of IDPS noise, although
some noise might be triggered by scanning and enumeration tools run by network users without
harmful intent.
 IDPS TERMINOLOGY

• Site policy: The rules and configuration guidelines governing the implementation and operation of
IDPSs within the organization.
• Site policy awareness: An IDPS’s ability to dynamically modify its configuration in response to
environmental activity. A so-called dynamic IDPS can adapt its reactions in response to administrator
guidance over time and the local environment.
• True attack stimulus: An event that triggers an alarm and causes an IDPS to react as if a real attack is
in progress. The event may be an actual attack, in which an attacker is attempting a system compromise,
or it may be a drill, in which security personnel are using hacker tools to test a network segment.
• Tuning: The process of adjusting an IDPS to maximize its efficiency in detecting true positives while
minimizing false positives and false negatives.
TYPES OF IDPSS
 TYPES OF IDPSS-NIDPS

• Network-Based IDPS- A network-based IDPS (NIDPS) resides on a computer or appliance

connected to a segment of an organization’s network and monitors traffic on that segment, looking for
indications of ongoing or successful attacks.
• When the NIDPS identifies activity that it is programmed to recognize as an attack, it responds by
sending notifications to administrators.
• When examining incoming packets, an NIDPS looks for patterns within network traffic such as large
collections of related items of a certain type, which could indicate that a DoS attack is under way.
• An NIDPS also examines the exchange of a series of related packets in a certain pattern, which could
indicate that a port scan is in progress.
 TYPES OF IDPSS-NIDPS

• An NIDPS is installed at a specific place in the network, such as inside an edge router, where it is
possible to monitor traffic into and out of a particular network segment.
• The NIDPS can be deployed to monitor a specific grouping of host computers on a specific
network segment, or it may be installed to monitor all traffic between the systems that make up
an entire network.
• When placed next to a hub, switch, or other key networking device, the NIDPS may use that
device’s monitoring port. A monitoring port, also known as a switched port analysis (SPAN) port
or mirror port, is capable of viewing all traffic that moves through the entire device.
 THE ADVANTAGES OF NIDPSS

• Good network design and placement of NIDPS devices can enable an organization to monitor a
large network using only a few devices.
• NIDPSs are usually passive devices and can be deployed into existing networks with little or no
disruption to normal network operations.
• NIDPSs are not usually susceptible to direct attack and may not be detectable by attackers.
 DISADVANTAGES OF NIDPSS

• An NIDPS can become overwhelmed by network volume and fail to recognize attacks it might
otherwise have detected.
• NIDPSs require access to all traffic to be monitored.
• NIDPSs cannot analyze encrypted packets, making some network traffic invisible to the process.
• NIDPSs cannot reliably ascertain whether an attack was successful, which requires ongoing effort
by the network administrator to evaluate logs of suspicious network activity.
 WIRELESS NIDPS

• A wireless IDPS monitors and analyzes wireless network traffic, looking for potential problems
with the wireless protocols (Layers 2 and 3 of the OSI model).
• Unfortunately, wireless IDPSs cannot evaluate and diagnose issues with higher-layer protocols
like TCP and UDP. \
• Wireless IDPS capability can be built into a device that provides a wireless access point (AP).
• Sensors for wireless networks can be located at the access points, on specialized sensor
components, or incorporated into selected mobile stations.
 WIRELESS NIDPS

Wireless NIDPS can detect the following:

• Unauthorized WLANs and WLAN devices
• Poorly secured WLAN devices
• Unusual usage patterns
• The use of wireless network scanners
• DoS attacks and conditions
• Impersonation and man-in-the-middle attacks
 HOST-BASED IDPS

• While a network-based IDPS resides on a network segment and monitors activities across that segment,
a host-based IDPS (HIDPS) resides on a particular computer or server, known as the host, and monitors
activity only on that system.
• HIDPSs are also known as system integrity verifiers because they benchmark and monitor the status of
key system files and detect when an intruder creates, modifies, or deletes monitored files.
• An HIDPS has an advantage over an NIDPS in that it can access encrypted information traveling over the
network and use it to make decisions about potential or actual attacks.
• Also, because the HIDPS works on only one computer system, all the traffic it examines traverses that
system.
• The packet delivery mode is not a factor.
 ADVANTAGES OF HIDPSS
• An HIDPS can detect local events on host systems and detect attacks that may elude a
network-based IDPS.
• An HIDPS functions on the host system, where encrypted traffic will have been decrypted and is
available for processing.
• The use of switched network protocols does not affect an HIDPS.
• An HIDPS can detect inconsistencies in how applications and systems programs were used by
examining the records stored in audit logs. This can enable the HIDPS to detect some types of
attacks, including Trojan horse programs.
 THE DISADVANTAGES OF HIDPSS

• HIDPSs pose more management issues because they are configured and managed on each monitored host. An HIDPS
requires more management effort to install, configure, and operate than a comparably sized NIDPS solution.
• An HIDPS is vulnerable both to direct attacks and to attacks against the host operating system. Either attack can result in the
compromise or loss of HIDPS functionality.
• An HIDPS is not optimized to detect multihost scanning, nor is it able to detect scanning from network devices that are not
hosts, such as routers or switches. Unless complex correlation analysis is provided, the HIDPS will not be aware of attacks
that span multiple devices in the network.
• An HIDPS is susceptible to some DoS attacks.
• An HIDPS can use large amounts of disk space to retain the host OS audit logs; for the HIDPS to function properly, it may be
necessary to add disk capacity to the system.
 IDPS DETECTION METHODS
• IDPSs use a variety of detection methods to monitor and evaluate network traffic. Three
methods dominate: signature-based detection, anomaly-based detection, and stateful protocol
analysis.
• Signature-Based Detection -An IDPS that uses signature-based detection (sometimes called
knowledge-based detection or misuse detection) examines network traffic in search of patterns
that match known signatures—that is, preconfigured, predetermined attack patterns.
• Signature-based technology is widely used because many attacks have clear and distinct
signatures:
• A potential problem with the signature-based approach is that new attack patterns must
continually be added to the IDPS’s database of signatures; otherwise, attacks that use new
strategies will not be recognized and might succeed.
 IDPS DETECTION METHODS
• Anomaly-Based Detection- Anomaly-based detection (or behavior-based detection) collects statistical
summaries by observing traffic that is known to be normal.
• This normal period of evaluation establishes a performance baseline over a period of time known as the training
period.
• Once the baseline is established, the IDPS periodically samples network activity and uses statistical methods to
compare the sampled activity to the baseline.
• When the measured activity is outside the baseline parameters—exceeding what is called the clipping level—the
IDPS sends an alert to the administrator.
• The baseline data can include variables such as host memory or CPU usage, network packet types, and packet
quantities.
• The advantage of anomaly-based detection is that the IDPS can detect new types of attacks because it looks for
abnormal activity of any type.
• Unfortunately, these systems require much more overhead and processing capacity than signature-based IDPSs
 IDPS DETECTION METHODS
• Stateful Protocol Analysis- SPA uses the opposite of a signature approach. Instead of
comparing known attack patterns against observed traffic or data, the system compares known
normal or benign protocol profiles against observed traffic.
• These profiles are developed and provided by the protocol vendors.
• Essentially, the IDPS knows how a protocol such as FTP is supposed to work, and therefore can
detect anomalous behavior.
• By storing relevant data detected in a session and then using it to identify intrusions that involve
multiple requests and responses, the IDPS can better detect specialized, multisession attacks.
• This process is sometimes called deep packet inspection because SPA closely examines packets
at the application layer for information that indicates a possible intrusion.
 IDPS RESPONSE BEHAVIOR
• IDPS Response Options When an IDPS detects a possible intrusion, it has several response options,
depending on the organization’s policy, objectives, and system capabilities.
• When configuring an IDPS’s responses, the system administrator must ensure that a response to an
attack or potential attack does not inadvertently exacerbate the situation.
• IDPS responses can be classified as active or passive.
• An active response is a definitive action that is automatically initiated when certain types of alerts are
triggered. These responses can include collecting additional information, changing or modifying the
environment, and taking action against the intruders.
• Passive-response IDPSs simply report the information they have collected and wait for the administrator
to act. Generally, the administrator chooses a course of action after analyzing the collected data.
 IDPS RESPONSE OPTIONS
• Audible/visual alarm: The IDPS can trigger a .wav file, beep, whistle, siren, or other audible or
visual notification to alert the administrator of an attack.
• SNMP traps and plug-ins: The Simple Network Management Protocol contains trap functions,
which allow a device to send a message to the SNMP management console indicating that a
certain threshold has been crossed, either positively or negatively.
• E-mail message: The IDPS can send e-mail to notify network administrators of an event.
• Page or phone message: The IDPS can be configured to dial a phone number and produce an
alphanumeric page or other type of signal or message.
• Log entry: The IDPS can enter information about the event into an IDPS system log file or
operating system log file.
 IDPS RESPONSE OPTIONS
• Evidentiary packet dump: Organizations that require an audit trail of IDPS data may choose
to record all log data in a special way. This method allows the organization to perform further
analysis on the data and to submit the data as evidence in a civil or criminal case.
• Take action against the intruder: Although it is not advisable, organizations can take action
against an intruder using trap-and-trace, back-hacking, or trace-back methods. Such responses
involve configuring intrusion detection systems to trace the data from the target system back to
the attacking system to initiate a counterattack.
• Launch program: An IDPS can be configured to execute a specific program when it detects
specific types of attacks.
• Reconfigure firewall: An IDPS can send a command to the firewall to filter out suspected
packets by IP address, port, or protocol.
 STRENGTHS OF INTRUSION DETECTION AND
PREVENTION SYSTEMS
• Monitoring and analysis of system events and user behaviors
• Testing the security states of system configurations
• Baselining the security state of a system, then tracking any changes to that baseline
• Recognizing patterns of system events that correspond to known attacks
• Recognizing patterns of activity that statistically vary from normal activity
• Managing operating system audit and logging mechanisms and the data they generate
• Alerting appropriate staff by appropriate means when attacks are detected
• Measuring enforcement of security policies encoded in the analysis engine
• Providing default information security policies
• Allowing people who are not security experts to perform important security monitoring functions
 LIMITATIONS OF INTRUSION DETECTION AND
PREVENTION SYSTEMS
• Compensating for weak or missing security mechanisms in the protection infrastructure, such as
firewalls, identification and authentication systems, link encryption systems, access control mechanisms,
and virus detection and eradication software
• Instantaneously detecting, reporting, and responding to an attack when there is a heavy network or
processing load
• Detecting newly published attacks or variants of existing attacks
• Effectively responding to attacks launched by sophisticated attackers
• Automatically investigating attacks without human intervention
• Resisting all attacks that are intended to defeat or circumvent them
• Compensating for problems with the fidelity of information sources
• Dealing effectively with switched networks
 SCANNING AND ANALYSIS TOOLS

• Port scanning utilities, or port scanners, are tools that can either perform generic scans or
those for specific types of computers, protocols, or resources. You need to understand the
network environment and the scanning tools at your disposal so you can use the one best suited
to the data collection task at hand.
• The more specific the scanner is, the more useful its information is to attackers and defenders.
• Probably the most popular port scanner is Nmap, which runs both on UNIX and Windows
systems.
 SCANNING AND ANALYSIS TOOLS

• A port is a network channel or connection point in a data communications system. Within the TCP/IP
networking protocol, TCP and User Datagram Protocol (UDP) port numbers differentiate the multiple
communication channels that connect to the network services offered on a network device.
• An attacker can use an open port to send commands to a computer, potentially gain access to a server,
and possibly exert control over a networking device.
• As a rule of thumb, any port that is not absolutely necessary for conducting business should be secured
or removed from service.
• The number and nature of the open ports on a system are an important part of its attack surface.
• As a general design goal, security practitioners seek to reduce the attack surface of each system to
minimize the potential for latent defects and unintended consequences to cause losses.
SCANNING AND ANALYSIS TOOLS
 SCANNING AND ANALYSIS TOOLS

• Firewall Analysis Tools-Understanding exactly where an organization’s firewall is located, and the
functions of its existing rule sets are very important steps for any security administrator.
• Several tools automate the remote discovery of firewall rules and assist the administrator (or attacker)
in analyzing the rules to determine what they allow and reject.
• Administrators who are wary of using the same tools that attackers use should remember two
important points.
• Regardless of the tool that is used to validate or analyze a firewall’s configuration, user intent dictates how the
gathered information is used.
• To defend a computer or network well, administrators must understand the ways it can be attacked. Thus, a
tool that can help close an open or poorly configured firewall will help the network defender minimize the risk
from attack.
 SCANNING AND ANALYSIS TOOLS

• Operating System Detection Tools-

• Vulnerability Scanners- Active vulnerability scanners examine networks for highly detailed
information.
• An active scanner is one that initiates traffic on the network to determine security holes.
• An example of a vulnerability scanner is Nessus, a professional freeware utility that uses IP packets to
identify hosts available on the network, the services (ports) they offer, their operating system and OS
version, the type of packet filters and firewalls in use, and dozens of other network characteristics.
 SCANNING AND ANALYSIS TOOLS

• A passive vulnerability scanner listens in on the network and identifies vulnerable versions of
both server and client software.
• Advantage of using passive scanners is that they do not require vulnerability analysts to obtain
approval prior to testing.
• These tools simply monitor the network connections to and from a server to obtain a list of
vulnerable applications.
 SCANNING AND ANALYSIS TOOLS
• Operating System Detection Tools- Packet Sniffer
• A packet sniffer or network protocol analyzer can provide a network administrator with
valuable information for diagnosing and resolving networking issues.
• In the wrong hands, however, a sniffer can be used to eavesdrop on network traffic.
• Commercial and open-source sniffers are both available—for example, Sniffer is a commercial
product and Snort is open-source software.
• An excellent network protocol analyzer is Wireshark (www.wireshark.org), formerly known as
Ethereal, which is available in open-source and commercial versions.
 WIRELESS SECURITY TOOLS

• A wireless connection is convenient, but it has many potential security holes.

• An organization that spends all of its time securing the wired network while ignoring wireless
networks is exposing itself to a security breach.
• As a security professional, you must assess the risk of wireless networks.
• A wireless security toolkit should include the ability to sniff wireless traffic, scan wireless hosts,
and assess the level of privacy or confidentiality afforded on the wireless network.
 WIRELESS SECURITY TOOLS
• Aircrack, a wireless network protocol cracking tool
• Kismet, a powerful wireless network protocol sniffer, network detector, and IDPS, which works
by passively sniffing the networks
• NetStumbler, a freeware Windows file parser available at www.netstumbler.org
• inSSIDer, an enhanced scanner for Windows, OS X, and Android
• KisMac, a GUI passive wireless stumbler for Mac OS X (a variation of Kismet)