Principles of Information Security 7E - Module 9
Principles of Information Security 7E - Module 9
Security Technology:
Intrusion Detection and
Prevention Systems and
Other Security Tools
Upon completion of this material, you should be able to: Do not wait; the time
1 Identify and describe the categories and models of intrusion detection and will never be just
prevention systems right. Start where
2 Describe the detection approaches employed by modern intrusion detection and you stand, and work
prevention systems with whatever tools
3 Define and describe honeypots, honeynets, and padded cell systems you may have at your
4 List and define the major categories of scanning and analysis tools and describe command, and better
the specific tools used within each category tools will be found as
you go along.
—Napoleon Hill (1883–1970),
Founder of The Science of Success
Opening Scenario
Miller Harrison was going to make them sorry and make them pay. Earlier today, his contract with SLS had been terminated,
and he’d been sent home. Oh, sure, the big-shot manager, Charlie Moody, had said Miller would still get paid for the two
weeks remaining in his contract and that the decision was based on “changes in the project and evolving needs as project work
continued,” but Miller knew better. He knew he’d been let go because of that know-nothing Kelvin and his simpering lapdog
Laverne Nguyen. And now he was going to show them and everyone else at SLS who knew more about security.
Miller knew that the secret to hacking into a network successfully was to apply the same patience, attention to detail,
and dogged determination that defending a network required. He also knew that the first step in a typical hacking protocol
was footprinting—that is, getting a fully annotated diagram of the network. Miller already had one—in a violation of company
policy, he had brought a copy home last week when Laverne first started trying to tell him how to do his job.
When they terminated his contract today, Miller’s supervisors made him turn in his company laptop and then actually
had the nerve to search his briefcase. By then, however, Miller had already stashed all the files and access codes he needed
to attack SLS’s systems.
338 Principles of Information Security
To begin, he thought about activating his VPN client to connect to the SLS network from his free Wi-Fi connection at his
favorite coffee shop. But then he remembered that Charlie Moody had confiscated the authentication token that enabled him
to use the VPN for remote access, and it would also be obvious who had attacked the system. No problem, Miller thought.
Let’s see how good SLS is at protecting its antiquated dial-up lines. He connected his laptop to its wireless cellular modem and
entered the number for SLS’s legacy modem bank; he had gotten the number from the network administrator’s desk. “Silly
man,” he thought, “writing passwords on sticky notes.” After the connection was established, Miller positioned his hands on
the keyboard and then read the prompt on his screen:
SLS Inc. Company Use Only. Unauthorized use is prohibited and subject to prosecution.
Enter Username:
Enter Password:
Enter Dynamic Authentication Code:
Miller muttered under his breath. Apparently, the SLS security team had rerouted all dial-up connections to the same
RADIUS authentication server that the VPN used. So, he was locked out of the back door, but no worries. Miller moved on
to his next option, which was to use a back door of his very own. It consisted of a zombie program he had installed on the
company’s extranet quality assurance server. No one at SLS worried about securing the QA server because it did not store
any production data. In fact, the server wasn’t even subject to the change control procedures that were applied to other sys-
tems on the extranet.
Miller’s action was risky, as there was a slight chance that SLS had added the server to the host intrusion detection
and prevention system it deployed last quarter. If so, Miller would be detected before he got too far. He activated the pro-
gram he used to remotely control the zombie program and typed in the IP address of the computer running the zombie.
No response. He opened a command window and pinged the zombie. The computer at that address answered each ping
promptly, which meant it was alive and well. Miller checked the zombie’s UDP port number and ran an Nmap scan against
the port on that system. No response. He cursed the firewall, the policy that controlled it, and the technicians who kept it
up to date.
With all of his planned payback cut off at the edge of SLS’s network, he decided to continue his hack by going back to the
first step—specifically, to perform a detailed fingerprinting of all SLS Internet addresses. Because the front door and both back
doors were locked, it was time to get a new floor plan. He launched a simple network port scanner from his Linux laptop and
configured it to scan the entire IP address range for SLS’s extranet. With a single keystroke, he unleashed the port scanner
on the SLS network.
anonymous, while others crave notoriety. While every intrusion is an incident, not every incident is an intrusion;
examples include service outages and natural disasters.
As you learned in Module 3, “Information Security Management,” intrusion prevention consists of activities that
deter or even stop an intrusion. Some important intrusion prevention activities are writing and implementing good
enterprise information security policy; planning and executing effective information security programs; installing
and testing technology-based information security countermeasures, such as firewalls and intrusion detection and
prevention systems; and conducting and measuring the effectiveness of employee training and awareness activities.
Intrusion detection consists of procedures and systems that identify system intrusions. Intrusion reaction encom-
passes the actions an organization takes when an intrusion is detected. These actions seek to limit the loss from an
intrusion and return operations to a normal state as rapidly as possible. Intrusion correction activities complete the
restoration of operations to a normal state and seek to identify the source and method of the intrusion to ensure that
the same type of attack cannot occur again—thus reinitiating intrusion prevention.
Information security intrusion detection systems (IDSs) became commercially intrusion detection
available in the late 1990s. An IDS works like a burglar alarm in that it detects a viola- system (IDS)
tion and activates an alarm. This alarm can be a sound, a light or other visual signal, A system capable of automati-
or a silent warning, such as an e-mail message or text alert. With almost all IDSs, cally detecting an intrusion into
an organization’s networks or host
system administrators can choose the configuration of various alerts and the alarm systems and notifying a designated
levels associated with each type of alert. Many IDSs enable administrators to config- authority.
ure the systems to notify them directly of trouble via their e-mail or smartphones.
The systems can also be configured—again like a burglar alarm—to notify an exter- intrusion detection
nal security service of a “break-in.” The configurations that enable IDSs to provide and prevention system
customized levels of detection and response are quite complex. A current extension (IDPS)
of IDS technology is the incorporation of intrusion prevention technology, which can The general term for a system
that can both detect and modify
prevent an intrusion from successfully attacking the organization by means of an its configuration and environment
active response. Because you seldom find a prevention system that does not also to prevent intrusions. An IDPS
have detection capabilities, the term intrusion detection and prevention system encompasses the functions of both
intrusion detection systems and
(IDPS) is commonly used.
intrusion prevention technology.
According to NIST Special Publication (SP) 800-94, Rev. 1, IDPSs use several
response techniques, which can be divided into the following groups:
An IDPS is capable of interdicting the attack by itself, without human intervention. This could be accomplished
by the following:
❍ Terminating the user session or network connection over which the attack is being conducted
❍ Blocking access to the target system or systems from the source of the attack, such as a compromised user
The IDPS can dynamically modify its environment by changing the configuration of other security controls to
disrupt an attack. This could include modifying a firewall’s rule set or configuring another network device to
shut down the communications channel to filter the offending packets.
Some IDPSs are capable of changing an attack’s components by replacing malicious content with benign mate-
rial or by quarantining a network packet’s contents.1
IDPS Terminology
To understand how an IDPS works, you must first become familiar with some IDPS terminology.
Alarm or alert—An indication or notification that a system has just been attacked or is under attack. IDPS
alerts and alarms take the form of audible signals, e-mail messages, pager notifications, or pop-up windows.
Alarm clustering and compaction—A process of grouping almost identical alarms that occur nearly at the
same time into a single higher-level alarm. This consolidation reduces the number of alarms, which reduces
administrative overhead and identifies a relationship among multiple alarms. Clustering may be based on
combinations of frequency, similarity in attack signature, similarity in attack target, or other criteria that are
defined by system administrators.
340 Principles of Information Security
Alarm filtering—The process of classifying IDPS alerts so they can be more effectively managed. An IDPS
administrator can set up alarm filtering by running the system for a while to track the types of false positives
it generates and then adjusting the alarm classifications. For example, the administrator may set the IDPS to
discard alarms produced by false attack stimuli or normal network operations. Alarm filters are similar to
packet filters in that they can filter items by their source or destination IP addresses, but they can also filter
by operating systems, confidence values, alarm type, or alarm severity.
Confidence value—The measure of an IDPS’s ability to correctly detect and identify certain types of attacks.
The confidence value an organization places in the IDPS is based on experience and past performance measure-
ments. The confidence value, which is based on fuzzy logic, helps an administrator determine the likelihood
that an IDPS alert or alarm indicates an actual attack in progress. For example, if a system deemed 90 percent
capable of accurately reporting a denial-of-service (DoS) attack sends a DoS alert, there is a high probability
that an actual attack is occurring.
Evasion—The process by which attackers change the format or timing of their activities to avoid being
detected by an IDPS.
False attack stimulus—An event that triggers an alarm when no actual attack is in progress. Scenarios that
test the configuration of IDPSs may use false attack stimuli to determine if the IDPSs can distinguish between
these stimuli and real attacks.
False negative—The failure of a technical control (such as an IDPS) to react to an actual attack event. This is
the most grievous IDPS failure, given that its purpose is to detect and respond to attacks.
False positive—An alert or alarm that occurs in the absence of an actual attack. A false positive can sometimes
be produced when an IDPS mistakes normal system activity for an attack. False positives tend to make users
insensitive to alarms and thus reduce their reactions to actual intrusion events.
Noise—In incident response, alarm events that are accurate and noteworthy but do not pose significant
threats to information security. Unsuccessful attacks are the most common source of IDPS noise, although
some noise might be triggered by scanning and enumeration tools run by network users without harmful
intent.
Site policy—The rules and configuration guidelines governing the implementation and operation of IDPSs
within the organization.
Site policy awareness—An IDPS’s ability to dynamically modify its configuration in response to environ-
mental activity. A so-called dynamic IDPS can adapt its reactions in response to administrator guidance over
time and the local environment. A dynamic IDPS logs events that fit a specific profile instead of minor events,
such as file modifications or failed user logins. A smart IDPS knows when it does not need to alert the admin-
istrator—for example, when an attack is using a known and documented exploit from which the system is
protected.
True attack stimulus—An event that triggers an alarm and causes an IDPS to react as if a real attack is in
progress. The event may be an actual attack, in which an attacker is attempting a system compromise, or it
may be a drill, in which security personnel are using hacker tools or performing port scanning to test a net-
work segment.
Tuning—The process of adjusting an IDPS to maximize its efficiency in detecting true positives while minimiz-
ing false positives and false negatives.
Intrusion Detection
The primary purpose of an IDPS is to identify and report an intrusion. By detecting the early signs of an intrusion, the
organization can quickly contain the attack and prevent or at least substantially mitigate loss or damage to information
assets. The notification process is critical; if the organization is not notified that an intrusion is under way, the IDPS
Module 9 Security Technology: Intrusion Detection and Prevention Systems and Other Security Tools 341
serves no real purpose. Once notified, the organization’s IR team can activate the IR plan and contain the intrusion.
Notification is described later in this module.
IDPSs can also help administrators detect the preambles to attacks; this is known as attack reconnaissance.
Most attacks begin with an organized and thorough probing of the organization’s network environment and its
defenses. This initial probing is called doorknob rattling and is accomplished through two general activities. Foot-
printing refers to activities that gather information about the organization, its network activities, and its assets,
while fingerprinting refers to activities that scan network locales for active systems and then identify the network
services offered by the host systems. A system that can detect the early warning signs of footprinting and finger-
printing functions like a neighborhood watch that spots would-be burglars as they case the community. This early
detection enables administrators to prepare for a potential attack or to minimize potential losses from an attack.
IDPSs can also help the organization protect its assets when its networks and
systems are still exposed to known vulnerabilities or are unable to respond to known vulnerability
a rapidly changing threat environment. Many factors can delay or undermine an A published weakness or fault in an
organization’s ability to secure its systems from attack and subsequent loss. For information asset or its protective
example, even though popular information security technologies such as scanning systems that may be exploited and
result in loss.
tools allow security administrators to evaluate the readiness of their systems,
they may still fail to detect or correct a known deficiency or check for vulnerabilities frequently enough. In addi-
tion, even when a vulnerability is detected in a timely manner, it cannot always be corrected quickly. Because such
corrective measures usually require administrators to install patches and upgrades, they are subject to fluctua-
tions in the administrator’s workload.
Note that vulnerabilities might be known to vulnerability-tracking groups without being known to the general pub-
lic. The number and complexity of reported vulnerabilities continue to increase, so it is extremely difficult to stay on
top of them. Instead, organizations rely on developers to identify problems and patch systems, yet there is inevitably a
delay between detection and distribution of a patch or update to resolve the vulnerability. Similarly, substantial delays
are common between the detection of a new virus or worm and the distribution of a signature that allows antimalware
applications to detect and contain the threat.
To further complicate the matter, services that are known to be vulnerable sometimes cannot be disabled or
otherwise protected because they are essential to ongoing operations. When a system has a known vulnerability or
deficiency, an IDPS can be set up to detect attacks or attempts to exploit existing weaknesses, an important part of
the strategy of defense in depth.
While a diligent organization may be well prepared against known vulnerabilities, zero-day
it’s the unknown that still causes the organization concern. Zero-day vulnerabilities vulnerability
(or zero-day attacks) are unknown or undisclosed vulnerabilities that can’t be pre- An unknown or undisclosed vul-
dicted or prepared for. They are called zero-day (or zero-hour) because once they are nerability in an information asset
or its protection systems that may
discovered, the technology owners have zero days to identify, mitigate, and resolve be exploited and result in loss; once
the vulnerability. Unfortunately, most of these vulnerabilities become “known” only it is discovered, there are zero days
when they are used in an attack. Therefore, it is critical for the organization to dili- to identify, mitigate, and resolve the
vulnerability.
gently monitor online trade press and industry user groups to stay abreast of such
issues.
Organizations continue to expand the number of items on networks they manage and where those items are
operated. The Internet of Things requires different types of devices to be connected, while cloud service use results
in valuable assets housed in places where defenses are established with software-defined perimeters instead of the
old-school hardware perimeter. These changes in how networks are used and what can be found on them make the
need for IDPS technologies even more pronounced.
Data Collection
In the process of analyzing data and network activity, IDPSs can be configured to log data for later analysis. This logging
function allows the organization to examine what happened after an intrusion occurred and why. As an accountability
function, logging may even provide the who if the individual responsible for the intrusion works within the organiza-
tion. Even when intruders are not internal, some information may be available, such as where they are connecting
342 Principles of Information Security
from (IP address) and how they connected (browser details). Logging also allows improvement in incident response;
evaluation by specialized log monitors, as described later in this module; and assessment of the effectiveness of the
IDPS itself.
Even if an IDPS fails to prevent an intrusion, it can still contribute to the after-attack review by assisting inves-
tigators in determining how the attack occurred, what the intruder accomplished, and which methods the attacker
employed. This information can be used to remedy deficiencies and to prepare the organization’s network environment
for future attacks. The IDPS can also provide forensic information that may be useful if the attacker is caught and then
prosecuted or sued.
Examining this information to understand attack frequencies and attributes can help identify insufficient, inap-
propriate, or compromised security measures. This process can also provide insight for management into threats the
organization faces and can help justify current and future expenditures to support and improve incident detection
controls. When asked for funding to implement additional security technology, upper management usually requires
documentation of the threat from which the organization must be protected.
Attack Deterrence
Another reason to install an IDPS is that it serves as a deterrent by increasing the fear of detection among would-be
attackers. If internal and external users know that an organization has deployed an IDPS, they are less likely to probe
the system or attempt to compromise it, just as criminals are much less likely to break into a house that appears to
have a burglar alarm.
Types of IDPSs
IDPSs generally operate as network- or host-based systems. A network-based IDPS is focused on protecting network
information assets by examining network communications traffic. Two specialized types of network-based IDPSs are
the wireless IDPS and the network behavior analysis (NBA) IDPS. The wireless IDPS focuses on wireless networks, as
the name indicates, while the NBA IDPS examines traffic flow on a network in an attempt to recognize abnormal pat-
terns like DDoS, malware, and policy violations.
A host-based IDPS protects a server or host’s information assets, usually by monitoring the files stored on the
system or by monitoring the actions of connected users; the example shown in Figure 9-2 monitors both network con-
nection activity and current information states on host servers. The application-based model works on one or more
host systems that support a single application and defends that application from special forms of attack.
Module 9 Security Technology: Intrusion Detection and Prevention Systems and Other Security Tools 343
Source: https://countuponsecurity.com/tag/kill-chain/.
Web sites such as automated tool targeted
triggers intruders’ on the victim hosts must beacon objectives.
conference (weaponizer). environment using
code. Most often, system allows the outbound to an Typically, this
proceedings and Increasingly, client vectors like e-mail
exploitation adversary to Internet controller objective is data
mailing lists for applications’ data attachments,
targets an maintain server to establish exfiltration that
e-mail addresses, files such as Adobe Web sites, and USB
application or persistence inside a C2 channel. involves collecting,
social PDF or Microsoft removable media.
operating system the environment. encrypting, and
relationships, or Office documents
vulnerability. extracting
information on serve as the
information from
specific weaponized
the victim
technologies. deliverable.
environment.
External router
Untrusted
network
Data Header 0100101011
Packet
flow
Network-Based IDPS
network-based IDPS
A network-based IDPS (NIDPS) consists of a specialized hardware appliance and (NIDPS)
software designed to monitor network traffic. The NIDPS may include separate man- An IDPS that resides on a computer
agement software, referred to as a console, and a number of specialized hardware or appliance connected to a seg-
and software components, referred to as agents or sensors. These agents can be ment of an organization’s network
and monitors traffic on that seg-
installed on other network segments and network technologies to remotely moni- ment, looking for indications of
tor traffic at multiple locations for a potential intrusion, reporting back to the cen- ongoing or successful attacks.
tral NIDPS application. When the NIDPS identifies activity that it is programmed to
344 Principles of Information Security
For more information on the Snort Network IDPS Engine, visit www.snort.org. For more information on the Snorby
i front end for Snort and other IDPSs, visit https://github.com/Snorby/snorby.
application protocol In application protocol verification, the higher-order protocols (HTTP, SMTP,
verification and FTP) are examined for unexpected packet behavior or improper use. Sometimes
The process of examining and an attack uses valid protocol packets but in excessive quantities; in the case of the
verifying the higher-order proto-
tiny fragment attack, the packets are also excessively fragmented. While protocol
cols (HTTP, FTP, and Telnet) in net-
work traffic for unexpected packet stack verification looks for violations in the protocol packet structure, application
behavior or improper use. protocol verification looks for violations in the protocol packet’s use. One example
Module 9 Security Technology: Intrusion Detection and Prevention Systems and Other Security Tools 345
Source: github.com/snorby.
Figure 9-3 Snorby demo
is DNS cache poisoning, in which valid packets exploit poorly configured DNS servers to inject false information and
corrupt the servers’ answers to routine DNS queries from other systems on the network. Unfortunately, this higher-
order examination of traffic can have the same effect on an IDPS as it can on a firewall—that is, it slows the throughput
of the system. It may be necessary to have more than one NIDPS installed, with one of them performing protocol stack
verification and one performing application protocol verification.
The advantages of NIDPSs include the following:
Good network design and placement of NIDPS devices can enable an organization to monitor a large network
using only a few devices.
NIDPSs are usually passive devices and can be deployed into existing networks with little or no disruption to
normal network operations.
NIDPSs are not usually susceptible to direct attack and may not be detectable by attackers.3
An NIDPS can become overwhelmed by network volume and fail to recognize attacks it might otherwise have
detected. Some IDPS vendors are accommodating the need for ever-faster network performance by improving
the processing of detection algorithms in dedicated hardware circuits. Additional efforts to optimize rule set
processing may also reduce the overall effectiveness of detecting attacks.
NIDPSs require access to all traffic to be monitored. As mentioned earlier, switches have replaced hubs.
Because some switches have limited or no monitoring port capability, some networks are not capable of pro-
viding aggregate data for analysis by an NIDPS. Even when switches do provide monitoring ports, they may
not be able to mirror all activity with a consistent and reliable time sequence.
NIDPSs cannot analyze encrypted packets, making some network traffic invisible to the process. The increas-
ing use of encryption that hides the contents of some or all packets by some network services (such as SSL,
SSH, and VPN) limits the effectiveness of NIDPSs.
NIDPSs cannot reliably ascertain whether an attack was successful, which requires ongoing effort by the net-
work administrator to evaluate logs of suspicious network activity.
Some forms of attack are not easily discerned by NIDPSs, specifically those involving fragmented packets.
In fact, some NIDPSs are so vulnerable to malformed packets that they may become unstable and stop
functioning.4
Wireless NIDPS A wireless IDPS monitors and analyzes wireless network traffic, looking for potential problems with
the wireless protocols (Layers 2 and 3 of the OSI model). Unfortunately, wireless IDPSs cannot evaluate and diagnose
issues with higher-layer protocols like TCP and UDP. Wireless IDPS capability can be built into a device that provides
a wireless access point (AP).
346 Principles of Information Security
Sensors for wireless networks can be located at the access points, on specialized sensor components, or in
selected mobile stations. Centralized management stations collect information from these sensors, much as other
network-based IDPSs do, and aggregate the information into a comprehensive assessment of wireless network intru-
sions. The implementation of wireless IDPSs includes the following issues:
Physical security—Unlike wired network sensors, which can be physically secured, many wireless sensors are
located in public areas like conference rooms, lobbies, galleries, and hallways to obtain the widest possible
network range. Some of these locations may even be outdoors; more and more organizations are deploying
networks in external locations to provide seamless connectivity across the organization. Thus, the physical
security of these devices may require additional configuration and monitoring.
Sensor range—A wireless device’s range can be affected by atmospheric conditions, building construction, and
the quality of the wireless network card and access point. Some IDPS tools allow an organization to identify
the optimal location for sensors by modeling the wireless footprint based on signal strength. Sensors are most
effective when their footprints overlap.
Access point and wireless switch locations—Wireless components with bundled IDPS capabilities must be care-
fully deployed to optimize the IDPS sensor detection grid. The minimum range is just that; you must guard
against the possibility of an attacker connecting to a wireless access point from a range far beyond the minimum.
Wired network connections—Wireless network components work independently of the wired network when
sending and receiving traffic between stations and access points. However, a network connection eventually
integrates wireless traffic with the organization’s wired network. In places where no wired network connection
is available, it may be impossible to deploy a sensor.
Cost—The more sensors you deploy, the more expensive the configuration. Wireless components typically
cost more than their wired counterparts, so the total cost of ownership of IDPSs for both wired and wireless
varieties should be carefully considered.
AP and wireless switch locations—The locations of APs and wireless switches are important for organizations
buying bundled solutions (APs with preinstalled IDPS applications).5
In addition to the traditional types of intrusions detected by other IDPSs, the wireless IDPS can detect existing
WLANs and WLAN devices for inventory purposes as well as detect the following:
Unauthorized WLANs and WLAN devices
Poorly secured WLAN devices
Unusual usage patterns
The use of wireless network scanners
DoS attacks and conditions
Impersonation and man-in-the-middle attacks6
Wireless IDPSs are generally more accurate than other types of IDPSs, mainly because of the reduced set of proto-
cols and packets they have to examine. However, they are unable to detect certain passive wireless protocol attacks, in
which the attacker monitors network traffic without active scanning and probing. Wireless IDPSs are also susceptible
to evasion techniques. By simply looking at wireless devices, which are often visible in public areas, attackers can
design customized evasion methods to exploit the system’s channel scanning scheme. Wireless IDPSs can protect their
associated WLANs, but they may be susceptible to logical and physical attacks on the wireless access point or the IDPS
devices themselves. Always keep in mind the physical vulnerabilities of security technologies: “The best-configured
security technology in the world cannot withstand an attack using a well-placed brick.”7
Network Behavior Analysis System NBA systems identify problems related to the flow of network traffic. They use
a version of the anomaly detection method described later in this section to identify excessive packet flows that might
occur in the case of equipment malfunction, DoS attacks, virus and worm attacks, and some forms of network policy
violations. NBA IDPSs typically monitor internal networks but occasionally monitor connections between internal and
external networks. Intrusion detection and prevention typically includes the following relevant flow data:
Source and destination IP addresses
Source and destination TCP or UDP ports or ICMP types and codes
Number of packets and bytes transmitted in the session
Starting and ending timestamps for the session8
Module 9 Security Technology: Intrusion Detection and Prevention Systems and Other Security Tools 347
Most NBA sensors can be deployed in passive mode only, using the passive mode
same connection methods (e.g., network tap, switch spanning port) as An IDPS sensor setting in which the
network-based IDPSs. Passive sensors that are performing direct network device simply monitors and ana-
lyzes observed network or system
monitoring should be placed so that they can monitor key network
traffic.
locations, such as the divisions between networks, and key network
segments, such as demilitarized zone (DMZ) subnets. Inline sensors are
inline sensor
typically intended for network perimeter use, so they would be deployed in
An IDPS sensor intended for net-
close proximity to the perimeter firewalls, often between the firewall and work perimeter use and deployed
the Internet border router to limit incoming attacks that could overwhelm in close proximity to a perimeter
the firewall.9 firewall to detect incoming attacks
that could overwhelm the firewall.
Figures 9-4 and 9-5 illustrate examples of inline and passive NIDPS sensor archi-
tecture, respectively.
NBA sensors can most commonly detect the following:
Source: Scarfone and Mell, “Guide to Intrusion Detection and Prevention Systems (IDPS),” NIST SP 800-94, Rev. 1.
Internet
Router
Switch
Firewall Monitoring
interface
Management
interface Management
switch
IDPS sensor
Monitoring
interface
Switch
Internal
network
IDPS IDPS
management console
server
Internet
Source: Scarfone and Mell, “Guide to Intrusion Detection and Prevention Systems (IDPS),” NIST SP 800-94, Rev. 1.
IDPS sensor
Network
tap
Router
IDPS sensor
IDS load
Switch balancer
Firewall
Spanning
port
Switch
Management
IDPS sensor switch
Internal
network
IDPS IDPS
management console
server
NBA sensors offer the following intrusion prevention capabilities, which are grouped by sensor type:
Passive only—A passive NBA sensor can attempt to end an existing TCP session by sending TCP reset packets
to both endpoints.
Inline only—Most inline NBA sensors offer firewall capabilities that can be used to drop or reject suspicious
network activity.
Both passive and inline—Many NBA sensors can instruct network security devices such as firewalls and routers
to reconfigure themselves to block certain types of activity or route it elsewhere, such as to a quarantined vir-
tual local area network (VLAN). Also, some NBA sensors can run an administrator-specified script or program
when certain malicious activity is detected.10
An HIDPS is also capable of monitoring system configuration databases, such as Windows registries, in addition to
stored configuration files like .ini, .cfg, and .dat files. Most HIDPSs work on the principle of configuration management,
which means that they record the sizes, locations, and other attributes of system files. The HIDPS triggers an alert
when file attributes change, new files are created, or existing files are deleted. An HIDPS can also monitor systems logs
for predefined events. The HIDPS examines these files and logs to determine if an attack is under way or has occurred;
it also examines whether the attack is succeeding or was successful. The HIDPS maintains its own log file so that an
audit trail is available even when hackers modify files on the target system to cover their tracks.
Once properly configured, an HIDPS is very reliable. The only time an HIDPS produces a false positive alert is when
an authorized change occurs for a monitored file. This action can be quickly reviewed by an administrator, who may
choose to disregard subsequent changes to the same set of files. If properly configured, an HIDPS can also detect when
users attempt to modify or exceed their access authorization level.
An HIDPS classifies files into various categories and then sends notifications when changes occur. Most HIDPSs
provide only a few general levels of alert notification. For example, an administrator can configure an HIDPS to report
changes in a system folder, such as C:\Program Files\Windows NT, and configure changes to a security-related applica-
tion, such as C:\TripWire. The configuration rules may classify changes to a specific application folder (for example,
C:\Program Files\Microsoft Office) as normal and hence unreportable. Administrators can configure the system to log
all activity but to send them a page or e-mail only if a reportable security event occurs. Because data files and internal
application files such as dictionaries and configuration files are frequently modified, a poorly configured HIDPS can
generate a large volume of false alarms.
Managed HIDPSs can monitor multiple computers simultaneously by creating a configuration file on each moni-
tored host and by making each HIDPS report back to a master console system, which is usually located on the system
administrator’s computer. This master console monitors the information provided by the managed hosts and notifies
the administrator when it senses recognizable attack conditions.
One of the most common methods of categorizing folders and files is by color coding. Critical system components
are coded red and usually include the system registry, any folders containing the OS kernel, and critical system driv-
ers. Critically important data could also be included in the red category. Support components, such as device drivers
and other relatively important files, are generally coded yellow. User data is usually coded green, not because it is
unimportant, but because monitoring changes to user data is practically difficult and strategically less urgent. User
data files are frequently modified, but system kernel files, for example, should only be modified during upgrades or
installations. If the preceding three-color system is too simplistic, an organization can use a scale of 0–10, as long as
the scale doesn’t become excessively granular. For example, an organization could easily create confusion for itself by
using a 100-point scale and then trying to classify level 67 and 68 intrusions. Sometimes simpler is better.
The advantages of HIDPSs include the following:
An HIDPS or one of its sensors can detect local events on host systems and can detect attacks that may elude
a network-based IDPS.
An HIDPS functions on the host system, where encrypted traffic will have been decrypted and is available for
processing.
The use of switched network protocols does not affect an HIDPS.
An HIDPS can detect inconsistencies in how applications and system programs were used by examining the
records stored in audit logs. This can enable the HIDPS to detect some types of attacks, including Trojan horse
programs.11
HIDPSs pose more management issues because they are configured and managed on each monitored host.
An HIDPS requires more management effort to install, configure, and operate than a comparably sized NIDPS
solution.
An HIDPS is vulnerable both to direct attacks and to attacks against the host operating system. Either attack
can result in the compromise or loss of HIDPS functionality.
An HIDPS is not optimized to detect multihost scanning, nor is it able to detect scanning from network devices
that are not hosts, such as routers or switches. Unless complex correlation analysis is provided, the HIDPS
will not be aware of attacks that span multiple devices in the network.
An HIDPS is susceptible to some DoS attacks.
350 Principles of Information Security
An HIDPS can use large amounts of disk space to retain the host OS audit logs; for the HIDPS to function prop-
erly, it may be necessary to add disk capacity to the system.
An HIDPS can inflict a performance overhead on its host systems and may sometimes reduce system perfor-
mance below acceptable levels.12
Note that many of these shortcomings can be mitigated by using distributed HIDPSs that aggregate data from
HIDPSs running on multiple hosts into a centralized management console, allowing comparison of findings across
multiple systems.
baselines. This can prove to be a vulnerability if the attacker uses a very slow attack, because the system using the
dynamic detection method interprets attack activity as normal traffic and updates its profile accordingly.
The advantage of anomaly-based detection is that the IDPS can detect new types of attacks because it looks for
abnormal activity of any type. Unfortunately, these systems require much more overhead and processing capacity than
signature-based IDPSs because they must constantly compare patterns of activity against the baseline. Another drawback
is that these systems may not detect minor changes to system variables and may generate many false positives. If the
actions of network users or systems vary widely, with periods of low activity interspersed with periods of heavy packet
traffic, this type of IDPS may not be suitable because the dramatic swings will almost certainly generate false alarms.
Because of the complexity of anomaly-based detection, its impact on the overhead computing load of the host computer,
and the number of false positives it can generate, this type of IDPS is less commonly used than the signature-based type.
Contextual data
Asset inventory
User information
Vulnerability scans
Threat intelligence
Queries
security information years, organizations and security professionals have sought ways to leverage exist-
and event ing systems and have them work together to maintain situation awareness, identify
management (SIEM) noteworthy issues, and enable response to adverse events.
An information management sys- A SIEM system supports threat detection and informs many aspects of threat
tem specifically tasked to collect
intelligence. It is also instrumental in managing aspects of compliance vulnerability
and correlate events and other
log data from a number of serv- management. It often plays a pivotal role in an organization’s security incident manage-
ers or other network devices for ment through data collection and analysis by enabling near real-time and historical
the purpose of interpreting, filter-
analysis of security events. It integrates data from multiple sources, including local
ing, correlating, analyzing, storing,
reporting, and acting on the result- events and contextual data sources. SIEM systems are derived from legacy log file mon-
ing information. itoring systems and procedures. The basics of SIEM data flows are shown in Figure 9-6.
The core capabilities of SIEM systems leverage the broad scope of log event collec-
tion and management from point sources across the organization, including firewalls,
threat intelligence
endpoint defenses, and IDPS systems. They give the organization the ability to analyze
A process used to develop knowl-
log events and other data from varied sources. The correlated results derived from
edge that allows an organization to
understand the actions and inten- advanced data analysis techniques enable several operational capabilities, including
tions of threat actors and develop dashboards for situational awareness, reporting of trends, and incident management.
methods to prevent or mitigate
SIEM systems have been a mainstay in larger organizations for many years, evolv-
cyberattacks.
ing from locally developed LFM systems whose capabilities have been integrated
into commercially developed platforms. This has allowed organizations with large
numbers of devices used in information systems to collect and act on events from these devices, including servers
and networking equipment, endpoint defense systems, desktop computing systems, and laptops. The reliance on
cloud-based computing solutions has added significant complexity to larger information technology systems, and
current-generation SIEM platforms have evolved to include those environments as well.
As the volume and complexity of data flowing into the SIEM system have multiplied, analysis of that data using
data analytics techniques has been included in SIEM solutions. This has enabled organizations using SIEM solutions
to keep pace as they monitor for intrusions and attacks against a mixed environment of on-premises systems, cloud
hosted services, and hybrid system models that include both enabling technologies. The current level of capability is
made possible by the analytics-driven SIEM system.
Larger organizations are faced with several needs that SIEM platforms can address:
Aggregation of security-related events from across the organization regardless of the source technology
Correlation of events with context from external sources, including vendor-specific updates and cooperative
industry associations
Integration of events from devices, systems, and technologies from disparate sources deployed throughout
the organization
Module 9 Security Technology: Intrusion Detection and Prevention Systems and Other Security Tools 353
These capabilities give the organization the ability to identify and respond to a wide variety of situations.
Real-Time Monitoring
Many successful attacks remain undetected for significant periods of time. Mandiant reports that in 2019, an attack’s
median dwell time—the duration between the start of a cyber intrusion and the time it is detected—was 56 days.16
While this is an improvement over recent years, it is still a significant delay. The longer the dwell times are, the more
likely it is that attackers can be successful in the exfiltration of data and causing other damage. The SolarWinds attack
in 2020 was massive in scope and had a dwell time of months; the attackers showed extraordinary skill in remaining
undetected after compromising systems.
Improvement in an organization’s capability to detect intrusions can reduce the dwell time and allow con-
tainment and recovery to be implemented more quickly. Reducing the time that attackers operate undetected
inside the organization reduces the potential for data loss. A SIEM system that can integrate contextual data
and ongoing events may be able to reduce the time attackers can operate and reduce losses. An example of a
real-time monitoring capability from AT&T Security’s Unified Security Management Anywhere demonstration is
shown in Figure 9-7.
Source: AT&T.
Incident Response
A properly implemented SIEM platform can enable the ability to identify incidents and enable a process to track and
respond to them. SIEM systems enable responders to track and document ongoing incidents and perform record keep-
ing and reporting on the incident response activities. In addition, the system can automatically or manually aggregate
multiple events, interact with other systems, and assist in the collection of evidentiary materials. Some SIEM platforms
can integrate response playbooks that guide staff on how to respond to specific incident categories. In some circum-
stances, a SIEM system can initiate predefined defensive scripts to automatically disrupt ongoing cyberattacks. An
effective SIEM system can be a central service to coordinate the response workflow, combining predefined automatic
response with notifications to enable a coordinated defensive reaction with a unified flow of information to the CSIRT
and the SOC staff.
User Monitoring
An effective SIEM system can analyze user access and authentication activities and provide alerts for suspicious behav-
iors and violation of policy. This is of particular importance for privileged user accounts and is often a requirement
for many compliance reporting functions. To be effective, these functions must be implemented to provide alerts of
reporting and alerts in near real time.
Threat Intelligence
To enable the threat intelligence function of the organization, the SIEM system must be able to integrate threat intel-
ligence services that provide current information on compromise indicators and adversary tactics, techniques, and
procedures (TTP) with knowledge of organizational asset criticality and usage behaviors. These will enable analysts to
recognize abnormal behaviors and assess the risks and impact of various forms of attack. Threat intelligence informa-
tion can then be integrated with data from the IT infrastructure to create watch lists and enable correlation of event
data with the nature of the infrastructure to identify and prioritize threats to organizational assets.
a course of action after analyzing the collected data. A passive IDPS is the more common implementation, although
most systems include some active options that are disabled by default.
The following list describes some of the responses an IDPS can be configured to produce. Note that some of these
responses apply only to a network-based or host-based IDPS, while others are applicable to both.17
Audible/visual alarm—The IDPS can trigger a sound file, beep, whistle, siren, or other audible or visual notifi-
cation to alert the administrator of an attack. The most common type of notification is the computer pop-up,
which can be configured with color indicators and specific messages. The pop-up can also contain specifics
about the suspected attack, the tools used in the attack, the system’s level of confidence in its own determina-
tion, and the addresses and locations of the systems involved.
SNMP traps and plug-ins—The Simple Network Management Protocol contains trap functions, which allow
a device to send a message to the SNMP management console indicating that a certain threshold has been
crossed, either positively or negatively. The IDPS can execute this trap to inform the SNMP console that an
event has occurred. Some advantages of this operation include the relatively standard implementation of
SNMP in networking devices; the ability to configure the network system to use SNMP traps in this manner; the
ability to use systems specifically to handle SNMP traffic, including IDPS traps; and the ability to use standard
communications networks.
E-mail message—The IDPS can send e-mail to notify network administrators of an event. Many administrators
use smartphones and other e-mail devices to check for alerts and other notifications frequently. Organizations
should use caution in relying on e-mail systems as the primary means of communication from an IDPS because
attacks or even routine performance issues can disrupt, delay, or block such messages.
Phone or SMS message—The IDPS can be configured to dial a phone number and deliver a recorded message
or send a preconfigured SMS text message.
Log entry—The IDPS can enter information about the event into an IDPS system log file or operating system
log file. This information includes addresses, times, involved systems, and protocol information. The log files
can be stored on separate servers to prevent skilled attackers from deleting entries about their intrusions.
Evidentiary packet dump—Organizations that require an audit trail of IDPS data may choose to record all log
data in a special way. This method allows the organization to perform further analysis on the data and to
submit the data as evidence in a civil or criminal case. Once the data has been written using a cryptographic
hashing algorithm (see Module 10), it becomes evidentiary documentation—that is, suitable for criminal or
civil court use, as described in Module 5. However, this packet logging can be resource-intensive, especially
during DoS attacks.
Taking action against the intruder—Although it is not advisable, organizations could—with authorization—take
action against an intruder using trap-and-trace, back-hacking, or trace-back methods. Such responses involve
configuring IDPSs to trace the data from the target system back to the attacking system to initiate a counterat-
tack. While this response may sound tempting, it may not be legal. An organization only owns a network to its
perimeter, so conducting traces or back-hacking to systems beyond that point may make the organization just
as criminally liable as the original attackers. Also, the attacking system is sometimes a compromised intermedi-
ary system; in other cases, attackers use address spoofing. In either situation, a counterattack would actually
harm an innocent third party. Any organization that plans to configure retaliation efforts into an automated
IDPS is strongly encouraged to seek legal counsel.
Launching a program—An IDPS can be configured to execute a specific program when it detects specific types
of attacks. Several vendors have specialized tracking, tracing, and response software that can be part of an
organization’s intrusion response strategy.
Reconfiguring a firewall—An IDPS can send a command to the firewall to filter out suspected packets by IP
address, port, or protocol. (Unfortunately, it is still possible for a skilled attacker to break into a network sim-
ply by spoofing a different address, shifting to a different port, or changing the protocols used in the attack.)
While it may not be easy, an IDPS can block or deter intrusions via one of the following methods:
❍ Establishing a block for all traffic from the suspected attacker’s IP address or even from the entire source
network the attacker appears to be using. This blocking can be set for a specific period of time and reset to
normal rules after that period has expired.
❍ Establishing a block for specific TCP or UDP port traffic from the suspected attacker’s address or source
❍ Blocking all traffic to or from the organization’s Internet connection or other network interface if the sever-
ity of the suspected attack warrants such a response.
Terminating the session—Terminating the session by using the TCP/IP protocol packet TCP close is a simple
process. Some attacks would be deterred or blocked by session termination, but others would simply continue
when the attacker issues a new session request.
Terminating the connection—The last resort for an IDPS under attack is to terminate the organization’s internal
or external connections. Smart switches can cut traffic to or from a specific port if the connection is linked
to a system that is malfunctioning or otherwise interfering with efficient network operations. As indicated
earlier, this response should be the last attempt to protect information, as termination may actually be the
goal of the attacker.
Note: The following sections have been adapted from NIST SP 800-94 and SP 800-94, Rev. 1, “Guide to Intrusion
Detection and Prevention Systems,” and their predecessor, NIST SP 800-31, “Intrusion Detection Systems.”
What are the goals of your enterprise? Some IDPSs are designed to accommodate the special needs of cer-
tain industries or market niches, such as electronic commerce, healthcare, or financial services. Define the
functional goals of your enterprise that are supported by your systems. Several goals can be associated with
a single organization.
How formal is the system environment and management culture in your organization? Organizational
styles vary depending on their function and traditional culture. For instance, the military and other
organizations that deal with national security tend to operate with a high degree of formality, especially
when contrasted with universities or other academic environments. Some IDPSs support enforcement
of formal use policies, with built-in configuration options that can enforce common issue-specific or
system-specific security policies. Some IDPSs also provide a library of reports for typical policy viola-
tions or routine matters.
What Are Your Security Goals and Objectives? The next step is to articulate the goals and objectives of using an
IDPS.
Is your organization primarily concerned with protecting itself from outside threats? Perhaps the easiest way
to identify security goals is by categorizing your organization’s threat concerns. Identify its concerns regard-
ing external threats.
Is your organization concerned about insider attacks? Address concerns about threats that originate within
your organization. For example, a shipping clerk might attempt to access and alter the payroll system, or an
authorized user might exceed his privileges and violate your organization’s security policy or laws. As another
example, a customer service agent might be driven by curiosity to access earnings and payroll records for
company executives.
Does your organization want to use the output of your IDPS to determine new needs? System usage monitoring
is sometimes a generic system management tool used to determine when system assets require upgrading or
replacement.
Does your organization want to use an IDPS to maintain managerial control over network usage rather than
security controls? Some organizations implement system use policies that may be classified as personnel
management rather than system security. For example, they might prohibit access to pornographic Web sites
or other sites, or prohibit the use of organizational systems to send harassing e-mail or other messages. Some
IDPSs provide features that detect such violations of management controls.
What Is Your Existing Security Policy? You should review your organization’s existing security policy because it is
the template against which your IDPS will be configured. You may find that you need to augment the policy or derive
the following items from it.
How is it structured? It is helpful to articulate the goals outlined in the security policy. These goals include
standard security goals, such as integrity, confidentiality, and availability, as well as more generic management
goals, such as privacy, protection from liability, and manageability.
What are the general job descriptions of your system users? List the general job functions of system users as well
as the data and network access that each function requires. Several functions are often applied to a single user.
Does the policy include reasonable use policies or other management provisions? As mentioned previously,
the security policies of many organizations include system use policies.
Has your organization defined processes for dealing with specific policy violations? It is helpful to know what
the organization plans to do when the IDPS detects that a policy has been violated. If the organization doesn’t
intend to react to such violations, it may not make sense to configure the IDPS to detect them. On the other
hand, if the organization wants to respond to such violations, IDPS administrators should be informed so they
can deal with alarms in an appropriate manner.
What Are Your Organization’s Resource Constraints? IDPSs can protect the systems of an organization, but at a
price. It makes little sense to incur additional expenses for IDPS features if your organization does not have sufficient
systems or personnel to handle the alerts they will generate.
What is the budget for acquisition and life cycle support of intrusion detection hardware, software, and
infrastructure? Remember that the IDPS software is not the only element of the total cost of ownership; you
may also have to acquire a system for running the software, obtain specialized assistance to install and con-
figure the system, and train your personnel. Ongoing operations may also require additional staff or outside
contractors.
Is there sufficient staff to monitor an IDPS full time? Some IDPSs require around-the-clock attendance by sys-
tems personnel. If your organization cannot meet this requirement, you may want to explore systems that
accommodate part-time attendance or unattended use.
Does your organization provide authority to instigate changes based on the findings of an IDPS? The organiza-
tion must be clear about how to address problems uncovered by an IDPS. If you are not empowered to handle
incidents that arise as a result of monitoring, you should coordinate your selection and configuration of the
IDPS with the person who is empowered.
Is the Product Sufficiently Scalable for Your Environment? Many IDPSs cannot function within large or widely
distributed enterprise network environments.
How Has the Product Been Tested? Simply asserting that an IDPS has certain capabilities does not demonstrate
they are real. You should request demonstrations of an IDPS to evaluate its suitability for your environment and
goals.
Has the product been tested against functional requirements? Ask the vendor about any assumptions made
for the goals and constraints of customer environments.
Has the product been tested for performance against anticipated load? Ask vendors for details about their
products’ ability to perform critical functions with high reliability under load conditions similar to those
expected in the production environment.
Has the product been tested to reliably detect attacks? Ask vendors for details about their products’ ability
to respond to attacks reliably.
Module 9 Security Technology: Intrusion Detection and Prevention Systems and Other Security Tools 359
Has the product been tested against attack? Ask vendors for details about their products’ security testing. If
the product includes network-based vulnerability assessment, ask whether test routines that produce system
crashes or other denials of service have been identified and flagged in system documentation and interfaces.
What User Level of Expertise Is Targeted by the Product? Different IDPS vendors target users with different
levels of technical and security expertise. Ask vendors to describe their assumptions about users and administrators
of their products.
Is the Product Designed to Evolve as the Organization Grows? An important goal of product design is the ability
to adapt to your needs over time.
Can the product adapt to growth in user expertise? Ask whether the IDPS’s interface can be configured on
the fly to accommodate shortcut keys, customizable alarm features, and custom signatures. Ask also whether
these features are documented and supported.
Can the product adapt to growth and change of the organization’s system infrastructure? This question
addresses the ability of the IDPS to scale to an expanding and increasingly diverse network. Most vendors
have experience in adapting their products as target networks grow. Ask also about commitments to support
new protocol standards and platform types.
Can the product adapt to growth and change in the security threat environment? This question is especially
critical given the current Internet threat environment, in which hundreds of new malware variants are posted
to the Web every day.
What Are the Support Provisions for the Product? Like other systems, IDPSs require maintenance and support
over time. These needs should be identified in a written report.
What are the commitments for product installation and configuration support? Many vendors provide expert
assistance to customers when installing and configuring IDPSs. Other vendors expect your own staff to handle
such functions and provide only telephone or e-mail support.
What are the commitments for ongoing product support? Ask about the vendor’s commitment to supporting
your use of its IDPS product.
Are subscriptions to signature updates included? Most IDPSs are misuse detectors, so their value is only as
good as the signature database against which events are analyzed. Most vendors provide subscriptions to
signature updates for a year or other specified period of time.
How often are subscriptions updated? In today’s threat environment, in which 30 to 40 new attacks are pub-
lished every month, this is a critical question.
After a new attack is made public, how quickly will the vendor ship a new signature? If you are using IDPSs to
protect highly visible or heavily traveled Internet sites, it is critical that you receive the signatures for new
attacks as soon as possible.
Are software updates included? Most IDPSs are software products and therefore subject to bugs and revi-
sions. Ask the vendor about its support for software updates and bug patches, especially for the product you
purchase.
How quickly will software updates and patches be issued after a problem is reported to the vendor? Software
bugs in IDPSs can allow attackers to nullify their protective effect, so any problems must be fixed reliably and
quickly.
Are technical support services included? What is the cost? In this category, technical support services mean
vendor assistance in tuning or adapting your IDPS to accommodate special needs. These needs might include
monitoring a custom or legacy system within your enterprise or reporting IDPS results in a custom protocol
or format. This also includes support in resolving installation or configuration issues.
What are the provisions for contacting technical support via e-mail, telephone, online chats, or Web-based
reporting? The contact provisions will probably tell you whether technical support services are accessible
enough to support incident handling or other time-sensitive needs.
360 Principles of Information Security
Are any guarantees associated with the IDPS? As with other software products, IDPSs traditionally come
with few guarantees; however, in an attempt to gain market share, some vendors are initiating guarantee
programs.
What training resources does the vendor provide? Once an IDPS is selected, installed, and configured, your
personnel can operate it, but they need to be trained in its use. Some vendors provide this training as part of
the product package.
What additional training resources are available from the vendor and at what cost? If the vendor does not
provide training as part of the IDPS package, you should budget appropriately to train your personnel.
Strengths of IDPSs
Intrusion detection and prevention systems perform the following functions well:
Limitations of IDPSs
IDPSs cannot perform the following functions:
Compensating for weak or missing security mechanisms in the protection infrastructure, such as firewalls,
identification and authentication systems, link encryption systems, access control mechanisms, and virus
detection and eradication software
Instantaneously detecting, reporting, and responding to an attack when there is a heavy network or process-
ing load
Detecting newly published attacks or variants of existing attacks
Effectively responding to attacks launched by sophisticated attackers
Automatically investigating attacks without human intervention
Resisting all attacks that are intended to defeat or circumvent them
Compensating for problems with the fidelity of information sources
Dealing effectively with switched networks
There is also the considerable challenge of configuring an IDPS to respond accurately to a perceived threat. Once
a device is empowered to react to an intrusion by filtering or even severing a communication session or by severing
a communication circuit, the impact from a false positive becomes significant. It’s one thing to fill an administrator’s
e-mail box or compile a large log file with suspected attacks; it’s quite another to shut down critical communications.
Some forms of attacks, conducted by attackers called IDPS terrorists, are designed to trip the organization’s IDPS,
essentially causing the organization to conduct its own DoS attack by overreacting to an actual but insignificant
attack.
Note: The preceding sections were developed and adapted from NIST SP 800-94 and SP 800-94, Rev. 1, “Guide to
Intrusion Detection and Prevention Systems,” and their predecessor, NIST SP 800-31, “Intrusion Detection Systems.”
Module 9 Security Technology: Intrusion Detection and Prevention Systems and Other Security Tools 361
Organizations should ensure that all IDPS components are secured appropriately, as IDPSs are a prime target for
attackers. If they can compromise the IDPS, they are then free to conduct unobserved attacks on other systems.
Organizations should consider using multiple types of IDPS technologies to achieve more comprehensive and
accurate detection and prevention of malicious activity. Defense in depth, even within IDPS technologies, is
key to detecting the wide and varied attack strategies the organization faces.
Organizations that plan to use multiple types of IDPS technologies or multiple products of the same IDPS tech-
nology type should consider whether the IDPSs should be integrated. Using integrated technologies provides
much easier configuration and administration. Using a common management platform provides multidevice
cross-assessments and reporting.
Before evaluating IDPS products, organizations should define the requirements that the products should meet.
Knowing what you need in an IDPS can prevent purchasing a product that does not solve the organization’s
problems, and can save time and money in the long haul.
When evaluating IDPS products, organizations should consider using a combination of data sources to evalu-
ate the products’ characteristics and capabilities. Vendors have been known to “influence” reviews by using
friendly reviewers or just writing their own and posting them to review sites. Finding multiple reviews from
different sources should provide more accurate insight into the strengths and weaknesses of any technology.18
IDS Console
Internet
Network Devices
Switch Firewall
VOIP Device
IDS Reporting Links Monitoring Links IDS Response Links Main Network Links
fully distributed IDPS Fully Distributed Control Strategy A fully distributed IDPS control strategy,
control strategy illustrated in Figure 9-9, is the opposite of the centralized strategy. All control func-
An IDPS implementation approach tions are applied at the physical location of each IDPS component; in the figure, these
in which all control functions are functions are represented as small square symbols enclosing a computer icon. Each
applied at the physical location of
each IDPS component. monitoring site uses its own paired sensors to perform its own control functions
and achieve the necessary detection, reaction, and response. Thus, each sensor/
partially distributed agent is best configured to deal with its own environment. Because the IDPSs do not
IDPS control strategy have to wait for a response from a centralized control facility, their response time to
An IDPS implementation approach individual attacks is greatly enhanced.
that combines the best aspects of
Partially Distributed Control Strategy A partially distributed IDPS control
the centralized and fully distributed
strategies. strategy, depicted in Figure 9-10, combines the best aspects of the other two strat-
egies. While the individual agents can still analyze and respond to local threats,
their reporting to a hierarchical central facility enables the organization to detect widespread attacks. This blended
approach to reporting is one of the more effective methods of detecting intelligent attackers, especially those who
probe an organization at multiple points of entry, trying to identify the systems’ configurations and weaknesses before
launching a concerted attack. The partially distributed control strategy also allows the organization to optimize for
economy of scale in the implementation of key management software and personnel, especially in the reporting areas.
When the organization can create a pool of security managers to evaluate reports from multiple distributed IDPS sys-
tems, it becomes more capable of detecting distributed attacks before they become unmanageable.
Module 9 Security Technology: Intrusion Detection and Prevention Systems and Other Security Tools 363
Network Devices
Switch Firewall
VOIP Device
IDPS Deployment
Given the highly technical skills required to implement and configure IDPSs and the imperfection of the technology,
great care must be taken when deciding where to locate the components, both in their physical connection to the
network and host devices and in how they are logically connected to each other. Because IDPSs are designed to detect,
report, and even react to anomalous stimuli, placing IDPSs in an area where such traffic is common can result in exces-
sive reporting. Moreover, locating the administrators’ monitoring systems in such areas can desensitize them to the
information flow and cause them to miss actual attacks in progress.
As an organization selects an IDPS and prepares for implementation, planners must select a deployment strategy
that is based on a careful analysis of the organization’s information security requirements and that integrates with
the existing IT infrastructure while causing minimal impact. After all, the purpose of the IDPS is to detect anomalous
situations, not create them. One consideration is the skill level of the personnel who install, configure, and maintain
the systems. An IDPS is a complex system in that it involves numerous remote monitoring agents both on individual
systems and networks, which require proper configuration to gain the needed authentication and authorization. As
the IDPS is deployed, each component should be installed, configured, fine-tuned, tested, and monitored. A mistake in
any step of the deployment process may produce a range of problems—from a minor inconvenience to a network-wide
disaster. Thus, the people who install the IDPS and the people who use and manage the system require proper training.
NIDPSs and HIDPSs can be used in tandem to cover the individual systems that connect to an organization’s
networks and the networks themselves. It is important to use a phased implementation strategy so the entire organi-
zation isn’t affected at once. A phased implementation strategy also allows security technicians to resolve problems
that arise without compromising the very information security the IDPS is installed to protect. When sequencing the
364 Principles of Information Security
Subnet
IDS Enterprise
Console IDS Console
Network Devices
Switch Firewall
VOIP Device
IDS Reporting Links Monitoring Links IDS Response Links Main Network Links
implementation, the organization should first implement the NIDPSs, as they are easier to configure than their host-
based counterparts. After the NIDPSs are configured and running without problems, the HIDPSs can be installed to
protect the critical systems on the host server. Once the NIDPSs and HIDPSs are working, administrators should scan
the network with a vulnerability scanner like Nmap or Nessus to determine if it picks up anything new or unusual and
to see if the IDPS can detect the scans.
Deploying Network-Based IDPSs The placement of the sensor agents is critical to the operation of all IDPSs, but
especially for NIDPSs. NIST recommends the following four locations for NIDPS sensors, as illustrated in Figure 9-11.
Location 1 is behind each external firewall, in the network DMZ. This location has the following characteristics:
The IDPS sees attacks that originate from the outside and may penetrate the network’s perimeter defenses.
The IDPS can identify problems with the network firewall’s policy or performance.
The IDPS sees attacks that might target the Web server or FTP server, both of which commonly reside in this
DMZ.
Even if the incoming attack is not detected, the IDPS can sometimes recognize patterns in the outgoing traffic
that suggest the server has been compromised.
Location 2 is outside an external firewall. This location has the following characteristics:
The IDPS documents the number of attacks originating on the Internet that target the network.
The IDPS documents the types of attacks originating on the Internet that target the network.
Location 3 is on major network backbones. This location has the following characteristics:
The IDPS monitors a large amount of a network’s traffic, thus increasing its chances of spotting attacks.
The IDPS detects unauthorized activity by authorized users within the organization’s security perimeter.
Module 9 Security Technology: Intrusion Detection and Prevention Systems and Other Security Tools 365
Internet
Switch Firewall
Location 2
Web Server Network Storage
Location 4
Location 1 VOIP Device
The IDPS detects attacks that target critical systems and resources.
This location allows organizations with limited resources to focus on the most valuable network assets.23
Deploying Host-Based IDPSs The proper implementation of HIDPSs can be a painstaking and time-consuming task,
as each HIDPS must be customized to its host systems. Deployment begins with implementing the most critical systems
first. This poses a dilemma for the deployment team because the first systems to be implemented are mission-critical,
and any problems with the installation could be catastrophic to the organization. Thus, it may be beneficial to practice
an implementation on one or more test servers configured on a network segment that resembles the mission-critical
systems. Practice helps the installation team gain experience and helps determine if the installation might trigger
any unusual events. Gaining an edge on the learning curve by training on nonproduction systems benefits the overall
deployment process by reducing the risk of unforeseen complications.
Installation continues until all systems are installed or the organization reaches the planned degree of coverage
it will accept, in terms of the number of systems or percentage of network traffic. To provide ease of management,
control, and reporting, each HIDPS should be configured to interact with a central management console.
Just as technicians can install the HIDPS in offline systems to develop expertise and identify potential problems, users
and managers can learn about the operation of the HIDPS by using a test facility. This facility could use the offline systems
configured by technicians and be connected to the organization’s backbone to allow the HIDPS to process actual network
traffic. This setup will also enable technicians to create a baseline of normal traffic for the organization. During system
testing, training scenarios can be developed that enable users to recognize and respond to common attacks. To ensure
effective and efficient operation, the management team can establish policy for the operation and monitoring of the HIDPS.
whitelist Blacklists, also known as hot lists, typically allow IDPSs to block activity that is
A list of systems, users, files, or highly likely to be malicious, and may also be used to assign a higher priority to
addresses that are known to be alerts that match blacklist entries. Some IDPSs generate dynamic blacklists that are
benign; it is commonly used to
used to temporarily block recently detected threats (e.g., activity from an attack-
expedite access to systems or
networks. er’s IP address). A whitelist is a list of discrete entities that are known to be benign.
Whitelists are typically used on a granular basis, such as protocol by protocol, to
reduce or ignore false positives involving known benign activity from trusted hosts.
Whitelists and blacklists are most commonly used in signature-based detection and stateful protocol analysis.
Alert settings—Most IDPS technologies allow administrators to customize each alert type. Examples of actions
that can be performed on an alert type include the following:
❍ Toggling it on or off
❍ Specifying what information should be recorded and what notification methods (e.g., e-mail, pager) should be used
❍ Specifying which prevention capabilities should be used. Some products also suppress alerts if an attacker
generates many alerts in a short period of time and may also temporarily ignore traffic from the attacker. This
is to prevent the IDPS from being overwhelmed by alerts.
Code viewing and editing—Some IDPS technologies permit administrators to see some or all of the detection-
related code. This is usually limited to signatures, but some technologies allow administrators to see additional
code, such as programs used to perform stateful protocol analysis.24
Once implemented, IDPSs are evaluated using two dominant metrics. First, administrators evaluate the number
of attacks detected in a known collection of probes. Second, the administrators examine the level of use at which the
IDPSs fail; this level is commonly measured in megabits per second of network traffic. An evaluation of an IDPS might
read something like this: At 1 Gbps, the IDPS was able to detect 99.84 percent of directed attacks. This is a dramatic change
from the previous method used for assessing IDPS effectiveness, which was based on the total number of signatures
the system was currently running—somewhat of a “more is better” approach. This evaluation method was flawed for
several reasons. Not all IDPSs use simple signature-based detection, and some systems use the almost infinite combi-
nation of network performance characteristics in anomaly-based detection to identify a potential attack. Also, some
sophisticated signature-based systems actually use fewer signatures or rules than older, simpler versions—in direct
contrast to traditional signature-based assessments, these systems suggest that less may actually be more. Recogniz-
ing that the size of the signature base is an insufficient measure of an IDPS’s effectiveness led to the development of
stress test measurements for evaluating its performance. These measurements only work, however, if the administra-
tor has a collection of known negative and positive actions that can be proven to elicit a desired response. Because
developing this collection can be tedious, most IDPS vendors provide testing mechanisms to verify that their systems
are performing as expected. Some of these testing processes enable the administrator to do the following:
Record and retransmit packets from a real virus or worm scan.
Record and retransmit packets from a real virus or worm scan with incomplete TCP/IP session connections
(missing SYN packets).
Conduct a real virus or worm attack against a hardened or sacrificial system.
This last measure is important because future IDPSs will probably include much more detailed information about
the overall site configuration. According to an expert in the field, “It may be necessary for the IDPSs to be able to
actively probe a potentially vulnerable machine, in order to either preload its configuration with correct information or
perform a retroactive assessment. An IDPS that performed some kind of actual system assessment would be a complete
failure in today’s generic testing labs, which focus on replaying attacks and scans against nonexistent machines.”25
With the rapid growth in technology, each new generation of IDPSs will require new testing methodologies. However,
the measured values that continue to hold interest for IDPS administrators and managers will most certainly include
some assessment of how much traffic the IDPS can handle, the numbers of false positives and false negatives it gener-
ates, and a measure of the IDPS’s ability to detect actual attacks. Vendors of IDPS systems could also include a report
of alarms sent and the relative accuracy of the system in correctly matching the alarm level to the true seriousness
of the threat. Some planned metrics for IDPSs include the flexibility of signatures and detection policy customization.
IDPS administrators may soon be able to purchase tools that test IDPS effectiveness. Until these tools are available
from a neutral third party, however, the diagnostics from IDPS vendors will always be suspect. No vendor, no matter
how reliable, would provide a test that its system would fail.
Module 9 Security Technology: Intrusion Detection and Prevention Systems and Other Security Tools 367
One note of caution: There is a strong tendency among IDPS administrators to use common vulnerability assess-
ment tools, such as Nmap or Nessus, to evaluate the capabilities of an IDPS. While this may seem like a good idea, the
tools will not work as expected because most IDPS systems are equipped to recognize the differences between a locally
implemented vulnerability assessment tool and a true attack.
To accurately assess the effectiveness of IDPS systems, the testing process should be as realistic as possible in
its simulation of an actual event. This means coupling realistic traffic loads with realistic levels of attacks. You cannot
expect an IDPS to respond to a few packet probes as if they represent a DoS attack. In one reported example, a program
was used to create a synthetic load of network traffic made up of many TCP sessions, with each session consisting of a
SYN (synchronization) packet, a series of data, and ACK (acknowledgment) packets, but no FIN or connection termina-
tion packets. Of the several IDPS systems tested, one of them crashed due to lack of resources while it waited for the
sessions to be closed. Another IDPS passed the test with flying colors because it did not perform state tracking on the
connections. Neither of the tested IDPS systems worked as expected, but the one that didn’t perform state tracking
was able to remain working and therefore received a better score on the test.26
to monitor the actions of an attacker. IDPS researchers have used padded cell and honeypot systems since the late
1980s, but until recently, no commercial versions of these products were available. It is important to seek guidance
from legal counsel before using either of these systems in your operating environment—using an attractant and then
launching a back hack or counterstrike might be illegal in some areas, and could make the organization vulnerable to
a lawsuit or criminal complaint.
The advantages and disadvantages of using honeypots or the padded cell approach are summarized in Table 9-1.
Trap-and-Trace Systems
Trap-and-trace applications, which are an extension of the attractant technologies
trap-and-trace discussed in the previous section, are still in use. These systems use a combination
application of techniques to detect an intrusion and then trace it back to its source. The trap
An application that combines the
usually consists of a honeypot or padded cell and an alarm. While the intruders are
function of honeypots or honeynets
with the capability to track the distracted (or trapped) by what they perceive to be successful intrusions, the system
attacker back through the network. notifies the administrator of their presence. The trace feature is an extension of the
Table 9-1 Advantages and disadvantages of using honeypots or padded cell systems27
Advantages Disadvantages
Attackers can be diverted to targets that they cannot The legal implications of using such devices are not well
damage. understood.
Administrators have time to decide how to respond to an Honeypots and padded cells have not yet been shown to
attacker. be generally useful security technologies.
Attackers’ actions can be easily and more extensively An expert attacker, once diverted into a decoy system,
monitored, and the records can be used to refine threat may become angry and launch a more aggressive attack
models and improve system protections. against an organization’s systems.
Honeypots may be effective at catching insiders who are Administrators and security managers need a high level
snooping around a network. of expertise to use these systems.
Module 9 Security Technology: Intrusion Detection and Prevention Systems and Other Security Tools 369
honeypot or padded cell approach. The trace—which is similar to caller ID—is a process by which the organization
attempts to identify an entity discovered in unauthorized areas of the network or systems. If the intruder is someone
inside the organization, administrators are completely within their power to track the person and turn the case over
to internal or external authorities. If the intruder is outside the organization’s security perimeter, numerous legal
issues arise. Products in this genre were initially popular in the early 2000s but quickly ran into legal issues when used
outside the boundaries of the organization. No commercial products have taken their place due to the drawbacks and
complications of using these technologies. However, similar applications are used by common carriers in conjunction
with law enforcement to allow legal tracing of computer communications as part of investigations.
Trap-and-trace systems are similar to pen registers, earlier versions of which
recorded numbers that were dialed in voice communications. Older pen registers pen register
were much like key loggers for phones, but more current models and trap-and-trace An application that records
systems are used in data networks as well as voice communications networks. information about outbound
communications.
According to the Electronic Frontier Foundation, trap-and-trace systems record attri-
butes of inbound communications such as phone numbers or IP addresses, while pen registers are used frequently in
law enforcement and antiterrorism operations to record attributes of outbound communications.28
On the surface, trap-and-trace systems seem like an ideal solution. Security is no longer limited to defense; secu-
rity administrators can now go on the offensive to track down perpetrators and turn them over to the appropriate
authorities. Under the guise of justice, less scrupulous administrators may even be tempted to back hack, or break
into a hacker’s system to find out as much as possible about the hacker. Vigilante
justice would be an appropriate term for these activities, which are deemed unethi- back hack
cal by most codes of professional conduct. In tracking the hacker, administrators The process of illegally attempting
may end up wandering through other organizations’ systems, especially if a more to determine the source of an intru-
sion by tracing it and trying to gain
wily hacker has used IP spoofing, compromised systems, or other techniques to
access to the originating system.
throw trackers off the trail. In other words, the back-hacking administrator becomes
the hacker.
Trap-and-trace systems and pen registers are covered under Title 18, U.S. Code Module 206, §3121, which essen-
tially states that you can’t use them unless you’re a service provider attempting to prevent misuse and (1) they are
used for systems maintenance and testing, (2) they are used to track connections, or (3) you have permission from
the user of the service.29
There are more legal drawbacks to trap and trace. The trap portion frequently
involves honeypots or honeynets; when using them, administrators should be care-
enticement
ful not to cross the line between enticement and entrapment. Enticement is legal
The act of attracting attention to a
and ethical, but entrapment is not. It is difficult to gauge the effect of such a system system by placing tantalizing infor-
on average users, especially if they have been nudged into looking at the informa- mation in key locations.
tion in the system. Administrators should also be wary of the wasp trap syndrome,
which describes a homeowner who installs a wasp trap in his backyard to trap the entrapment
insects he sees flying about. Because these traps use scented bait, however, they The act of luring a person into com-
wind up attracting far more wasps than were originally present. Security administra- mitting a crime in order to get a
tors should keep the wasp trap syndrome in mind before implementing honeypots, conviction.
unrestricted internal link into the company’s corporate data center. The attacker can attack the weaker site at the
remote facility and use the compromised internal network to attack the true target. While it may seem trite or cli-
chéd, the old saying that “a chain is only as strong as its weakest link” is very relevant to network and computer
security. If a company has a trusted network connection with 15 business partners, one weak business partner can
compromise all 16 networks.
To assist in footprint intelligence collection, you can use an enhanced Web scanner that examines entire Web
sites for valuable pieces of information, such as server names and e-mail addresses. One such scanner is called Sam
Spade (see Figure 9-13), which you can obtain by searching the Web for a copy of the last version (1.14). Although
antiquated, Sam Spade can perform a host of scans and probes, such as sending multiple ICMP information requests
(pings), attempting to retrieve multiple and cross-zoned DNS queries, and performing network analysis queries known
as traceroutes from the commonly used UNIX command. All of these scans are powerful diagnostic and hacking activi-
ties, but Sam Spade is not considered hackerware (hacker-oriented software). Rather, it is a utility that is useful to
network administrators and miscreants alike.
For Linux or BSD systems, a tool called GNU Wget allows a remote user to “mirror” entire Web sites. With this tool,
attackers can copy an entire Web site and then go through the source HTML, JavaScript, and Web-based forms at their
leisure, collecting and collating all of the data from the source code that will help them mount an attack.
The next phase of the attack protocol is a data-gathering process called
fingerprinting. Fingerprinting deploys various tools that are described in the fol- fingerprinting
lowing sections to reveal useful information about the internal structure and nature The systematic survey of a targeted
of the target system or network to be attacked. These tools were created to find organization’s Internet addresses
collected during the footprinting
vulnerabilities in systems and networks quickly and with a minimum of effort. They
phase to identify the network ser-
are valuable to the network defender because they can quickly pinpoint parts of the vices offered by the hosts in that
systems or network that need prompt repair to close vulnerabilities. range.
372 Principles of Information Security
A port is a network channel or connection point in a data communications system. Within the TCP/IP networking
protocol, TCP and User Datagram Protocol (UDP) port numbers differentiate the multiple communication channels
that connect to the network services offered on a network device. Each application within TCP/IP has a unique port
number. Some have default ports but can also use other ports. Some of the well-known port numbers are shown in
Table 9-2. In all, 65,536 port numbers are in use for TCP and another 65,536 port numbers are used for UDP. Services
that use the TCP/IP protocol can run on any port; however, services with reserved ports generally run on ports 1–1023.
Port 0 is not used. Port numbers greater than 1023 are typically referred to as ephemeral ports and may be randomly
allocated to server and client processes.
Why secure open ports? Simply put, an attacker can use an open port to send commands to a computer, potentially
gain access to a server, and possibly exert control over a networking device. As a rule of thumb, any port that is not
absolutely necessary for conducting business should be secured or removed from service. For example, if a business
doesn’t host Web services, there is no need for port 80 to be available on its servers.
The number and nature of the open ports on a system are an important part of
attack surface its attack surface. As a general design goal, security practitioners seek to reduce
The functions and features that a the attack surface of each system to minimize the potential for latent defects and
system exposes to unauthenticated unintended consequences to cause losses.
users.
At this point, we must caution that some activities performed routinely by secu-
rity professionals—specifically, port scanning—may cause problems for casual sys-
tem users. Even the use of the network ping command can cause issues at some organizations. Some organizations
have strong policy prohibitions for activities that test network security. Many endpoint protection products trigger
alarms when these activities are detected. Always ask permission from the organization’s security office before “test-
ing” network security.
For more information on Firewalk, read Goldsmith and Schiffman’s article at http://packetfactory.openwall.net/
i projects/firewalk/firewalk-final.pdf. Firewalk can be obtained from https://packetstormsecurity.com/UNIX/audit/
firewalk.
A final firewall analysis tool worth consideration is HPING (www.hping.org), which is a modified ping client. It sup-
ports multiple protocols and has a command-line method of specifying nearly any ping parameter. For instance, you
can use HPING with modified TTL values to determine the infrastructure of a DMZ. You can use HPING with specific
ICMP flags to bypass poorly configured firewalls that allow all ICMP traffic to pass through and find internal systems.
Administrators who are wary of using the same tools that attackers use should remember two important points.
Regardless of the tool that is used to validate or analyze a firewall’s configuration, user intent dictates how the gath-
ered information is used. To defend a computer or network well, administrators must understand the ways it can be
attacked. Thus, a tool that can help close an open or poorly configured firewall will help the network defender minimize
the risk from attack.
IP stack with its own internal database of known responses. Because most OSs have a unique way of responding to ICMP
requests, XProbe is very reliable in finding matches and thus detecting the operating systems of remote computers.
Therefore, system and network administrators should restrict the use of ICMP through their organization’s firewalls
and, when possible, within their internal networks.
active vulnerability
scanner
Vulnerability Scanners
Active vulnerability scanners examine networks for highly detailed information. An
An application that scans networks
to identify exposed usernames and active scanner is one that initiates traffic on the network to determine security holes.
groups, open network shares, con- An example of a vulnerability scanner is Nessus, a professional freeware utility that
figuration problems, and other vul-
uses IP packets to identify hosts available on the network, the services (ports) they
nerabilities in servers.
offer, their operating system and OS version, the type of packet filters and firewalls
in use, and dozens of other network characteristics. Figures 9-14 and 9-15 show sample screens from Nessus.
Vulnerability scanners should be proficient at finding known, documented holes, but what happens if a Web server
is from a new vendor or a new application was created by an internal development team? In such cases, you should
consider using a class of vulnerability scanners called black-box scanners or fuzzers. Fuzz testing is a straightforward
technique that looks for vulnerabilities in a program or protocol by feeding random input to the program or a network
running the protocol. Vulnerabilities can be detected by measuring the outcome of the random inputs. One example
of a fuzz scanner is Spike, which has two primary components. The first is the Spike Proxy (www.spikeproxy.com),
which is a full-blown proxy server. As Web site visitors use the proxy, Spike builds a database of each traversed page,
form, and other Web-specific asset. When the Web site owner determines that enough history has been collected to
completely characterize the full site, Spike can be used to check for bugs. In other words, administrators can use
the usage history collected by Spike to traverse all known pages, forms, and active programs such as asp and cgibin,
and then can test the system by attempting overflows, SQL injection, cross-site scripting, and many other classes of
Web attacks.
Source: Nessus.
Source: Nessus.
Figure 9-15 Tenable’s Nessus: detail
A list of the top commercial and residential vulnerability scanners includes the following products:30
Nessus
OpenVAS
Core Impact
Nexpose
GFI LanGuard
QualysGuard
Microsoft Baseline Security Analyzer (MBSA)
Retina
Secunia PSI
Nipper
Security Administrator’s Integrated Network Tool (SAINT)
The Nessus scanner features a class of attacks called destructive attacks. If enabled, Nessus attempts common
overflow techniques against a target host. Fuzzers or black-box scanners and Nessus in destructive mode can be very
dangerous tools, so they should be used only in a lab environment. In fact, these tools are so powerful that even expe-
rienced system defenders are not likely to use them in the most aggressive modes on their production networks. At the
time of this writing, the most popular scanners seem to be Nessus, OpenVAS, and Nexpose. The Nessus scanner was
originally open source, but it is now strictly commercial. OpenVAS was created as a variant from the last free version
of Nessus and is therefore a good open-source alternative. Nexpose offers free and commercial versions.
Members of an organization often require proof that a system is vulnerable to a certain attack. They may require
such proof to avoid having system administrators attempt to repair systems that are actually not broken or because
they have not yet built a satisfactory relationship with the vulnerability assessment team. In these instances, a class
of scanners is available that actually exploits the remote machine and allows the vulnerability analyst (sometimes
called a penetration tester) to create an account, modify a Web page, or view data. These tools can be very dangerous
and should be used only when absolutely necessary. Three such tools are Core Impact, Immunity’s CANVAS, and the
Metasploit Framework.
Of these three tools, only the Metasploit Framework is available without a license fee. The Metasploit Framework
is a collection of exploits coupled with an interface that allows penetration testers to automate the custom exploitation
376 Principles of Information Security
of vulnerable systems. For instance, if you wanted to exploit a Microsoft Exchange server and run a single command
(perhaps add the user “security” into the administrators group), the tool allows you to customize an overflow in this
manner. Figure 9-16 shows the Metasploit Framework.
passive vulnerability A passive vulnerability scanner listens in on the network and identifies vulner-
scanner able versions of both server and client software. At the time of this writing, two pri-
A scanner that listens in on a net- mary vendors offer this type of scanning solution: Tenable Network Security, with its
work and identifies vulnerable Passive Vulnerability Scanner (PVS), and Watcher Web Security Scanner from Casaba
versions of both server and client
software. (see Figure 9-17). The advantage of using passive scanners is that they do not require
Source: Casaba Security, LLC.
vulnerability analysts to obtain approval prior to testing. These tools simply monitor the network connections to and
from a server to obtain a list of vulnerable applications. Furthermore, passive vulnerability scanners can find client-
side vulnerabilities that are typically not found by active scanners. For instance, an active scanner operating without
domain admin rights would be unable to determine the version of Internet Explorer running on a desktop machine,
but a passive scanner could make that determination by observing traffic to and from the client.
Figures 9-18 and 9-19 show Tenable’s PVS.
Packet Sniffers
packet sniffer
A packet sniffer or network protocol analyzer can provide a network administra-
A software program or hardware
tor with valuable information for diagnosing and resolving networking issues. In appliance that can intercept, copy,
the wrong hands, however, a sniffer can be used to eavesdrop on network traffic. and interpret network traffic.
378 Principles of Information Security
Source: Wireshark.
Figure 9-20 Wireshark
Commercial and open-source sniffers are both available—for example, Sniffer is a commercial product and Snort is
open-source software. The dominant network protocol analyzer is Wireshark (www.wireshark.org), formerly known as
Ethereal, which is available in open-source and commercial versions. Wireshark allows the administrator to examine
data both from live network traffic and captured traffic. Wireshark’s features include a language filter and a TCP ses-
sion reconstruction utility. Figure 9-20 shows a sample screen from Wireshark. To use these types of programs most
effectively, the user must be connected to a network from a central location using a monitoring port. Simply tapping
into an Internet connection floods you with more data than you can readily process, and the action technically consti-
tutes a violation of the U.S. Wiretap Act.
To use a packet sniffer legally, the administrator must (1) be on a network that the organization owns, (2) have
authorization of the network’s owners, and (3) have knowledge and consent of the content creators. If all three con-
ditions are met, the administrator can selectively collect and analyze packets to identify and diagnose problems on
the network. Consent is usually obtained by having all system users sign a release when they are issued a user ID
and passwords; the release states that “use of the systems is subject to monitoring.” These three conditions are the
same requirements for employee monitoring in general; therefore, packet sniffing should be construed as a form of
employee monitoring.
Many administrators feel safe from sniffer attacks when their computing environment is primarily a switched
network, but they couldn’t be more wrong. Several open-source sniffers support alternate networking approaches
and can enable packet sniffing in a switched network environment. Two of these approaches are ARP spoofing and
session hijacking, which use tools like Ettercap (www.ettercap-project.org/). To secure data in transit across any net-
work, organizations must use a carefully designed and implemented encryption solution to ensure uncompromised
content privacy.
and assess the level of privacy or confidentiality afforded on the wireless network. Sectools.org identified the top wire-
less tools in current use:
Aircrack, a wireless network protocol cracking tool
Kismet, a powerful wireless network protocol sniffer, network detector, and IDPS, which works by passively
sniffing networks
NetStumbler, a freeware Windows file parser available at www.netstumbler.org
inSSIDer, an enhanced scanner for Windows, OS X, and Android
KisMAC, a GUI passive wireless stumbler for Mac OS X (a variation of Kismet)31
Another wireless tool, AirSnare (https://airsnare.en.softonic.com/), is freeware that can be run on a low-end wire-
less workstation. AirSnare monitors the airwaves for any new devices or access points. When it finds one, AirSnare
sounds an alarm to alert administrators that a new and potentially dangerous wireless apparatus is attempting access
on a closed wireless network.
The tools discussed in this module help the attacker and the defender prepare themselves to complete the next
steps in the attack protocol: attack, compromise, and exploit. These steps are beyond the scope of this text and are
usually covered in more advanced classes on computer and network attack and defense.
For more information on these and other security tools, visit Insecure.org’s security Web site at sectools.org. For
i more information on the site’s owner and the author of the Nmap tool, visit Fyodor’s page at insecure.org/fyodor/.
Closing Scenario
Miller Harrison was still working his way through his attack protocol.
Nmap started as it usually did, by giving the program identification and version number. Then it started reporting back
on the first host in the SLS network. It reported all of the open ports on this server. The program moved on to a second host
and began reporting back the open ports on that system, too. Once it reached the third host, however, it suddenly stopped.
Miller restarted Nmap, using the last host IP address as the starting point for the next scan. No response. He opened
another command window and tried to ping the first host he had just port-scanned. No luck. He tried to ping the SLS firewall.
Nothing. He happened to know the IP address for the SLS edge router. He pinged that and got the same result. He had been
“blackholed,” meaning his IP address had been put on a list of addresses from which the SLS edge router would no longer
accept packets. Ironically, the list was his own doing. The IDPS he had been helping SLS configure seemed to be working fine
at the moment. His attempt to hack the SLS network was shut down cold.
Discussion Questions
1. Do you think Miller is out of options as he pursues his vendetta? If you think he could take additional actions in
his effort to damage the SLS network, what are they?
2. Suppose that a system administrator at SLS read the details of this case. What steps should he or she take to
improve the company’s information security program?
3. Consider Miller’s hacking attempt in light of the intrusion kill chain described earlier and shown in Figure 9-1. At
which phase in the kill chain has SLS countered his vendetta?
1. Would such an action by SLS be ethical? Do you think its action would be legal?
2. Suppose instead that Miller had written a routine to constantly change his assigned IP address to other
addresses used by his ISP. If the SLS intrusion system determined what Miller was doing and then added the
entire range of ISP addresses to the banned list, thus stopping any user of the ISP from connecting to the SLS
network, would SLS’s action be ethical?
3. What if SLS were part of an industry consortium that shared IP addresses flagged by its IDPSs, and all companies
in the group blocked all of the ISP’s users for 10 minutes? These users would be blocked from accessing perhaps
hundreds of company networks. Would that be an ethical response by members of the consortium? What if
these users were blocked for 24 hours?
Selected Readings
• Intrusion Detection and Prevention, by Carl Endorf, Gene Schultz, and Jim Mellander. 2003. McGraw-Hill Osborne Media.
• “Intrusion Detection Systems,” by Rebecca Bace and Peter Mell. National Institute of Standards and Technology
(NIST) Special Publication 800-31. Available from the archive section of the NIST Computer Security Resource Center
at http://csrc.nist.gov.
• “Guide to Intrusion Detection and Prevention Systems,” by Karen Scarfone and Peter Mell. NIST Special Publication
800-94. Available from the NIST Computer Security Resource Center at http://csrc.nist.gov.
Module Summary
Intrusion detection systems (IDSs) identify potential intrusions and sound an alarm. The more recently devel-
oped intrusion detection and prevention systems (IDPSs) also detect intrusions and can take action to defend
the network.
An IDPS works like a burglar alarm by detecting network traffic that violates the system’s configured rules
and activating an alarm.
A network-based IDPS (NIDPS) monitors network traffic and then notifies the appropriate administrator when
a predefined event occurs. A host-based IDPS (HIDPS) resides on a particular computer or server and moni-
tors activity on that system.
Signature-based IDPSs, also known as knowledge-based IDPSs, examine data traffic for patterns that match
signatures—preconfigured, predetermined attack patterns. Anomaly-based IDPSs, also known as behavior-
based IDPSs, collect data from normal traffic and establish a baseline. When an activity is found to be outside
the baseline parameters (or clipping level), these IDPSs activate an alarm to notify the administrator.
Selecting IDPS products that best fit an organization’s needs is a challenging and complex process. A wide
array of products and vendors are available, each with different approaches and capabilities.
Deploying and implementing IDPS technology is a complex undertaking that requires knowledge and experi-
ence. After deployment, each organization should measure the effectiveness of its IDPS and then continue
with periodic assessments over time.
Honeypots are decoy systems designed to lure potential attackers away from critical systems. In the security
industry, these systems are also known as decoys, lures, or flytraps. Two variations on this technology are
known as honeynets and padded cell systems.
Trap-and-trace applications are designed to react to an intrusion event by tracing it back to its source. This
process is fraught with professional and ethical issues—some people in the security field believe that the back
hack in the trace process is as significant a violation as the initial attack.
Active intrusion prevention seeks to limit the damage that attackers can perpetrate by making the local net-
work resistant to inappropriate use.
Scanning and analysis tools are used to pinpoint vulnerabilities in systems, holes in security components, and unse-
cured aspects of the network. Although these tools are used by attackers, they can also be used by administrators
to learn more about their own systems and to identify and repair system weaknesses before they result in losses.
Module 9 Security Technology: Intrusion Detection and Prevention Systems and Other Security Tools 381
Review Questions
1. What common security system is an IDPS most 12. Why do many organizations ban port scanning
like? In what ways are these systems similar? activities or the use of hacker tools on their inter-
2. How does a false positive alarm differ from a false nal networks?
negative alarm? From a security perspective, which 13. Why would ISPs ban outbound port scanning by
is less desirable? their customers?
3. How does a network-based IDPS differ from a host- 14. What is an open port? Why is it important to limit
based IDPS? the number of open ports to those that are abso-
4. How does a signature-based IDPS differ from a lutely essential?
behavior-based IDPS? 15. What is a system’s attack surface? Why should it be
5. What is a monitoring (or SPAN) port? What is it minimized when possible?
used for? 16. What is a vulnerability scanner? How is it used to
6. List and describe the three control strategies pro- improve security?
posed for IDPSs. 17. What is the difference between active and passive
7. What is a honeypot? How is it different from a vulnerability scanners?
honeynet? 18. What is Metasploit Framework? Why is it consid-
8. How does a padded cell system differ from a ered riskier to use than other vulnerability scan-
honeypot? ning tools?
9. What is network footprinting? 19. What kind of data and information can be found
10. What is network fingerprinting? using a packet sniffer?
11. How are network footprinting and network finger- 20. What capabilities should a wireless security toolkit
printing related? include?
Exercises
1. A key feature of hybrid IDPS systems is event correlation. After researching event correlation online, define
the following terms as they are used in the process: compression, suppression, and generalization.
2. ZoneAlarm is a PC-based firewall and IDPS tool. Visit the product manufacturer at www.zonelabs.com and find
the product specification for the IDPS features of ZoneAlarm. Which ZoneAlarm products offer these features?
3. Using the Internet, search for commercial IDPS systems. What classification systems and descriptions are
used, and how can they be used to compare the features and components of each IDPS? Create a compari-
son spreadsheet to identify the classification systems you find.
4. Use the Internet to search for “live DVD security toolkit.” Read a few Web sites to learn about this class of
tools and their capabilities. Write a brief description of a live DVD security toolkit.
5. Shodan is the world’s first search engine for Internet-connected devices. Visit the Shodan Web site at
http://shodan.io and register for a free account. When you have the account, use the Explore option to look
around the site. Find the Webcam category, which is usually the top-rated category, and click it. What do you
see on this page? What kind of information is shown for the webcam listed there? Continue to explore the site.
References
1. Scarfone, K., and Mell, P. Special Publication (SP) 800-94, Rev. 1 (Draft), “Guide to Intrusion Detection and
Prevention Systems (IDPS).” National Institute of Standards and Technology. 2012. Accessed October 2,
2020, from https://csrc.nist.gov/publications/detail/sp/800-94/rev-1/draft.
2. Ibid.
3. Scarfone, K., and Mell, P. SP 800-94, “Guide to Intrusion Detection and Prevention Systems (IDPS).”
National Institute of Standards and Technology. 2007. Accessed October 2, 2020, from https://csrc.nist.gov/
publications/detail/sp/800-94/final.
4. Ibid.
382 Principles of Information Security
5. Scarfone, K., and Mell, P. SP 800-94, Rev. 1 (Draft), “Guide to Intrusion Detection and Prevention Systems
(IDPS).” National Institute of Standards and Technology. 2012. Accessed October 2, 2020, from https://csrc.
nist.gov/publications/detail/sp/800-94/rev-1/draft.
6. Ibid.
7. Ibid.
8. Ibid.
9. Ibid.
10. Ibid.
11. Scarfone, K., and Mell, P. SP 800-94, “Guide to Intrusion Detection and Prevention Systems (IDPS).”
National Institute of Standards and Technology. 2007. Accessed October 2, 2020, from https://csrc.nist.gov/
publications/detail/sp/800-94/final.
12. Ibid.
13. Scarfone, K., and Mell, P. SP 800-94, Rev. 1 (Draft), “Guide to Intrusion Detection and Prevention Systems
(IDPS).” National Institute of Standards and Technology. 2012. Accessed October 2, 2020, from https://csrc.
nist.gov/publications/detail/sp/800-94/rev-1/draft.
14. Ibid.
15. Ibid.
16. “FireEye Mandiant M-Trends 2020 Report Reveals Cyber Criminals Are Increasingly Turning to Ransom-
ware as a Secondary Source of Income.” FireEye. Accessed April 27, 2020, at www.fireeye.com/company/
press-releases/2020/fireeye-mandiant-m-trends-2020-report-reveals-cyber-criminals-are-increasingly-turning-to-
ransomware.html.
17. Scarfone, K., and Mell, P. SP 800-94, Rev. 1 (Draft), “Guide to Intrusion Detection and Prevention Systems
(IDPS).” National Institute of Standards and Technology. 2012. Accessed October 2, 2020, from https://csrc.
nist.gov/publications/detail/sp/800-94/rev-1/draft.
18. Ibid.
19. Bace, R., and Mell, P. SP 800-31, “Intrusion Detection Systems (IDS).” National Institute of Standards and
Technology. 2001. Accessed October 2, 2020, from https://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecial-
publication800-31.pdf.
20. Ibid.
21. Ibid.
22. Scarfone, K., and Mell, P. SP 800-94, “Guide to Intrusion Detection and Prevention Systems (IDPS).”
National Institute of Standards and Technology. 2007. Accessed October 2, 2020, from https://csrc.nist.gov/
publications/detail/sp/800-94/final.
23. Scarfone, K., and Mell, P. SP 800-94, Rev. 1 (Draft), “Guide to Intrusion Detection and Prevention Systems
(IDPS).” National Institute of Standards and Technology. 2012. Accessed October 2, 2020, from https://csrc.
nist.gov/publications/detail/sp/800-94/rev-1/draft.
24. Ibid.
25. Scarfone, K., and Mell, P. SP 800-94, “Guide to Intrusion Detection and Prevention Systems (IDPS).”
National Institute of Standards and Technology. 2007. Accessed October 2, 2020, from https://csrc.nist.gov/
publications/detail/sp/800-94/final.
26. Scarfone, K., and Mell, P. SP 800-94, Rev. 1 (Draft), “Guide to Intrusion Detection and Prevention Systems
(IDPS).” National Institute of Standards and Technology. 2012. Accessed October 2, 2020, from https://csrc.
nist.gov/publications/detail/sp/800-94/rev-1/draft.
27. “Acquiring and Deploying Intrusion Detection Systems.” ITL Bulletin. National Institute of Standards and
Technology. Accessed October 2, 2020, from https://csrc.nist.gov/csrc/media/publications/shared/docu-
ments/itl-bulletin/itlbul1999-11.pdf.
28. EFF. “Pen Registers.” Accessed August 20, 2016, from www.eff.org/cases/pen-registers.
29. 18 U.S. Code Module 206. “Pen Registers and Trap-and-Trace Devices.” Accessed October 2, 2020, from
https://uscode.house.gov/view.xhtml?path=/prelim@title18/part2/Module206&edition=prelim.
30. “SecTools.Org: Top 125 Network Security Tools—Vulnerability Scanners.” Accessed October 2, 2020, from
https://sectools.org/tag/vuln-scanners/.
31. “SecTools.Org: Top 125 Network Security Tools—Wireless Scanners.” Accessed October 10, 2020, from
https://sectools.org/tag/wireless/.