Lec 04.

Instrusion Detection/Prevention
System

Learning Objectives

• Identify the components of an intrusion detection

and prevention system
• Describe options for implementing intrusion
detection and prevention systems
• Explain the steps of intrusion detection
• Explain different detection approaches

2
 1. Introduction to IDPS

Recall: Instrusion

• Footprint: The attacker performs to Footprint

collect information about the
system
• Scan & Enumerate: The attacker Scan &
identifies technical information Enumera
te
about the system
• Grant access: The attacker exploits Grant
and gains access to the system access

• Maintain presence
Maintain
• Clean and hide presence

Clean
and
hide

4
 What is an IDPS?

• Intrusion Detection and Prevention System: The

system is capable of monitoring the events
occurring in a computer or network, analyzing
them for signs of intrusions and attempting to
stop them.
• Consists of more than one application or hardware
device
• Incorporates more than just detection
• Analyze more than network traffic
• IDPSs are only a layer of network defense
• Defense-in-depth: firewalls, VPNs

Goals of an IDPS

• Assess large volumes of network traffic or system

activity to find signs of unauthorized access
• Record its findings in a log so that administrators
can examine past activity
• Detect and record unauthorized access without
compromise to produce evidence admissible in
court
• Respond almost immediately
• Make itself and systems it protects as inaccessible
as possible to attackers

6
 Example: dot-dot-slash attack (1)

• FooCorp stores each employee's records in a .txt

file
• Partners can view FooCorp employees' capability
profile information from the address:
[Link]

Example: dot-dot-slash attack (2)

2. GET /[Link]?profile=xxx

Internet
FooCorp
FooCorp’s Servers
border router

3. GET /[Link]?profile=xxx

Front-end web server

1. [Link]
2. GET /[Link]?profile=xxx 4. [Link]?
profile=xxx
Remote client
5. bin/display -p xxx
8

8
 Example: dot-dot-slash attack (2)

8. 200 OK
Output of bin/amazeme

Internet
FooCorp
FooCorp’s Servers
border router
7. 200 OK
Output of bin/amazeme

9. 200 OK
Output of bin/display Front-end web server
10. Browser renders
6. Output of bin/display sent back
Remote client
5. bin/display -p xxx

Example: dot-dot-slash attack (3)

[Link]

Expose sensitive
data

10

10
 Example: How to detect?

• Method 1: Inspect Web traffic on the network or

server
• Detect the pattern of dot-dot-slash attack
• Similar to traffic inspection on firewalls
• Pros and Cons?
• Method 2: Inspect system call on server
• Detect access to sensitive files
• Pros and Cons
• Method 3: Analyze service log
• Pros and Cons

11

11

IDPS Components (1)

Knowledge DB Configuration

Operational parameters Operational

IoA, IoC
parameters
Alert
Detection Engine Responder

Event records
Handling
Sensor/Agent Event
Raw data

Monitored System

12

12
 IDPS Components (2)

• Sensor/Agent:
• Installed on monitored system
• Collects raw data and standardizes
• Detection Engine: analyzes data and identify events
• Knowledge DB: stores information about known instrusion
• Indicator of Attack(IoA): evidence that reveals the intent of a
potential or ongoing attack
• Indicator of Compromise (IoC): signs that show a system has
already been compromised
• Responder: Handles instrusion events
• System Configuration
• Others: Database, UI Console…

13

13

Active response options for IDPSs

• Active IDS responses are automated actions taken when

certain types of intrusions are detected
• Collect additional information to decide whether you
should take action.
• Change the Environment, such as
• Terminate the connection
• Reconfiguring routers and firewalls
• Take action against the intruder:
• Launching attacks against or attempt to actively gain information
about the attacker’s host or site
• Usually on cyber warface
• Need to obtain legal advice

14

14
 Passive response options for IDPSs

• Passive IDS responses provide information to system

users
• Alarms and Notifications
• SNMP Traps:
• Reporting them to a network management system

15

15

IDPS Operation (1)

• Step 1: Installing the IDPS

16

16
 IDPS Operation (2)

• Step 2: Gathering data

17

17

IDPS Operation (3)

• Step 3: An alert message is transmitted when an event

matches an IoA/IoC

18

18
 IDPS Operation (4)

• Step 4: The IDPS takes preventive or response actions

19

19

IDPS Operation (5)

• Step 5: The administrator has the responsibility to monitor

alerts and determine whether countermeasures need to
be taken

20

20
 IDPS Operation (6)

• Step 6: Organization spells out in the security policy and

followed if the IDPS detects a true positive

21

21

IDPS Operation (7)

• Step 7: The event that caused the alert is entered in the

IDPS log. The administrator to determine whether
patterns of misuse have occurred.

22

22
 2. Instrusion detection approaches

23

23

Misuse/Signature detection

24

24
 Misuse/Signature detection

• Signature: a pattern that corresponds to a known threat

• Signature-based detection: comparing observed events
against signatures to identify possible incident
• The signatures need to be large enough
• Challenge: defining signatures
• Threats disguised by the use of evasion techniques
• Many variants of known threats
• Complex communications have many state
• Advantages:
• Effective at detecting known threat
• Easy to obtain signatures
• Disadvantages:
• Ineffective at detecting previously unknown threats

25

25

Rule-based detection

• Rules represent known instrusions

• Rule-based detection involves the use of rules for
identifying known penetrations
• Rules can also be defined that identify suspicious
behavior
• Example: Snort rules
alert tcp $EXTERNAL_NET any -> $HTTP_SERVERS $HTTP_PORTS
(msg:"WEB-MISC cross site scripting attempt";
flow:to_server,established; content:"<SCRIPT>";
nocase; classtype:web-application-attack; sid:1497;
rev:6;)

26

26
 Anomaly detection

27

27

Anomaly detection (1)

• Anomaly detection:
• Define profiles that represent the normal behavior
• Compare the characteristics of current activity to normal profile
• Raise alert for any anomaly difference
• Challenge: training profiles
• Static profile: unchanged unless the IDPS is specifically directed
to generate a new profile
• Dynamic profile: adjusted constantly as additional events are
observed
• Advantages:
• Capability of detecting previously unknown threats
• Disavantages:
• Produces often produce many false alarms
• Difficult to determine why a particular alert was generated

28

28
 Anomaly detection (2)

• Training phase: developing profiles of normal events by

collecting data from the normal operations of monitored
system
• Classification phase: the observered events are compared
with the model in order to classify them as either
legitimate or abnormal
• Approaches:
• Statistical
• Knowledge based
• Machine learning

29

29

Statistical approaches (1)

• Parametric methods: Normal data points follow a specific

statistical distribution
• Anomalies deviate significantly from this distribution
• Steps of statistical anomaly detection
1) Define the expected range of normal values by using statistical
measures(mean, standard derivation, median…)
2) Define a threshold beyond which data points are considered
anomalous
3) Anomaly scoring, e.g. z-score, IQR, Grubbs' Test
4) Anomalies deviate significantly from this distribution
• Example:
|𝑥 − 𝑚𝑒𝑎𝑛|
𝑧_𝑠𝑐𝑜𝑟𝑒 𝑥 =
𝑠𝑡𝑎𝑛𝑑𝑎𝑟𝑑 𝑑𝑒𝑟𝑖𝑣𝑎𝑡𝑖𝑜𝑛
• thresshold = 3
• If z_score(x) > 3, x is abnormal

30

30
 Statistical approaches (2)

• Time series-based methods: predict the current state of

the series based on previous observations
• Steps of time series-based detection:
1) Fit a predictive function to historical data
2) Use statistical measures or confidence intervals to define
thresholds for anomalies
3) Predict current state
4) Compute residuals (the difference between observed and
predicted states)
5) Identify time points where the residuals exceed the threshold as
anomalies.
• Example: Hidden Markov Model, Autoregressive
Model(AR), Moving Average Model(MA), ARMA, ARIMA…

31

31

Statistical approaches (3)

• Non-parametric methods: do not rely on any assumptions

about the distribution of the data
• Focus on the inherent properties of the data without relying on
parameter estimation
• Application: data distribution is unknown or complex
• More flexible and robust than parametric
methods
• Example: Kernel Density Estimation

Source: [Link]

32

32
 Statistical approaches (4)

• Advantages:
• Relative simplicity
• Low computation cost
• Disadvantages:
• Sensitive to threshold
• Not all events can be modeled using these approaches

33

33

Knowledge-based approaches

• Detection leverage domain-specific knowledge, rules, or

heuristics to detect outliers or unusual behavior in the
data
• Key Characteristics:
• Expert Knowledge: rely on expert knowledge about the system
• Rule-based: setting up a set of rules or conditions that describe
what constitutes normal behavior
• Steps of knowledge-based approaches:
1) Define Normal Behavior. Example: Employees access to
resource from 8.00 am to 8.00 pm.
2) Establish Rules or Constraints
3) Monitor Data
4) Flag Anomalies

34

34
 Knowledge-based approaches

• Advantages:
• Robust
• Flexible
• Disadvantage:
• Difficultty and time required to develop knowledge
• Human experts must assist with the process

35

35

Machine learning approaches

• Use ML algorithms to develop a normal

model that can classify event as
normal or abnormal
• Steps of ML-based approaches:
1) Data collection
2) Data engineering
3) Model training: abnormal
• Supervised learning: use labeled data
• Unsupervised learning: use unlabeled data K-means
• Semi-supervised learning: use mixture of
labeled and unlabeled data
4) Model evaluating
5) Instrusion detection

36

36
 Machine learning approaches

• Advantages:
• Flexibility
• Adaptability
• Ability to capture interdependencies between
observed metrics
• Disadvantages:
• High computation cost

37

37

Accuracy of detection

38

38
 Accuracy of detection

• True Positive Rate (TPR): The

probability the detector alerts,
given there is an attack
• True Negative Rate (TNR): The
probability the detector does
not alert, given there is no
attack
• False Negative Rate (FNR):
The probability the detector
does not alert, given there is
an attack Receiver Operating Curve
• False Positive Rate (FPR): The
probability the detector alerts,
given there is no attack

39

39

Perfect Detectors

• Can we build a detector with a false positive rate of 0%?

void detector_with_no_false_positives(char *input) {

printf("Nope, not an attack!");
}

• How about a detector with a false negative rate of 0%?

void detector_with_no_false_negatives(char *input) {

printf("Yep, it's an attack!");
}

40

40
 Perfect Detectors

0% false negative rate 0% false positive rate

You must always have tradeoffs between false positive and

false negative rates

41

41

Detection Tradeoffs

• Should we choose a system with low FPR or low FNR?

• Depends on the level of damage to the system for each type of
error and the cost of remediation
• Depends on the actual attack rate
• Example: Assume the system has FPR = 0.1% and FNR =
2%
• Case 1: The system has 1000 accesses per day, of which 5
are attacks:
• False Alarms: 995 x 0.1% ~ 1 alert/day
• Missed Attacks: 5 x 2% ~ 0.1 (missing <1 attack/week)
• Case 2: 1,000,000 accesses per day, with 5 attacks:
• False Alarms: 999.995 x 0.1% ~ 1000 alerts/day 

42

42
 Base Rate Fallacy

• Consider the detector from before: 0.1% false positive rate

• Assume a 0% false negative rate: Every attack is detected
• Scenario from before: Our server receives 10,000,000 non-attacks and 5
attacks per day
• Expected number of false positives per day: 10,000,000 × 0.1% = 10,000
• You see the detector alert. What is the probability this is really an
attack?
• Of the 10,005 detections, 5 are real attacks, and 10,000 are false positives
• There is an approximately 0.05% probability that the detector found a real
attack
• Base rate fallacy: Even though the detector alerted, it’s still highly
unlikely that you found an attack, because of the high false positive rate
• Detecting is hard when the base rate of attacks is low

43

43

3. IDPS Architecture

44

44
 3.1. Network-based IDPS

45

45

Network-based IDPS (NIDPS)

• A detector installed on the network, between the local

network and the rest of the Internet
• Sensors can be devices capable of analyzing network traffic
• Software sensors installed on certain network nodes
• Types of sensors used for NIDPS (Network Intrusion
Detection and Prevention System):
• Inline sensors: placed at locations where network traffic must
pass through
• Passive sensors: can copy network traffic and do not require
network traffic to pass through them

46

46
 Inline sensor vs passive sensor

• Inline Sensor • Passive sensor

47

47

Implement NIDPS sensors

Sensor behind firewall, in DMZ: Sensor on critical subnet:
1) Sees attacks that originate from 1) Detects attacks targeting
outside. critical systems.
2) Sees attacks that might target the 2) Allows organizations
servers on DMZ.
3) By observing outgoing traffic,
with limited resources to
possible to recognize that a server focus resources on network
has been compromised. assets of greatest value.

Sensor on major LAN backbone:

1) Monitors large amount of traffic
thus increasing chances of spotting an
attack.
2) Detect unauthorized activity by
authorized users.

48

48
 Functions of NIDPS

• Network Traffic Analysis

• Able to extract data objects from network traffic
• Shadow execution
• Analyze events and behaviors:
• Detect known attack patterns
• Identify abnormal events
• Logging

49

49

Prevention Capabilities

• Passive Only:
• Ends the current TCP session by sending TCP reset
packets
• Inline Only:
• Performing Inline Firewalling
• Throttling Bandwidth Usage
• Altering Malicious Content
• Both Passive and Inline:
• Reconfiguring Other Network Security Devices
• Running a Third-Party Program or Script

50

50
 Data collection

Data from SNMP (Simple Network Management Protocol)

• Provides statistical information about network traffic:
• By interface ports
• By protocols
• Benefits:
• SNMP is pre-installed on network devices, allowing sensors to
collect data through SNMP procedure calls.
• The information is already aggregated and standardized, reducing
computational costs for Network Intrusion Detection and
Prevention Systems (NIDPS).

51

51

Data collection

Packet sniffer
• Can provide detailed information about network traffic.
• Simple data collection method.
• The collection scope is limited to the network segment
where the sensor is placed.
• Challenge: Selecting an appropriate feature set for each
detection method.

52

52
 Extract feature from network traffic (1)

• Extract feature from packet header

Number Feature Description Number Feature Description
1 Ethernet Size 10 IP Source
2 Ethernet Destination 11 IP Destination
3 Ethernet Source 12 TCP Source Port
4 Ethernet Protocol 13 TCP Destination Port
5 IP header Length 14 UDP Source Port
6 IP Type of Service 15 UDP Destination Port
7 IP Length 16 UDP Length
8 IP Time To Live 17 ICMP Type
9 IP Protocol 18 ICMP Code

53

53

Extract feature from network traffic (2)

• Extract feature from TCP connections

Number Feature Description
19 Source IP
20 Destination IP
21 Duration of Connection
22 Connection Starting Time
23 Connection Ending Time
25 Number of packets sent from Source to Destination
26 Number of packets sent from Destination to Source
27 Number of data bytes sent from Source to Destination
28 Number of data bytes sent from Destination to Source
29 Number of Fragmented packets
30 Number of Overlapping Fragments
31 Number of Acknowledgement packets
32 Number of Retransmitted packets
33 Number of Pushed packets
34 Number of SYN packets Number of FIN packets
35 Number of TCP header Flags
36 Number of Urgent packets

54

54
 Extract feature from network traffic (3)

• Statistical features:
• Time window: Detects instrusions occurring over a short period.
• Connection window: Detects instrusions occurring over a long
period.

Number Feature Description

No. of unique connections used by the same SrcIP as the current
37
record
No. of unique connections used by the same SrcIP on the same Dst-
38
Port as the current record
No. of unique connections used by the same SrcIP on different Dst-
39
Port as the current record
No. of unique connections used by the same SrcIP as the current
40
record that have SYN flag

55

55

Extract feature from network traffic (4)

• Statistical features
Number Feature Description
No. of unique Ports used by the same SrcIP to connect on the same
47
DstIP and the same DstPort as the current record
No. of unique Ports opened on the sane DstIP by the same SrcIP as
48
the current record
No. of unique connections that use the same service as the current
49
packet
No. of unique connections that use the same service and have
50
different DstIP as the current packet
No. of unique connections that use the same service as the current
51
packet that have SYN flag
No. of unique connections that use the same service as the current
52
packet that have RST flag

56

56
 NIDPS

• Advantages:
• Cheap: A single detector can cover a lot of systems
• Easy to scale: As the network gets larger, add computing power to
the NIDPS
• Simple management: Easy to install and manage a single detector
• End systems are unaffected
• Disadvantages: Inconsistent or ambiguous interpretation
between the detector and the end host
• What the NIDPS sees doesn’t match what the end system sees
• Inputs are interpreted and parsed differently between the NIDS
and the end system
• Encrypted traffic

57

57

NIDPS: Evasion Attacks

• Assume the NIDPS is configured to identify traffic

containing the keyword 'attack' as malicious traffic.
• Method: Scanning the content of each packet for the
keyword.
• Algorithms: Boyer-Moore, Bloom filter, etc.
…..…attack………………………….
• However:
• TCP transmits data as a byte stream, with undefined data
boundaries.
• IP packet fragmentation.
As a result, malicious traffic may be split across multiple packets.

…………..…at tack……………..

58

58
 NIDPS: Evasion Attacks

• Send TCP packets having same Seq# or IP fragments

having same fragmentation offset
NIDPS observes:
A T T A I C K

NIDPS

Attacker Target

A T T A I C K A T T A I C K

Same TCP seq #

or same IP frag offset

59

59

Solution

• Reassembling packets across the entire TCP

stream:
• Store out-of-order packets in a buffer.
• Sort packets into the correct sequence.
• Reassemble the packets into the original data.
• Challenges:
• Requires state management.
• Vulnerable to attacks where the attacker sends a large
volume of fragmented or disjointed TCP packets.

60

60
 NIDPS: Evasion Attacks

• Short TTL attack

• Assume the number of hops from the attack source to the
NIDPS is H1, and to the target is H2, where H2 > H1 (How
does the attacker know this?).
• The attacker injects packets with H1 < TTL < H2 to disrupt
and obfuscate the attack traffic.

61

61

Example: Short TTL attack

NIDPS observes:
a t r t u a c k

NIDPS NIDPS(H1= 20)

Attacker Target Target(H2 = 25)

a Seq=1,TTL = 27
a
Seq=2,TTL = 27
t t
Seq=3,TTL = 22
r
Seq=3,TTL = 27
t t
Seq=4,TTL = 22 Solution?
u
Seq=4,TTL = 27
a a
Seq=5,TTL = 27
c c
Seq=6,TTL = 27
k k

62

62
 3.2. Host-based IDPS

63

63

Host-based IDPS (HIDPS)

• Collecting and analyzing information to detect

attacks on specific network nodes:
• Network traffic
• System status: processes, resource management, file
access, logs, configuration changes...
• Activities and states of applications
• Centralized model:
• Sensors placed on network nodes to collect
information
• HIDPS Server: analyzes information collected by
sensors and sends alerts to network nodes
• Distributed model: sensors and HIDPS Server
deployed on the same network node

64

64
 HIDPS implementation

65

65

Centralized vs Distributed implementation

• Centralized HIDPS • Distributed HIDPS

Sensor Server

66

66
 Functions of HIDPS

• Monitors an analyzes incoming and outgoing

traffic
• Profiles entities interacting with network nodes
• Monitors an analyzes accessing to the file system
• Monitors an analyzes main memory
• Analyzing process activities and behaviors
• Detects instrusion
• Create a sandbox execution environment
• Logging

67

67

HIDPS

• Advanatges:
• Fewer problems with inconsistencies or ambiguities: The HIDPS is
on the end host, so it will interpret packets exactly the same as the
end host!
• Works for encrypted messages
• Can protect against non-network threats too (e.g. malicious user
inside the network)
• Performance scales better than NIDPS: one NIDPS is more
vulnerable to being overwhelmed than many HIDPS
• Disadvanatges
• Expensive: Need to install one detector for every end host
• HIDPS and other programs compete for resources on the same
host

68

68
 3.3. Hybrid IDPS

69

69

Hybrid IDPS

• Combine features of HIDPSs & NIDPSs to gain

flexibility and increase security
• Advantage:
• Monitor network as a whole with NIDPS sensors, and
monitor attacks on each individual computer with
HIDPS sensors
• Disadvantage:
• Complications in getting various components to work
together
• Data gathered from two systems can be difficult to
absorb and analyze

70

70
 Centralized Hybrid IDPS

• There is only one

management system
• All monitoring, detection
and reporting is
controlled directly from a
central location
• Advantages:
• Low cost
• Sumple and easy
management

71

71

Fully distributed hybrid IDPS

• Monitoring and
detection is done using
an agent-based
approach, where
response decisions are
made at the point of
analysis.
• Since IDPS does not
have to wait for a
response from a
centralized control
facility, reaction to
attacks is greatly
speeded up.

72

72
 Partially distributed hybrid IDPS

• Monitoring and
detection is controlled
from a local control
node, with hierarchical
reporting to one or more
central location(s).
• Individual agents can
respond to local threats
in real-time
• Reporting to a central
facility enables detection
of widespread attacks.

73

73