Journal of Network and Computer Applications 36 (2013) 42–57

Contents lists available at SciVerse ScienceDirect

Journal of Network and Computer Applications

journal homepage: www.elsevier.com/locate/jnca

Review

A survey of intrusion detection techniques in Cloud

Chirag Modi a,n, Dhiren Patel a, Bhavesh Borisaniya a, Hiren Patel b,
Avi Patel c, Muttukrishnan Rajarajan c
a
NIT Surat, Gujarat, India
b
S.P. College of Engineering, Gujarat, India
c
City University London, UK

a r t i c l e i n f o a b s t r a c t

Article history: In this paper, we survey different intrusions affecting availability, confidentiality and integrity of Cloud
Received 3 January 2012 resources and services. Proposals incorporating Intrusion Detection Systems (IDS) and Intrusion
Received in revised form Prevention Systems (IPS) in Cloud are examined. We recommend IDS/IPS positioning in Cloud
15 May 2012
environment to achieve desired security in the next generation networks.
Accepted 16 May 2012
Available online 2 June 2012
& 2012 Elsevier Ltd. All rights reserved.

Keywords:
Cloud computing
Firewalls
Intrusion detection system
Intrusion prevention system

Contents

1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
2. Intrusions to Cloud systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
2.1. Insider attack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
2.2. Flooding attack. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
2.3. User to root attacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
2.4. Port scanning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
2.5. Attacks on virtual machine (VM) or hypervisor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
2.6. Backdoor channel attacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
3. Firewalls: common solution to intrusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
4. IDS and IPS techniques: evolution. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
4.1. Signature based detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
4.2. Anomaly detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
4.3. Artificial neural network (ANN) based IDS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
4.4. Fuzzy logic based IDS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
4.5. Association rule based IDS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
4.6. Support vector machine (SVM) based IDS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
4.7. Genetic algorithm (GA) based IDS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
4.8. Hybrid techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
5. Various types of IDS/IPS used in Cloud computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
5.1. Host based intrusion detection systems (HIDS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
5.2. Network based intrusion detection system (NIDS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
5.3. Distributed intrusion detection system (DIDS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
5.4. Hypervisor-based intrusion detection system. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
5.5. Intrusion prevention system (IPS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
5.6. Intrusion detection and prevention system (IDPS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

n
Corresponding author. Tel.: þ91 9408883560.
E-mail address: [email protected] (C. Modi).

1084-8045/$ - see front matter & 2012 Elsevier Ltd. All rights reserved.
http://dx.doi.org/10.1016/j.jnca.2012.05.003
 C. Modi et al. / Journal of Network and Computer Applications 36 (2013) 42–57 43

6. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

1. Introduction 2.2. Flooding attack

Cloud computing aims to provide convenient, on-demand, In this attack, attacker tries to flood victim by sending huge
network access to a shared pool of configurable computing number of packets from innocent host (zombie) in network.
resources (e.g. networks, servers, storage, applications, and ser- Packets can be of type TCP, UDP, ICMP or a mix of them.
vices), which can be rapidly provisioned and released with This kind of attack may be possible due to illegitimate network
minimal management effort or service provider interactions connections.
(Mell and Grance, 2011). Cloud provides services in various In case of Cloud, the requests for VMs are accessible by anyone
forms: Software as a Service-SaaS (e.g. Google apps, 2011), Plat- through Internet, which may cause DoS (or DDoS) attack via
form as a Service-PaaS (e.g. Google app engine (2011)), Micro- zombies. Flooding attack affects the service’s availability to
soft’s Azure (Azure services platform, 2011)) and Infrastructure as authorized user. By attacking a single server providing a certain
Service-IaaS (e.g. Amazon web services, 2011(AWS); Eucalyptus, service, attacker can cause a loss of availability of the intended
2011; Open Nebula (Opennebula, 2011)). service. Such an attack is called direct DoS attack. If the server’s
As Cloud services are provisioned through the Internet; secur- hardware resources are completely exhausted by processing the
ity and privacy of Cloud services are key issues to be looked upon. flood requests, the other service instances on the same hardware
International Data Corporation (IDC) survey (Gens, 2009) showed machine are no longer able to perform their intended tasks. Such
that security is the greatest challenge of Cloud computing. The type of attack is called indirect DoS attack.
recent Cloud computing security white paper by Lockheed Martin Flooding attack may raise the usage bills drastically as the
Cyber Security division (Martin, 2010) shows that the major Cloud would not be able to distinguish between the normal usage
security concern after data security is intrusion detection and and fake usage.
prevention in Cloud infrastructures. Cloud infrastructure makes
use of virtualization techniques, integrated technologies and runs 2.3. User to root attacks
through standard Internet protocols. These may attract intruders
due to many vulnerabilities involved in it. Here, an attacker gets an access to legitimate user’s account by
Cloud computing also suffers from various traditional attacks sniffing password. This makes him/her able to exploit vulnerabil-
such as IP spoofing, Address Resolution Protocol spoofing, Routing ities for gaining root level access to system. For example, Buffer
Information Protocol attack, DNS poisoning, Flooding, Denial of overflows are used to generate root shells from a process running
Service (DoS), Distributed Denial of Service (DDoS), etc. For e.g. as root. It occurs when application program code overfills static
DoS attack on the underlying Amazon Cloud infrastructure caused buffer. The mechanisms used to secure the authentication process
BitBucket.org, a site hosted on AWS to remain unavailable for few are a frequent target. There are no universal standard security
hours (Brooks, 2009). Computing-cost using current crypto- mechanisms that can be used to prevent security risks like weak
graphic techniques cannot be overlooked for Cloud (Chen and password recovery workflows, phishing attacks, keyloggers, etc.
Sion, 2010). Firewall can be a good option to prevent outside In case of Cloud, attacker acquires access to valid user’s
attacks but does not work for insider attacks. Efficient intrusion instances which enables him/her for gaining root level access to
detection systems (IDS) and intrusion prevention systems (IPS) VMs or host.
should be incorporated in Cloud infrastructure to mitigate these
attacks. 2.4. Port scanning
Rest of the paper is organized as follows: Section 2 discusses
various attacks applicable to Cloud environment. Traditional Port scanning provides list of open ports, closed ports and
firewalls as a security solution are discussed briefly in Section 3. filtered ports. Through port scanning, attackers can find open
Section 4 presents various techniques for IDS/IPS. Section 5 ports and attack on services running on these ports. Network
surveys existing IDS/IPS types and examines Cloud specific work related details such as IP address, MAC address, router, gateway
on IDS with conclusion and references at the end. filtering, firewall rules, etc. can be known through this attack.
Various port scanning techniques are TCP scanning, UDP scan-
ning, SYN scanning, FIN scanning, ACK scanning, Window scan-
ning etc. In Cloud scenario, attacker can attack offered services
2. Intrusions to Cloud systems through port scanning (by discovering open ports upon which
these services are provided).
There are several common intrusions affecting availability,
confidentiality and integrity of Cloud resources and services. 2.5. Attacks on virtual machine (VM) or hypervisor

By compromising the lower layer hypervisor, attacker can gain

control over installed VMs. For e.g. BLUEPILL (Rutkowska, 2006),
2.1. Insider attack SubVir (King et al., 2006) and DKSM (Bahram et al., 2010) are
some well-known attacks on virtual layer. Through these attacks,
Authorized Cloud users may attempt to gain (and misuse) hackers can be able to compromise installed-hypervisor to gain
unauthorized privileges. Insiders may commit frauds and disclose control over the host.
information to others (or modify information intentionally). This New vulnerabilities, such as zero-day vulnerability, are found
poses a serious trust issue. For example, an internal DoS attack in Virtual Machines (VMs) (NIST: National vulnerability database,
demonstrated against the Amazon Elastic Compute Cloud (EC2) 2011) that attract an attacker to gain access to hypervisor or other
(Slaviero, 2009). installed VMs. Zero-day exploits are used by attackers before the
44 C. Modi et al. / Journal of Network and Computer Applications 36 (2013) 42–57

developer of the target software knows about the vulnerability. used. For flooding attack and backdoor channel attack, either
A zero-day vulnerability was exploited in the HyperVM virtuali- signature based intrusion detection or anomaly based intrusion
zation application which resulted in destruction of many virtual detection techniques can be used.
server based websites (Goodin, 2009).

3. Firewalls: common solution to intrusions

2.6. Backdoor channel attacks
Firewall protects the front access points of system and is
It is a passive attack which allows hacker to gain remote access treated as the first line of defense. Firewalls are used to deny or
to the infected node in order to compromise user confidentiality. allow protocols, ports or IP addresses. It diverts incoming traffic
Using backdoor channels, hacker can control victim’s resources according to predefined policy. Basic firewall installation is shown
and can make it as zombie to attempt DDoS attack. It can also be in Fig. 1 (2011, http://teleco-network.blogspot.com/), where it is
used to disclose the confidential data of victim. Due to this, installed at entry point of servers. Several types of firewalls are
compromised system faces difficulty in performing its regular discussed in Sequeira (2002).
tasks. In Cloud environment, attacker can get access and control In Table 1, we summarize different firewalls used in network
Cloud user’s resources through backdoor channel and make VM as for security purpose. As firewalls sniff the network packets at the
Zombie to initiate DoS/DDoS attack. boundary of a network, insider attacks cannot be detected by
Firewall (in Cloud) could be the common solution to prevent traditional firewalls. Few DoS or DDoS attacks are also too
some of the attacks listed above. To prevent attacks on VM/ complex to detect using traditional firewalls. For instance, if there
Hypervisor, anomaly based intrusion detection techniques can be is an attack on port 80 (web service), firewalls cannot distinguish
good traffic from DoS attack traffic (2011, http://en.wikipedia.org/
wiki/Denial-of-service_attack).

4. IDS and IPS techniques: evolution

Another solution is to incorporate IDS or IPS in Cloud. However

the efficiency of IDS/IPS depends on parameters like technique
used in IDS, its positioning within network, its configuration, etc.
Traditional IDS/IPS techniques such as signature based detection,
anomaly detection, artificial intelligence (AI) based detection etc.
can be used for Cloud.

4.1. Signature based detection

Signature based intrusion detection attempts to define a set of

rules (or signatures) that can be used to decide that a given
pattern is that of an intruder. As a result, signature based systems
are capable of attaining high levels of accuracy and minimal
number of false positives in identifying intrusions. Little variation
in known attacks may also affect the analysis if a detection
system is not properly configured (Brown et al., 2002). Therefore,
signature based detection fails to detect unknown attacks or
variation of known attacks. One of the motivating reasons to
use signature based detection is ease in maintaining and updating
preconfigured rules. These signatures are composed by several
elements that identify the traffic. For example, in SNORT (2011,
Fig. 1. Basic firewall installation (2011, http://teleco-network.blogspot.com/). https://www.snort.org/) the parts of a signature are the header

Table 1
Summary of firewalls.

Firewall type Summary

Static packet filtering firewalls  Allow/deny packet by inspecting only header information such as source or destination address, port numbers etc.
 Do not detect malicious code in packets and cannot prevent against spoofing and fragment attack.

Stateful packet filtering  Used in client server environment where client initiates request and server responses which are allowed in bypassing the
firewalls firewall rules.
 Requires additional resources like memory for state tables maintained in hardware or software.

Stateful inspection firewalls  Enhanced form of stateful packet filtering firewalls.

 Used for applications like FTP where multiple ports are used and examine the payload and open or close the ports as per the
protocol.

Proxy firewalls  Can isolate internal network within Internet. Analyze the protocol syntax by breaking up client/server connection.
 Require lots of network resources.
 C. Modi et al. / Journal of Network and Computer Applications 36 (2013) 42–57 45

(e.g. source address, destination address, ports) and its options Cannady (1998) proposed a three layer neural network for
(e.g. payload, metadata), which are used to determine whether or misuse detection in network. The feature vector used in Cannady
not the network traffic corresponds to a known signature. (1998) was composed of nine network features (Protocol ID, Source
Stiawan et al. (2010) presented some issues regarding signature Port, Destination Port, Source IP Address, Destination IP Address,
based intrusion prevention system and showed different possible ICMP Type, ICMP Code, Raw Data Length, Raw Data). However,
frameworks. intrusion detection accuracy is very low. Moradi and Zulkernine
In Cloud, signature based intrusion detection technique can be (2004) presented MLP based IDS. They showed that inclusion of
used to detect known attack. It can be used either at front-end of more hidden layers increase detection accuracy of IDS. This
Cloud to detect external intrusions or at back end of Cloud to approach improves detection accuracy of the approach proposed
detect external/internal intrusions. Like traditional network, in Cannady (1998). Grediaga et al. (2006) compared the rate of
it cannot be used to detect unknown attacks in Cloud. Approaches successively finding intrusion with MLP and self organization map
presented by Roschke et al. (2009), bakshi and Yogesh (2010), (SOM) and showed that SOM has high detection accuracy than ANN.
Lo et al. (2008), and Mazzariello et al. (2010) use signature based It is claimed that, Distributed Time Delay Neural Network (DTDNN)
intrusion detection system for detecting intrusions on VMs (or (Ibrahim, 2010) has higher detection accuracy for most of the
front end of Cloud environment). These approaches are discussed network attacks. DTDNN is a simple and efficient solution for
in the later section. classifying data with high speed and fast conversion rates. Accuracy
of this approach can be improved by combining it with other soft
computing techniques mentioned above.
4.2. Anomaly detection
ANN based IDS is an efficient solution for unstructured net-
work data. The intrusion detection accuracy of this approach is
Anomaly (or behavioral) detection is concerned with identify-
based on number of hidden layers and training phase of ANN.
ing events that appear to be anomalous with respect to normal
An approach proposed by Vieira et al. (2010), uses ANN based
system behavior. A wide variety of techniques including data
anomaly detection technique for Cloud environment, which
mining, statistical modeling and hidden markov models have
requires more training samples as well as more time for detecting
been explored as different ways to approach the anomaly detec-
intrusions effectively.
tion problem. Anomaly based approach involves the collection of
data relating to the behavior of legitimate users over a period of
4.4. Fuzzy logic based IDS
time, and then apply statistical tests to the observed behavior,
which determines whether that behavior is legitimate or not.
Fuzzy logic (Han and Kamber, 2006) can be used to deal with
It has the advantage of detecting attacks which have not been
inexact description of intrusions.
found previously. The key element for using this approach
Tillapart et al. (2002) proposed Fuzzy IDS (FIDS) for network
efficiently is to generate rules in such a way that it can lower
intrusions like SYN and UDP floods, Ping of Death, E-mail Bomb,
the false alarm rate for unknown as well as known attacks.
FTP/Telnet password guessing and port scanning. Evolving fuzzy
Dutkevyach et al. (2007) provided anomaly based solution to
neural network (EFuNN) is introduced in Chavan et al. (2004) for
prevent intrusion in real time system, which analyzes protocol
reducing training time of ANN. It uses mixture of supervised and
based attack and multidimensional traffic. However, there is a
unsupervised learning. The experimental results shown indicate
scope of optimization to reduce number of IPS. Zhengbing et al.
that using reduced number of inputs EFuNN has better classifica-
(2007) presented lightweight intrusion detection system to detect
tion accuracy for IDS than only using ANN. The approaches
the intrusion in real-time, efficiently and effectively. In this work,
proposed by Tillapart et al. (2002) and Chavan et al. (2004)
behavior profile and data mining techniques are automatically
cannot be used in real time for detecting network intrusions as
maintained to detect the cooperative attack.
the training time is significant by more. Fuzzy association rules
Anomaly detection techniques can be used for Cloud to detect
presented by Su et al. (2009 are used to detect network intrusion
unknown attacks at different levels. In Cloud, large numbers of
in real time. Two rule sets are generated and mined online from
events (network level or system level) occur, which makes
training data. Features for comparison are taken from network
difficult to monitor or control intrusions using anomaly detection
packet header. This approach is used for large scale DoS/DDoS
technique. Garfinkel and Rosenblum (2003), Vieira et al. (2010),
attacks.
Dastjerdi et al. (2009) and Guan and Bao (2009) proposed
To reduce training time of ANN (Vieira et al., 2010), fuzzy logic
anomaly detection techniques are proposed to detect intrusions
with ANN can be used for fast detection of unknown attacks
at different layers of Cloud.
in Cloud.
The ability of soft computing techniques to deal with uncertain
and partially true data makes them attractive to be applied in
4.5. Association rule based IDS
intrusion detection (Moradi and Zulkernine, 2004). There are
many soft computing techniques such as Artificial Neural Net-
Some intrusion attacks are formed based on known attacks or
work (ANN), Fuzzy logic, Association rule mining, Support Vector
variant of known attacks. To detect such attacks, signature apriori
Machine (SVM), Genetic Algorithm (GA), etc. that can be used to
algorithm (Han et al., 2002) can be used, which finds frequent subset
improve detection accuracy and efficiency of signature based IDS
(containing some features of original attack) of given attack set.
or anomaly detection based IDS.
Han et al. (2002) proposed network based intrusion detection
using data mining technique. In this approach, signature based
4.3. Artificial neural network (ANN) based IDS algorithm generates signatures for misuse detection. However,
drawback of the proposed algorithm is its time consumption
The goal of using ANNs (Han and Kamber, 2006) for intrusion for generating signatures. Zhengbing et al. (2008) solved the
detection is to be able to generalize data (from incomplete data) database scanning time problem examined in Han et al. (2002).
and to be able to classify data as being normal or intrusive They proposed scanning reduction algorithm to reduce number of
(Ibrahim, 2010). Types of ANN used in IDS are as (Ibrahim, database scans for effectively generating signatures from pre-
2010): Multi-Layer Feed-Forward (MLFF) neural nets, Multi-Layer viously known attacks. However, it has very high false positive
Perceptron (MLP) and Back Propagation (BP). alarm rate since unwanted patterns are produced. Lei et al. (2010)
46 C. Modi et al. / Journal of Network and Computer Applications 36 (2013) 42–57

proposed length decreasing support based apriori algorithm to framework for fitness function, which is simple and flexible.
detect intrusions to reduce production of short pattern as Generated rules are used to detect network intrusions. The paper
derived by Han et al. (2002) and Zhengbing et al. (2008) and uses quantitative as well as categorical features of network for
allows some interesting patterns. It is faster than other apriori generating classification rules. This increases the detection rate
based approaches. and improves accuracy. However, limitation of this approach is
In Cloud, association rules can be used to generate new the best fit problem. Lu and Traore (2004) presented GP based
signatures. Using newly generated signatures, variations of approach to generate rules from network features. They used
known attacks can be detected in real time. support confidence based fitness function for deriving rules,
which classifies network intrusions effectively. However, training
period for the fitness function takes more time. Xiao et al. (2005)
4.6. Support vector machine (SVM) based IDS
proposed information theory and GA based approach that is used
to detect abnormal behavior. It identifies small number of net-
SVM (Han and Kamber, 2006) is used to detect intrusions
work features closely with network attacks based on mutual
based on limited sample data, where dimensions of data will not
information between network features and type of intrusion.
affect the accuracy.
However, this approach only considers discrete features.
In Chen et al. (2005), it is shown that the results (regarding
Dhanalakshmi and Ramesh Babu (2008) proposed a method
false positive rate) are better in case of SVM compared with that
which is used to detect misuse and anomaly by combining fuzzy
of ANN, since ANN requires large amount of training samples for
and genetic algorithms. Fuzzy is used to include quantitative
effective classification, whereas SVM has to set fewer parameters.
parameters in intrusion detection, whereas genetic algorithm is
However, SVM is used only for binary data. Nevertheless, detec-
used to find best fit parameters of introduced numerical fuzzy
tion accuracy can be improved by combining SVM with other
function. This approach solves best fit problem as reported by Lu
techniques (Li and Lu, 2010). Li and Lu, 2010 designed an
and Traore (2004).
intelligent module for network intrusion prevention system with
In Cloud environment, selection of optimal parameters (net-
a combination of SNORT and configurable firewall. The SVM
work features) for intrusion detection will increase the accuracy
classifier is also used with SNORT to reduce false alarm rate and
of underlying IDS. For that, Genetic algorithm (GA) based IDS can
improve accuracy of IPS.
be used in Cloud.
In Cloud, if limited sample data are given for detecting
intrusions, then use of SVM is an efficient solution; since dimen-
sions of data are not affecting accuracy of SVM based IDS.
4.8. Hybrid techniques
4.7. Genetic algorithm (GA) based IDS
Hybrid techniques use the combination of two or more of
Genetic algorithms (GAs) (Dhanalakshmi and Ramesh Babu, above techniques.
2008; Li, 2004) are used to select network features (to determine As shown in Fig. 2 (Botha et al., 2002), NeGPAIM is based on
optimal parameters) which can be used in other techniques for hybrid technique combining two low level components including
achieving result optimization and improving accuracy of IDS. fuzzy logic for misuse detection and neural networks for anomaly
Gong et al. (2005) used seven features (Duration, Protocol, detection, and one high level component which is a central engine
Source_port, Destination_port, Source_IP, Destination_IP, Attack_- analyzing outcome of two low level components. It is an effective
name) of captured packet. They used support confidence based model, which does not require dynamic updates of rules.

CLIENT EXT HOST INT HOST

Internal GUI: Ext

Host-base Manager Manager
Information
Source
Information Central Analysis
Refiner Engine
C
Network-base O C C C C
Information L O Ext
O O O
U Source L U Responder
U U U
Central
S E P P P P
DBMS
E C L L L L
R Application T E E E E Fuzzy Engine
S Information O R R R R
Source R Central
Database Template DB

Internal
Responder User
Neural Engine
Behavior DB

Fig. 2. Architecture of NeGPAIM (Botha et al., 2002).

 C. Modi et al. / Journal of Network and Computer Applications 36 (2013) 42–57 47

To improve performance of IDS, Katar (2006) presented an collecting information such as file system used, network events,
approach which uses combination of Naı̈ve Bayes, ANN and system calls, etc. HIDS observes modification in host kernel, host
Decision Tree (DT) classifiers on three separate sets of data input. file system and behavior of the program. Upon detection of
Independent output of each classifier is generated and combined deviation from expected behavior, it reports the existence of
using the multiple fusion techniques. This approach uses the attack. The efficiency of HIDS depends on chosen system char-
advantages of each classifier and improves overall performance acteristics to monitor. Each HIDS detects intrusion for the
of IDS. machines in which it is placed as shown in Fig. 3.
It is advantageous to use soft computing techniques on With respect to Cloud computing, HIDS can be placed on a host
traditional IDS for Cloud environment. However, each technique machine, VM or hypervisor to detect intrusive behavior through
has some advantages and limitations, which affect the monitoring and analyzing log file, security access control policies,
performance of IDS. For e.g. Higher time consumption to and user login information. If installed on VM, HIDS should be
learn ANN network and lesser flexibility are the major drawbacks monitored by Cloud user whereas in case of installing it on
of ANN. Combining fuzzy logic to data mining techniques Hypervisor, Cloud provider should monitor it (cox, 2011).
improves flexibility. GA with fuzzy logic enhances performance HIDS based architecture for Cloud environment is proposed
of IDS, since GA selects best fit rules for IDS. GA has better by (Vieira et al., 2010). In this architecture, each node of Grid/
efficiency for matching patterns but in specific manner Cloud contains IDS which provides interaction among service
rather than general (Beg et al., 2010). For handling large number offered (e.g. IaaS), IDS service and storage service. As shown in
of network features, SVM is preferable. Association rule based Fig. 4 (Vieira et al., 2010), IDS service is composed of two
IDS is efficient for only correlated attacks. However, an efficiency components: Analyzer and Alert System.
of association rule based IDS depends on the used knowledge The event auditor captures data from various resources like
base. system logs. Based on the data received from event auditor, the
In Table 2, a summary of existing IDS/IPS techniques is IDS service is used for detecting intrusion by using behavior based
presented with their strengths and limitations. technique or knowledge based technique. Knowledge based
technique is used to detect known attacks, whereas the behavior
based technique is used to detect unknown attacks. For detecting
5. Various types of IDS/IPS used in Cloud computing unknown attacks, artificial neural network (ANN) is used in this
approach. When any attack or intrusion is detected, alert system
There are mainly four types of IDS used in Cloud: Host based informs other nodes. So, this approach is efficient even for
intrusion detection system (HIDS), Network based intrusion detecting unknown attacks by applying feed forward ANN.
detection system (NIDS), Hypervisor based intrusion detection The experiments demonstrated by Vieira et al. (2010) show
system and Distributed intrusion detection system (DIDS). that the false positive and false negative alarm rate is very low
when large numbers of training samples are applied for behavior
5.1. Host based intrusion detection systems (HIDS) analysis method. The limitation of this approach is that it cannot
detect any insider intrusions which are running on VMs.
HIDS monitors and analyzes the information collected from a For effective usage of Cloud resources, multilevel IDS and log
specific host machine. HIDS detects intrusion for the machine by management (Lee et al., 2011) is applied at different level of

Table 2
Summary of IDS/IPS techniques.

IDS/IPS technique Characteristics/advantages Limitations/challenges

Signature based  Identifies intrusion by matching captured patterns with  Cannot detect new or variant of known attacks.
detection preconfigured knowledge base.  High false alarm rate for unknown attacks.
 High detection accuracy for previously known attacks.
 Low computational cost.

Anomaly detection  Uses statistical test on collected behavior to identify intrusion.  More time is required to identify attacks.
 Can lower the false alarm rate for unknown attacks.  Detection accuracy is based on amount of collected behavior or
features.

ANN based IDS  Classifies unstructured network packet efficiently.  Requires more time and more samples training phase.
 Multiple hidden layers in ANN increase efficiency of classification.  Has lesser flexibility.

Fuzzy Logic based  Used for quantitative features.  Detection accuracy is lower than ANN.
IDS  Provides better flexibility to some uncertain problems.

Association rules  Used to detect known attack signature or relevant attacks in misuse  It cannot detect totally unknown attacks.
based IDS detection.  It requires more number of database scans to generate rules.
 Used only for misuse detection.

SVM based IDS  It can correctly classify intrusions, if limited sample data are given.  It can classify only discrete features. So, preprocessing of those
 Can handle massive number of features. features is required.

GA based IDS  It is used to select best features for detection.  It is complex method.
 Has better efficiency.  Used in specific manner rather than general.

Hybrid techniques  It is an efficient approach to classify rules accurately.  Computational cost is high.
48 C. Modi et al. / Journal of Network and Computer Applications 36 (2013) 42–57

security strength (e.g. high, medium, and low) to user based on IDS is installed) is requested to assign guest OS image to user.
the degree of anomaly. As shown in Fig. 5, AAA is used for Database stores user information, system log, transaction of user
authentication, authorization and accounting. Authenticated and system, whereas storage center stores user’s private data
user’s information (stored in database) is used to calculate which are isolated from one user to another. This approach
anomaly level. AAA uses anomaly level to select proper IDS that provides fast detection mechanism. However, it requires more
has corresponding security level. Then host OS (where selected guest OSs (having IDS) for high level users.
Guan and Bao (2009) have proposed change point based idea
to detect all types of attacks in attack space. In this approach, all
attacks are taken as a sample space. Then the set is decomposed
using statistics based on mutually exclusive sets. The generated
subsets which belong to sample space are used to construct
intrusion detection algorithm. However, no experimental results
or deployment mechanisms are reported yet.
In self-similarity based lightweight intrusion detection
method for Cloud Computing (Kwon et al., 2011), the number of
events from the Windows’ security event log is extracted. Feature
selection procedure makes groups by combining security ID (SID)
and EventID in Windows system. Then each VM measures self-
similarity. Self-similarity is calculated using two techniques viz;
cosine and hybrid (Kwon et al., 2011). If calculated similarity
deviated from normal behavior, IDS generates alerts. Outlier
source procedure identifies intruder and associated IP address.
Then IDS reports the information to a system administrator. This
approach is cost effective and efficient for detecting anomaly in
Cloud environment. However, it works only for Windows system.
Arshad et al. (2011) proposed an abstract model for intrusion
detection and severity analysis to provide the overall security of
the Cloud. It consists of six components viz; system call handler,
detection module, security analysis module, profile engines,
global components and intrusion response system. System call
handler collects system calls executed by guest VM. Detection
module applies anomaly or signature based techniques to col-
lected system calls for detecting intrusions in VM. Severity
Fig. 3. Host based intrusion detection system (HIDS) (2011, http://maltainfosec. analysis module calculates severity of detected intrusion for
org/archives/26-The-concept-of-Intrusion-Detection-Systems.html). victim VM. Profile engine generates and manages profiles specific

Grid node Grid node

Service
Service
IDS Service
Event auditor

IDS Service
Event auditor

Analyzer
Analyzer
Alert system
Alert system

Storage service
Storage service

Knowledge Behavior
Knowledge Behavior
base base
base base

Grid node
Service

IDS Service
Event auditor

Analyzer
Alert system

Storage service

Knowledge Behavior
base base

Fig. 4. IDS architecture for Grid/Cloud environment (Vieira et al., 2010).

 C. Modi et al. / Journal of Network and Computer Applications 36 (2013) 42–57 49

Terminal

Guest Guest Guest Guest

OS OS OS OS

Multi-IDS Multi-IDS Internet

Host OS Host OS

Hypervisor Hypervisor AAA

Storage Center Database

Fig. 5. Multilevel IDS architecture (Lee et al., 2011).

to VM. These profiles are used for differentiating malicious hypervisor. However, it has several limitations. It cannot help
behavior and normal behavior of user. Global components are when attack is within a virtual network that runs entirely inside
used for overall infrastructure including resource manager, sche- the hypervisor. In Cloud environment, installing NIDS is the
duler, profile and other security components. Intrusion response responsibility of Cloud provider.
system is used to select appropriate response mechanism for a VM compatible IDS architecture proposed by Roschke et al.
particular intrusion. This approach has minimal response time (2009) is shown in Fig. 7. There are mainly two components used
and human intervention. However, experimental results are not in this approach: IDS management unit and IDS sensor.
evaluated. IDS management unit consists of event gatherer, event data-
base, analysis component and remote controller. Event gatherer
5.2. Network based intrusion detection system (NIDS) collects malicious behavior identified by IDS sensor and stores in
event database. Event database stores information regarding
NIDS monitors network traffic to detect malicious activity such captured events. Analysis component (configured by users)
as DoS attacks, port scans or even attempts to crack into accesses event database and analyze events. IDS-VMs are mana-
computers. The information collected from network is compared ged by the IDS Remote Controller which can communicate with
with known attacks for intrusion detection. NIDS has stronger IDS-VMs and IDS sensors. IDS sensors on the VM detects and
detection mechanism to detect network intruders by comparing reports malicious behavior and transmits triggered event to event
current behavior with already observed behavior in real time. gatherer. Sensors can be NIDS configured by IDS remote con-
NIDS mostly monitors IP and transport layer headers of individual troller. In this approach, new sensors can be easily integrated,
packet and detects intrusion activity. NIDS uses signature based which require only sender/receiver pair to connect event gath-
and anomaly based intrusion detection techniques. NIDS has very erer. IDS-VM management controls, monitors and configures VM.
limited visibility inside the host machines. If the network traffic is The VM management can also recover VMs. This approach is used
encrypted, there is no effective way for the NIDS to decrypt the in virtualized environment to prevent VMs from being compro-
traffic for analysis. mised. However, this approach requires multiple instances of IDS.
Hemairy et al. (2009) surveyed about the security solutions In the approach proposed (bakshi and Yogesh, 2010), for
that can be applicable to detect ARP spoofing attacks through detecting DDoS attack in VM, IDS is installed in virtual switch
experiments and implementation. They concluded that XArp 2 to log incoming or outgoing traffic into database. To detect known
tool (2011, http://www.filecluster.com/Network-Tools/Network- attacks, the logged packets are analyzed and compared by the IDS
Monitoring/Download-XArp.html) is an efficient available secur- in real time with known signature. The IDS determines nature of
ity solution that can accurately detect ARP spoofing attacks attacks and notifies virtual server. Then virtual server drops
among other tools. By combining it to ARP request storm and packets coming from the specified IP address. If attack type is
ARP scanning detection mechanism, its performance can be DDoS, all the zombie machines are blocked. The virtual server
improved. then transfers targeted applications to other machines hosted by
Fig. 6 represents positioning of NIDS in a typical network with separate data center and routing tables are immediately updated.
aim to direct the traffic through the NIDS. NIDS placed between Firewall (placed at new server) blocks all the packets coming from
firewall and various hosts of the network. identified IP address. This approach can block the DDoS attack in
NIDS can be deployed on Cloud server interacting with virtualized environment and can secure services running on
external network, for detecting network attacks on the VMs and virtual machines.
50 C. Modi et al. / Journal of Network and Computer Applications 36 (2013) 42–57

Mazzariello and Bifulco (2010) presented SNORT based misuse of this service is that it can detect only known attacks at
detection in open source Eucalyptus Cloud. In this approach, network level.
SNORT is deployed at Cloud controller (CC) as well as on physical Sandar and Shenai (2012) introduced new type of DDoS attack,
machines (hosting virtual machines) to detect intrusions coming called Economic Denial of Sustainability (EDoS) in Cloud services
from external network. This approach solves the problem of and proposed solution framework for EDoS protection. EDoS attack
deploying multiple instances of IDS as in bakshi and Yogesh can be called as HTTP and XML based DDoS attack. EDoS protection
(2010). It is a fast and cost effective solution. However, it can framework uses firewall and puzzle server to detect EDoS attack.
detect only known attacks since only SNORT (2011, https://www. Firewall is used to detect EDoS at entry point of Cloud, where as
snort.org/) is involved. puzzle server is used to authenticate user. In this work, authors
Hamad and Hoby (2012) proposed a method for providing demonstrated EDoS attack in the Amazon EC2 Cloud. However,
intrusion Detection as a Service in Cloud, which delivers Snort for proposed solution is not efficient since it uses only traditional
Cloud clients in a service-based manner. Fig. 8 shows subscription firewall. Research is still needed to detect EDoS attack in Cloud.
and IDS operation request of Cloud intrusion detection service Houmansadr et al. (2011) proposed Cloud based intrusion
(CIDS). User request related to his subscription details is for- detection and response system for mobile phones. In this
warded to the database layer, whereas the IDS operation requests approach, intrusion detection and response services are delivered
are forwarded to the system layer. The system layer and the to registered smartphones. It copies smartphone to VM in Cloud
database layer can communicate with each other to translate using proxy that copies incoming traffic to device. This traffic is
preferences (that exist in the database layer) into runtime- used for intrusion detection. If any intrusion is detected, intrusion
configurations that are used at the system layer. The limitation response mechanism selects an action for detected intrusion and
sends a non-intrusive software agent in the device.

5.3. Distributed intrusion detection system (DIDS)

A Distributed IDS (DIDS) consists of several IDS (e.g. HIDS,

NIDS, etc.) over a large network, all of which communicate with
each other, or with a central server that enables network
monitoring. The intrusion detection components collect the
system information and convert it into a standardized form to
be passed to central analyzer. Central analyzer is machine that
aggregates information from multiple IDS and analyzes the same.
Combination of anomaly and signature based detection
approaches are used for the analysis purpose. DIDS can be used
for detecting known and unknown attacks since it takes advan-
tages of both the NIDS and HIDS (Jones and Sielken, 2000). Fig. 9,
demonstrates the working of DIDS.
In Cloud environment, DIDS can be placed at host machine or
at the processing server (in backend).
In cooperative agent based approach (Lo et al., 2008), indivi-
dual NIDS module is deployed in each Cloud region as shown in
Fig. 10 (Lo et al., 2008). If any Cloud region detects intrusions, it
alerts other region. Each ID sends alert to each other, to judge
severity of this alert. If new attack is detected, the new blocking
Fig. 6. Network based intrusion detection system (2011, http://maltainfosec.org/ rule is added to block list. So, this type of detection and preven-
archives/26-The-concept-of-Intrusion-Detection-Systems.html). tion helps to resist attacks in Cloud.

Fig. 7. Architecture of VM integrated IDS management (Roschkeet al., 2009).

 C. Modi et al. / Journal of Network and Computer Applications 36 (2013) 42–57 51

User
Internet
Internet

User Requests

Firewall
User Layer
Dispatch protection Dispatch subscriptions IDS 1
requests requests
IDS 2

Network
System Layer Database Layer Router
Configure requests to
Runtime configuration
Update Protection Update subscription
Review Alerts IDS 3
IDS 4
IDS 5

Fig. 9. Distributed intrusion detection system (DIDS).

Cloud IDS Subscription DB

Cooperative Cooperative
Fig. 8. Intrusion detection as a service in Cloud (Hamad and Hoby, 2012).
Agent Agent

Internet
The system architecture consists of intrusion detection, alert
Response & Response &
clustering, threshold check, intrusion response and blocking and
Block Block
cooperative agent. In case of intrusion detection, it drops attacker
packet, then sends alert message about the attack detected by
itself to other region. Alert clustering module collects alert
produced by other regions. The decision about alert (whether it Threshold Threshold
is true or false) is identified after calculating severity of collected Check Check
alerts. This approach is suitable for preventing Cloud system from
single point of failure caused by DDoS attack.
Dastjerdi and Bakar (2009) proposed scalable, flexible and cost Alert Clustering Alert Clustering
effective method to detect intrusion for Cloud applications regardless
of their locations using mobile agent. This method aims for protecting
VMs that are outside the organization. Mobile agent collects evi- Intrusion Intrusion
dences of an attack from all the attacked VM for further analysis and Detection Detection
auditing. This approach is used to detect intrusion in VM migrated
outside the organization. However, it produces more network load. Fig. 10. Block diagram of cooperative agent based approach (Lo et al., 2008).
Ram (2012) proposed mutual agent based approach to detect
DDoS attack in Cloud computing. In this approach, IDS module is
deployed in each Cloud region, as presented by Lo et al. (2008). If system. Hypervisor based IDS is one of the important techniques,
any region finds intrusion, mutual agent at that region notifies other specifically in Cloud computing, to detect intrusion in virtual
regions. Each region calculates severity of alerts generated from environment.
other regions. If new attack is found after calculating severity of Virtual machine introspection based IDS (VMI-IDS) architecture
intrusion, new blocking rule is added into block table at each region. is shown in Fig. 11 (Garfinkel and Rosenblum, 2003). VMI-IDS is
In such a way, DDoS attack is detected in whole Cloud by using different from traditional HIDS since it directly observes hardware
mutual cooperation among Cloud regions. For intrusion detection, states, events and software states of host and offers more robust
Snort is used in this approach. Therefore, known attacks in network view of the system than HIDS. Virtual machine monitor (VMM) is
can be detected. However, it cannot detect unknown attack. Also, it responsible for hardware virtualization and also offers isolation,
requires high computation cost for exchanging alerts. monitoring and interposition properties. VMI-IDS has greater access
to the VMM than the code running in monitored VM.
5.4. Hypervisor-based intrusion detection system VMM interface is used for VMI-IDS to communicate with
VMM, which allows VMI-IDS to get VM state information,
Hypervisor is a platform to run VMs Hypervisor-based intru- monitoring certain events and controlling VMs. This VMM inter-
sion detection system is running at hypervisor layer. It allows face is composed of Unix socket to send commands or receive
user to monitor and analyze communications between VMs, responses to/from VMM. It also supports physical memory access
between hypervisor and VM and within the hypervisor based of monitored VM. OS interface library is used to provide low level
virtual network. Availability of information is one of the benefits machine states from VMM in terms of higher level OS structure.
of hypervisor-based IDS. Policy engine is incorporated for making high-level queries about
VM introspection based IDS (Garfinkel and Rosenblum, 2003) the OS of monitored host. Policy engine responds in appropriate
is one of the examples of hypervisor based intrusion detection manner, even if system is compromised. VMI-IDS implements
52 C. Modi et al. / Journal of Network and Computer Applications 36 (2013) 42–57

Fig. 11. VMI-based IDS architecture (Garfinkel and Rosenblum, 2003).

Attacker

Internet IPS

IPS IPS
Router/Firewall/Proxy

Ethernet Switch
IPS

Protected DMZ

Protected Internal
Resources

Protected Internal Subnet

Fig. 12. Network based intrusion prevention system (2011, http://www.javvin.com/networksecurity/IPS.html).

complex anomaly detection. It is used for lie detection, signature IPS should be configured accurately for expected results; other-
detection, program integrity detection and row socket detection. wise it stops flow of packets resulting in network unavailability.
According to results shown by (Garfinkel and Rosenblum, 2003), For intrusion prevention, mostly firewall with IDS is used which
performance of policy engine is good in terms of workload and contains signature specifying network traffic rules. Based on the
time. However, VMM or OS library can be compromised. preconfigured rules, IPS decides whether network traffic should
Recently IBM Research is pursuing virtual machine introspec- be passed or blocked. In response to detected attack, IPS can stop
tion approach to create a layered set of security services inside the attack itself, can change the attack contents or change
protected VM running on same physical machine as the guest security environment.
VMs running in the Cloud (2011, http://www.zurich.ibm.com/csc/ Ahmed et al. (2009) proposed efficient network based intru-
security/securevirt.html#top). sion detection and prevention approach, which does not require
installing IDS on every node. This approach solves trust problem
5.5. Intrusion prevention system (IPS) and transferring alert message problem. It has less overhead and
no false alarm rate. Leu and Li (2009) proposed Cumulative-Sum-
IPS monitors network traffic and system activities to detect based Intrusion Prevention System (CSIPS) for preventing DoS or
possible intrusions (With the help of IDS) and dynamically DDoS attacks. In this work, authors used packet classification
responds to intrusions for blocking the traffic or quarantine it. algorithm and three detection algorithms (namely inbound,
 C. Modi et al. / Journal of Network and Computer Applications 36 (2013) 42–57 53

outbound, and forwarded) which cooperatively detect DDoS extension uploaded dynamically when the firewall is running. It is
attack and send their logs to remote IPS machine. developed based on Netfilter/Iptables. When network packet goes
IPSs are mainly classified into two categories: Host based IPS through HOOK, HOOK function is called.
(HIPS) and Network based IPS (NIPS). The possible positioning of The HOOK function identifies whether the data packet matches
IPS in a typical network is shown in Fig. 12. the preconfigured rules or not and returns the result to kernel which
In Cloud computing architecture, HIPS can be used to detect will decide to accept or to drop the packet. General data structure
and prevent intrusion on VM, Hypervisor or host system where it then transferred to HOOK function which transforms data structure
is deployed. NIPS can be used to protect the whole network (or to another structure defined as Iptable application module. Also
part of network) to safeguard multiple systems (such as VMs) at pointer to skb buffer storing the packet information is transferred to
a time.
Fagui et al. (2009) presented Xen based host system firewall
and its extensions. In this approach, Netfilter and Iptables are
used to build firewall on host Linux system which inspects
network data. Netfilter is the framework which Linux kernel
implements. Iptables is a firewall management program based
on Netfilter framework. As shown in Fig. 13 (Fagui et al., 2009),
Iptables extensions consist of two parts: First part is interacting
with Iptables application layer which is developed as shared
library and second part is Iptable kernel developed as kernel
dynamic library. Kernel dynamic library is uploaded at runtime.
Moreover, a firewall GUI is used to configure firewall rules.
Iptables application extension is used for authentication of rules
configured by users and to parse the parameters of the rules. Each
rule filled in data structure supplied by Iptables. Iptable kernel

Firewall GUI

Iptables
Iptables Application
Application Extend Module

Netfilter Iptables Kernel

Extend Iptables Kernel Extend Module
Module

Netfilter Kernel

Fig. 13. The architecture of Xen based firewall and its extension (Fagui et al., 2009). Fig. 15. Positioning IDPS in network (Scarfone and Mell, 2007).

The defending
agents
The credible
knowledge base
learning

External The data The expert Internal

Network switcher system network

The feature rules

repository or
policies The detecting The
and identifying monitoring
agents workstation

Fig. 14. Architecture of dynamic intelligence Cloud firewall (Jia and Wang, 2011).
54 C. Modi et al. / Journal of Network and Computer Applications 36 (2013) 42–57

Fig. 16. Placement of IDS on VMs and hypervisor/host system.

Table 3
Summary of IDS/IPS types.

IDS/IPS Characteristics/strengths Limitations/Challenges Positioning in Cloud Deployment and monitoring

Type authority

HIDS  Identify intrusions by monitoring host’s file  Need to install on each On each VM, Hypervisor or Host On VMs: Cloud Users. On
system, system calls or network events. machine (VMs, system. Hypervisor: Cloud provider.
 No extra hardware required. hypervisor or host
machine).
 It can monitor attacks
only on host where it is
deployed.

NIDS  Identify intrusions by monitoring network traffic.  Difficult to detect In external network or in virtual Cloud provider.
 Need to place only on underlying network. intrusions from network.
 Can monitor multiple systems at a time. encrypted traffic.
 It helps only for
detecting external
intrusions.
 Difficult to detect
network intrusions in
virtual network.

Hypervisor  It allows user to monitor and analyze New and difficult to In hypervisor. Cloud provider.
based communications between VMs, between understand.
IDS hypervisor and VM and within the hypervisor
based virtual network.

DIDS  Uses characteristics of both NIDS and HIDS, and  Central server may be In external network, on Host, on On VMs: Cloud Users. For
thus inherits benefits from both of them. overloaded and difficult Hypervisor or on VM. other cases: Cloud provider.
to manage in
centralized DIDS.
 High communication and
computational cost.

IPS  Prevents intrusion attacks.  Detection accuracy for For NIPS: In external/internal NIPS: Cloud provider. HIPS on
 NIPS prevent network attacks. preventing attacks is network. For HIPS: On VM or VM: Cloud user. HIPS on
 HIPS prevent system level attacks. lower than IDS. Hypervisor. Hypervisor: Cloud provider.

IDPS  Effectively detect and prevent intrusion attacks.  Complex architecture. Network based IDPS: In NIDPS: Cloud provider. HIDPS
external/internal network. Host (on VM): Cloud user. HIDPS
based IDPS: On VM or (on Hypervisor): Cloud
hypervisor. provider.

HOOK function to identify the rules irrespective of the rules Jia and Wang (2011) designed an IPS model based on
matching the data. The skb buffer saves the data of the packet, such dynamically distributed Cloud firewall linkage. Authors intro-
as source IP address, destination port number, which is captured duced the structure and function of Cloud firewall. As shown in
when it goes through the HOOK. However, Unknown attacks cannot Fig. 14, external information is trained using data switcher
be prevented by this approach. through credible database. Then this information is learned
 C. Modi et al. / Journal of Network and Computer Applications 36 (2013) 42–57 55

Table 4
Summary of existing IDS approaches in Cloud

Title IDS type Technique used Positioning Pros Cons

IDS architecture for Cloud HIDS Signature based On each node False rate for unknown attack Requires more training time and
environment (Vieira et al., 2010) and Anomaly is lower since ANN used. samples for detection accuracy.
detection using
ANN.
Multi-level IDS (Lee et al., 2011) HIDS Anomaly detection On each Guest OS Provides fast detection Requires more resources for high level
mechanism. users.
Self-similarity based IDS HIDS Anomaly detection On each VM Can be used in real time. Works only for Windows system.
(Kwon et al., 2011)
Abstract model of IDS HIDS Signature based On each VM It has minimal response time Experimental results are not
(Arshad et al., 2011) and anomaly and human intervention. evaluated.
detection
VM compatible IDS architecture NIDS Signature based On each VM Secures VM based on user Multiple instances of IDS are required
(Roschke et al., 2009) detection configuration. which degrades performance.
DDoS attack detection in virtual NIDS Signature based On each VM Secures VM from DDoS Can only detects known attacks.
machine (bakshi and Yogesh, 2010) detection attacks.
NIDS in open source Cloud NIDS Signature based On traditional network Can detect several known It cannot detect insider attacks as well
(Mazzariello et al., 2010) detection attacks. as unknown attacks.
IDS as a Service (Hamad and NIDS Signature based Snort is provided as a Provides user to detect It cannot detect unknown attacks.
Hoby, 2012) detection web service known attack on his/her
running service.
EDoS protection NIDS Signature based On traditional network Blocks HTTP and XML based It cannot detect unknown attacks.
(Sandar and Shenai, 2012) detection DDoS attack.
Cloud based IDS for mobile phones NIDS Anomaly detection On VM Detects malicious behavior on It cannot be used as general purpose.
(Houmansadr et al., 2011) smartphones.
Cooperative agent based approach DIDS Signature based On each Cloud region Prevents system from single Cannot be used for all types of
(Lo et al., 2008) detection point failure. attacks.Computational overhead high.
Mobile agent based approach DIDS Anomaly detection On each VM Provides IDS for Cloud Produce network load with increase
(Dastjerdi et al., 2009) application regardless by of VMs attached to mobile agent.
their location.
Mutual agent based approach DIDS Signature based On each Cloud region Detects DDoS attack in whole Cannot be used to detect unknown
(Ram, 2012) detection cloud environment. attacks.High computational cost.
VMI-IDS based architecture Hypervi- Anomaly detection. On hypervisor Detects attacks on VMs VMI IDS can be attacked.
(Garfinkel and Rosenblum, 2003) sor based Very complex method
Xen based Host system firewall (Fagui - Prevention On each Host Prevention using user Not used for preventing unknown
et al., 2009) configured rules attacks
IPS model based on cloud firewall HIPS Anomaly In internal network Can be used for real time Experimental results are not yet
linkage (Jia and Wang, 2011) prevention. interactive defense and better available
optimization to Cloud firewall
CP based approach - Anomaly detection - Used to detect all types of Experimental results are not yet
(Guan and Bao, 2009) attacks. Solves limitation of available
computing time

using knowledge base and compared with predefined rules or 2007). Proper configuration and management of IDS and IPS
policies. Rules or policies are generated by using data mining combination can improve security. NIST (Scarfone and Mell,
techniques. The defending agents, expert system, and the 2007) explained how intrusion detection and prevention can be
detecting and identifying agents are used for real time defense, used together to strengthen security, and also discussed different
detection of intrusions and identification. If the intrusions are ways to design, configure, and manage IDPS.
detected, the monitor station calls defending filter, prevention IDPS is classified into three broad categories: Signature-based,
and generates alerts, then give auditing record. The monitoring anomaly-based, and stateful protocol analysis. There are many
work station is used to monitor internal intrusions. An intelli- types of IDPS technologies. IDPS are divided into four groups
gent IPS module based on dynamically distributed Cloud fire- based on the type of events that they monitor and the ways in
wall linkage is used for real time interactive defense and better which they are deployed (Scarfone and Mell, 2007): (a) Network-
optimization of Cloud firewall. When user of internal network Based (b) Wireless (c) Network Behavior Analysis (NBA) (d) Host-
accesses external network resources, IPS uses feature detection Based. Positioning of network based IDPS in typical network is
and recognition mode of Cloud security for analyzing and shown in Fig. 15 (Scarfone and Mell, 2007).
deciding safety of resources which are accessed by users. It Considering the Cloud scenario, network-based IDPS can be used
uses expert system used in Cloud firewall. In this approach to protect multiple VMs from network end points. Host-based IDPS
user’s behaviors, files, web pages etc are used for calculating can be deployed at VMs or hypervisors to protect the machines on
resources’ reputation and detecting intrusions. Experimental which it is placed.
results of this approach are not evaluated. Concluding the whole section, we now graphically represent
positioning of various types of IDS/IPS (mentioned above) in the
5.6. Intrusion detection and prevention system (IDPS) different layers of Cloud architecture. Fig. 16 demonstrates the
same followed by its summary.
Having their own strengths and weaknesses, individual IDS Incorporating IDS on VM allows monitoring the activity of VM
and IPS are not capable of providing full-fledged security. It is itself. Cloud user should be held responsible to deploy, manage
very effective to use combination of IDS and IPS, which is called and monitor IDS on VM. Placing IDS on underlying hypervisor
IDPS. Apart from identifying possible intrusions, IDPS stops and provides ability to detect intrusion activity including communica-
reports them to security administrators (Scarfone and Mell, tion between VMs on that hypervisor. However large amount
56 C. Modi et al. / Journal of Network and Computer Applications 36 (2013) 42–57

of communicating data reduces performance of IDS or causes engineering computing and applications in sciences, 2009. ADVCOMP ’09;
packet dropping. Deploying, managing and monitoring IDS should 2009: pp. 175–180.
Dhanalakshmi Y, Ramesh Babu I. Intrusion detection using data mining along
be done by Cloud provider. The virtual network (established in fuzzy logic and genetic algorithms. International Journal of Computer Science
host system) allows VMs to communicate directly without & Security 2008;8(2):27–32.
using external network. IDS can be located within such network Eucalyptus, Website, /http://eucalyptus.cs.ucsb.edu/S; 2011.
Fagui Liu L, Xiang S Wenqianl Su, L. The design and application of xen-based host
to monitor traffic between the VMs as well as between the system firewall and its extension. In: The 2009 international conference on
VM and host. Cloud provider can be given duties to manage IDS. electronic computer technology; 2009: pp. 392–5.
IDS can be deployed in external network, which is a door to Google apps, Website, /http:/www.google.comS; 2011.
Google app engine, Website, /http://code.google.com/appengine/S; 2011.
Cloud system for users. It allows monitoring of network traffic Gens F, New IDC IT Cloud Service Survey: Top Benefits and Challenges, IDC
over the traditional network. Cloud provider should be the proper Exchange, /http://blogs.idc.com/ie/?p=730S; 2009.
entity to serve here. Summary of various IDSs are shown in Goodin, D, Webhost Hack Wipes Out Data for 100,000 Sites, /http://www.
theregister.co.uk/2009/06/08/webhost_attack/S; 2009.
Table 3.
Garfinkel T, Rosenblum M. A Virtual Machine Introspection Based Architecture for
In Tables 4, we summarize presented approaches with their Intrusion Detection. Proc. Network and Distributed Systems Security Sympo-
type, technique, positioning in Cloud, pros and cons. This illus- sium 2003:191–206.
trates several challenges which need to be addressed before a Guan Y, Bao J. A CP Intrusion detection strategy on cloud computing, in
international symposium on web information systems and applications
standard security framework for the Cloud can be proposed. (WISA); 2009: pp 84–7.
Grediaga A, Ibarra F, Garcı́a F, Ledesma B, Brotons F. Application of neural networks in
network control and information security. LNCS 2006:208–13.
Gong RH, Zulkernine M, Abolmaesumi P. A software implementation of a genetic
6. Conclusions
algorithm based approach to network intrusion detection. In: Proceedings of
the sixth international conference on software engineering, artificial intelligence,
We discussed several intrusions which can threat integrity, networking and parallel/distributed computing and first ACIS international
confidentiality and availability of Cloud services. Firewall only workshop on self-assembling wireless networks (SNPD/SAWN‘05); 2005.
Han J, Kamber M. Data mining concepts and techniques. 2nd edition Morgan
may not be sufficient to solve Cloud security issues. This paper Kaufmann Publishers; 2006.
emphasized the usage of alternative options to incorporate Han H, Lu XL, Ren LY.Using data mining to discover signatures in network-based
intrusion detection and intrusion prevention techniques into intrusion detection. In: Proceedings of the first international conference on
machine learning and cybernetics, Beijing (1) (2002).
Cloud and explored locations in Cloud where IDS/IPS can be
Hemairy MA, Amin S, Trabelsi Z. Towards more sophisticated ARP Spoofing
positioned for efficient detection and prevention. Recent research detection/prevention systems in LAN networks. In: International conference
findings incorporating IDS/IPS in Cloud have been discussed with on the current trends in information technology (CTIT); 2009: pp. 1–6.
their advantages and disadvantages. The adoption of soft comput- Hamad H, Hoby MA. Managing intrusion detection as a service in cloud networks.
International Journal of Computer Applications 2012;41(1):35–40.
ing techniques in IDS/IPS can improve the security. We finally Houmansadr A, Zonouz SA, Berthier, R, Cloud-based, A. Intrusion detection and
identify several security challenges that need to be addressed by response system for mobile phones. In: Proceedings of the 2011 IEEE/IFIP 41st
the research community to make Cloud a secure and trusted international conference on dependable systems and networks workshops;
2011: pp. 31–2.
platform for the delivery of future Internet of Things. Ibrahim LM. Anomaly network intrusion detection system based on distributed
time-delay neural network. Journal of Engineering Science and Technology
2010;5(4):457–71.
References Jones AK, Sielken RS. Computer system intrusion detection: a survey, /http://
www.cs.virginia.edu/  jones/IDS-research/Documents/jones-sielken-sur
Azure services platform, Website, /http://www.microsoft.com/azureS; 2011. vey-v11.pdfS; 2000.
Amazon web services, Website, /http://aws.amazon.comS; 2011. Jia T, Wang X. The research and design of intelligent IPS model based on dynamic
Arshad J, Townend P, Xu J. An abstract model for integrated intrusion detection cloud firewall linkage. International Journal of Digital Content Technology and
and severity analysis for clouds. International Journal of Cloud Applications its Applications 2011;5(3):304–9.
and Computing 2011;1(1):1–17. King S, Chen P, Wang Y-M. SubVirt: Implementing malware with virtual machines.
Ahmed M., Pal, R., Hossain, H.M., Bikas, M., Hasan, M.K., NIDS: A Network Based In: 2006 IEEE symposium on security and privacy; 2006: pp 314–27.
Approach to Intrusion Detection and Prevention, Computer Science and Katar C. Combining multiple techniques for intrusion detection. International
Information Technology—Spring Conference;2009: pp. 141–4. Journal of Computer Science & Network Security 2006;6(2B):208–18.
Brooks C, Amazon EC2 Attack Prompts Customer Support Changes. Tech Target, Kwon H, Kim,T, Yu, SJ, Kim HK. Self-similarity based lightweight intrusion
/http://searchcloudcomputing.techtarget.com/news/article/0,289142,sid201_ detection method for cloud computing. In: Proceedings of the third interna-
gci1371090,00.htmlS; 2009. tional conference on intelligent information and database systems—Volume
Bahram S, Jiang X, Wang Z, Grace M. DKSM: subverting virtual machine Part II; 2011: pp. 353–62.
introspection for fun and profit.In: Proceedings of the 29th IEEE international Lo CC, Huang CC, Ku J. Cooperative Intrusion detection system framework for
symposium on reliable distributed systems; 2010. cloud computing networks. In: First IEEE International Conference on Ubi-
Brown DJ, Suckow B, Wang T, A Survey of Intrusion Detection Systems. Depart- Media Computing; 2008: pp. 280–4.
ment of Computer Science, University of California, San Diego; 2002. Lei L, Yang D-Z, Shen F-C. A Novel rule based Intrusion Detection system using
Bakshi A, Yogesh, B. Securing cloud from DDOS attacks using intrusion detection Data Ming. 3rd IEEE International Conference on Computer Science and
system in virtual machine. In: Second international conference on commu- Information Technology 2010;6:169–72.
nication software and networks; 2010: pp. 260–4. Li H, Liu D. Research on intelligent intrusion prevention system based on snort.
Botha M, Solms R, Perry K, Loubser E, Yamoyany G. The utilization of artificial International Conference on Computer, Mechatronics, Control and Electronic
intelligence in a hybrid intrusion detection system. SAICSIT 2002:149–55. Engineering (CMCE), 2010;1:251–3.
Beg S, Naru1 U, Ashraf M, Mohsin S. Feasibility of intrusion detection system with Li W. A genetic algorithm approach to network intrusion detection.USA: SANS
high performance computing: a survey. International Journal for Advances in Institute; 2004.
Computer Science 2010;1(1). Lu W, Traore I. Detecting new forms of network intrusion using genetic program-
Chen Y, Sion R. On securing untrusted clouds with cryptography. In WPES 2010;10: ming. Computational Intelligence 2004;20(3):475–94.
109–14. Lee, J-H, Park M-W, Eorn J-H, Chung T-M. Multi-level Intrusion detection system
Cannady J. Artificial neural networks for misuse detection, National Information and log management in cloud computing. In: 13th International conference on
Systems Security Conference, 1998. advanced communication technology (ICACT); 2011, pp. 552–5.
Chavan S, Shah K, Dave N, Mukherjee S, Adaptive neuro-fuzzy intrusion detection Leu FY, Li ZY. Detecting DoS and DDoS attack using an intrusion detection and
systems, IEEE international conference on information technology: coding and remote prevention system. Fifth International Conference on Information
computing (ITCC’04); 2004: pp 70–4. Assurance and Security 2009;2:251–4.
Chen W-H, Su S-H, Shen H-P. Application of svm and ann for intrusion detection. Mell P, Grance T, The NIST definition of cloud computing (draft), NIST, /http://
Computer Oper Res 2005;32(10):2617–34. csrc.nist.gov/publications/drafts/800-145/Draft-SP-800-145_cloud-definition.
Dutkevyach T, Piskozub A, Tymoshyk, N. Real-time intrusion prevention and pdfS ; 2011.
anomaly analyze system for corporate networks. In: Fourth IEEE workshop Martin L, White Paper, /http://www.lockheedmartin.com/data/assets/isgs/docu
on intelligent data acquisition and advanced computing systems: technology ments/CloudComputingWhitePaper.pdfS; 2010.
and applications, 2007. IDAACS 2007: 2007: pp. 599–602. Mazzariello C, Bifulco R, Canonoco R. Integrating a network IDS into an open
Dastjerdi AV, Bakar KA, Tabatabaei SGH. Distributed intrusion detection in source cloud computing. In: Sixth international conference on information
clouds using mobile agents. In: Third international conference on advanced assurance and security (IAS); 2010; pp. 265–70.
 C. Modi et al. / Journal of Network and Computer Applications 36 (2013) 42–57 57

Moradi M, Zulkernine M, A neural network based system for intrusion detection Vieira K, Schulter A, Westphall C, Westphall C. Intrusion detection techniques in
and classification of attacks. In: Proceedings of the 2004 IEEE international grid and cloud computing environment. IEEE IT Professional Magazine 2010.
conference on advances in intelligent systems—theory and applications; 2004. Xiao T, Qu G, Hariri S, Yousif M. An efficient network intrusion detection method
NIST: National vulnerability database, Website, Available from: /http://web.nvd. based on information theory and genetic algorithm. In: Proceedings of the
nist.gov/view/ vuln/detail?vulnId=CVE-S2009-3733; 2011. 24th IEEE international performance computing and communications con-
Opennebula, Website, /http://www.opennebula.orgS; 2011. ference (IPCCC ‘05), Phoenix, AZ, USA; 2005.
Rutkowska J, Subverting VistaTM Kernel for Fun and Profit, Black Hat Conference; Zhengbing H, Jun S, Shirochin VP. An intelligent lightweight intrusion detection
2006. system with forensic technique. In: 4th IEEE workshop on intelligent data
Roschke S, Feng C, Meinel C. An extensible and virtualization compatible IDS acquisition and advanced computingsystems: technology and applications,
management architecture. In: Fifth international conference on information 2007. IDAACS; 2007: pp. 647–51.
assurance and security, 2; 2009: pp. 130–4. Zhengbing H, Zhitang L, Jumgi W, Novel A. Intrusion detection system (NIDS)
Ram S. Secure cloud computing based on mutual intrusion detection system. based on signature search of datamining, WKDD First International Workshop
International journal of computer application 2012;2(1):57–67. on Knowledge discovery and Data Ming; 2008: pp. 10–6.
Slaviero M. BlackHat presentation demo vids: Amazon, /http://www.sensepost. cox P. Intrusion detection in a cloud computing environment. /http://searchcloud
com/blog/3797.htmlS; 2009. computing.techtarget.com/tip/Intrusion-detection-in-a-cloud-computing-en
Sequeira D, Intrusion Prevention Systems- Security’s Silver Bullet? SANS Institute vironmentS; 2011.
InfoSec Reading Room 2002, /http://www.sans.org/reading_room/whitepa Firewall, Telecom-Network Tech,/http://teleco-network.blogspot.com/S; 2011.
pers/detection/intrusion_prevention_systems_securitys_silver_bullet_366? Denial-of-service attack, Website, /http://en.wikipedia.org/wiki/Denial-of-servi
show=366.php&cat=detectionS; 2002. ce_attackS; 2011.
Su M-Y, Yu G-J, Lin C-Y. A real-time network intrusion detection system for large- Snort-Home page, Website, /https://www.snort.org/S; 2011.
scale attacks based on an incremental mining approach. Computer Security The concept of Intrusion Detection System, Website, /http://maltainfosec.org/
2009:301–9. archives/26-The-concept-of-Intrusion-Detection-Systems.htmlS (2011).
Sandar SV, Shenai S. Economic denial of sustainability (EDoS) in cloud services XArp 2.2.2, Website, /http://www.filecluster.com/Network-Tools/Network-Moni
using HTTP and XML based DDoS attacks. International Journal of Computer toring/Download-XArp.htmlS; 2011.
Applications 2012;41(20):11–6. IBM Research-Zurich, Website, /http://www.zurich.ibm.com/csc/security/secure
Scarfone K, Mell P, Guide to intrusion detection and prevention systems (IDPS), virt.html#topS; 2011.
Recommendations of the National Institute of Standards and Technology, IPS: Intrusion Prevention System. Javvin, Website,/http://www.javvin.com/net
/http://csrc.nist.gov/publications/nistpubs/800-94/SP800-94.pdfS; 2007:175–180 worksecurity/IPS.htmlS; 2011.
457–471. stiawanD, Abdullah, AH, Idris, MY.The trends of intrusion prevention system
Tillapart P, Thumthawatworn T, Santiprabhob P. Fuzzy intrusion detection system. network. In: Second international conference on education technology and
Assump University J Technology (A.U. J.T.) 2002;6(2):109–14. computer (ICETC) 4; 2010: 217–21.