Comparison of Tools and Simulators for

Control System Security Studies

Jun Wu, Kazukuni Kobara

Research Institute for Secure System (RISEC)
National Institute of Advanced Industrial Science and Technology (AIST)
Tsukuba, Japan
jun-wu@[Link], kobara_conf@[Link]

Abstract—Security is one of the key concerns for control purpose of testing is often viable for testing network security,
systems. Tools and simulators for the control system security but would be prohibitively expensive for testing complex
have attracted a lot of attentions recently, because testing new infrastructure establishment. Instead, the complexity of
solutions is a very important work for the control system security SCADA systems calls for a thorough efficient simulation to
studies. In this paper, we firstly analyze the challenges of the help test the benefits and consequences of novel security
security simulations for control system. Then, classification study solutions.
is done for the existing tools and simulators. The implementation
principles the typical simulator examples and security examples Recently, various kinds of tools and simulators [2]-[11] are
are also analyzed. Finally, we compare the important capabilities used for testing the security of related control system [12]-[22].
and characteristic of the tools and simulators, which is valuable Scientific analysis and comparison of the tools and simulators
to efficient simulations for security studies in control systems. for control system security studies are needed, which and
provide an efficient reference method for using the tools to test
Keywords—security, control system, SCADA, simulator, test the control system security.
bed
In this paper, we analyze the tools and simulators used for
I. INTRODUCTION control system security studies. The rest of the paper is
Industrial control systems form the backbone of countless organized as follows. Sect. II analyzes the design issues of
industries affecting nearly every basic service modern society simulators for control system security. Sect. III presents the
requires. The computer systems used to monitor and control classification studies of the simulators and tools. The
major infrastructure are known by various names, among the comparison of these tools and simulators are shown in Sect. IV.
most common Supervisory Control and Data Acquisition Finally, Sect. V concludes this paper.
(SCADA) systems. Many of them control nations’ critical
II. ISSUES IN SIMULATION OF CONTROL SYSTEM SECURITY
components, like nuclear power generation, public transport,
wastewater plants and so on [1]. The typical architecture of a control system and its security
simulation issues are shown in Fig. 1. In control systems, the
Securing these control systems is a complex and difficult process is the physical phenomenon that operators seek to
task and little attention has been paid on it in the past. In light control. This portion of the system will be distinct in all
of the growing prevalence of cyber attacks on the computer SCADA systems. The control center acts as the master
networks and systems in the infrastructure of major nations, an controller, maintaining the high level operation of the process.
understanding of the vulnerabilities of industrial control The control center sends control commands and receives sensor
systems and an investigation of appropriate and effective updates from the field devices. The connection between the
mitigation techniques is of vital importance. control center and field devices is provided by the control
The differences in SCADA security and traditional IT network. This may be a wired or wireless network and may
approaches mean that even reliable and trusted solutions cannot operate with a variety of network [Link] schemes
be applied in real control systems without significant testing. for control systems have basic properties that separate them
As a matter of fact, whether cyber or physical, the studies of from conventional distributed systems. This section discusses
control system security can be carried out by related simulators. the design issues of the simulation for control system security
The potential damage of malfunctions and loss of availability and the potential effect of the issues on performance.
of critical infrastructure further necessitate thorough testing A. Closed Relations with Control Device Actions
based on simulation. However, testing new security solutions
for SCADA systems is not easily and simply performed. The security events in control systems are have closed
Developing parallel and realistic simulation systems for the relations with the control device actions. In other words, the
security schemes are usually performed for protecting the

978-1-4673-0311-8/12/$31.00 ©2012 IEEE 45

978-1-4673-0311-8/12/$31.00 ©2012 IEEE
information of the key device actions in control systems. For III. CLASSIFICATION STUDY OF SIMULATION TOOLS
example, in order to ensure the security actions of a circuit In this section, we give the detailed classification analysis
breaker in a power control system, security schemes should be of the tools and simulators for control system security studies.
designed for the breaker actions [23]. Therefore, the security Based on the implementation architecture and deployment
solutions and its simulation for this case must satisfy the purposes, we classify the tools and simulators into four
requirement of the action. different types, which are single simulation for devices,
B. Real-Time Requirements network simulation, integration simulation, and experiment-
based simulators.
In many control systems, the real-time characteristic is a
very important issue. For instance, the control of electrical A. Single Simulation Tool
grids and oil and gas pipelines involves aggregating sensor The single simulation tool is performed for the devices in
measurements from hundreds of widely dispersed field devices control system, such as controllers, plants, control centers, etc.
so that operators use a centralized control interface to manage It can perform the simulation for verifying the efficiency and
the whole process in real time [24]. Depending on the process, validity of the security events and actions which is
it may be important to provide real time guarantees during the implemented in an individual device in a control system. For
security simulation. example, the individual components of the simulation can be
C. Variety and Complexity of Control Devices implemented as Simulink subsystems [2] that include the plant
There are various kinds of control systems for different simulation, sensor simulations, simulations for the data
deployment puroposes, such as power control, oil and gas acquisition and control activities on the remote terminal unit
pipelines monitor, etc. The characteristics of these control (RTUs), simulation of the computations performed on the
systems, including architecutres, topology, security SCADA servers, etc.
requirements etc., are different. On the other hand, there are
Features Parameters
different kinds of devices in the control systems, such as of Device Action
SCADA master, controller, field devices, sensors, actuators, etc. Mathematical Model Computing Modeling
The characteristics of these devices are also different. During Features Parameters
Of the Device (i. e. State Graph)
the seuciry simulation, variety and complexity of the systems of Device
and devices should be considered.
D. Heterogeneous Data Security Proposal for
Device Simulation
Interaction data types are distinct from objects because they Device Combining Result
Model
with Its Action
do not persist in the federation, but instead are shared between
federates at a single point in time [25]. In order to facilitate
Figure 2. Security Simulation based on Single Simulator
interoperability and reuse, the means and type of data that will
be exchanged by the various kinds of devices and domains. The principle of the security simulation based on single
E. Different Simulators of Heterogeneous Domains simulator is illustrated in Fig. 2. Because the security of control
devices have closed relations with device actions, such as the
In security simulation for control systems, there are breaker actions in a power control system, the mathematical
heterogeneous models for different domains: plant models, model of the device must be established firstly based on the
network models, controller models, etc. For example, the features parameters of the device and its typical actions. The
simulators for the sensor devices and the simulators for the mathematical model can be made based on related computing
control netwroks are in different domains. How to integrate modeling method, such as state graph. Then, the security
seamlessly these simulators is very important for the simulation proposals for related devices can be added and the device
of control system security. security simulation model can be got. These security studies
include the verification of effects of new and/or existing
Control System Issues
mechanisms for user access control, secure actions, fault
Control Center/
Remote Access
tolerance, sever node based intrusion detection, etc.
Closed Relations with
Device Actions In the security studies in [12], [13], Simulink and/or
Real-Time Requirements
Stateflow are used to perform the plant simulation, sensor
Control Networks
simulations, simulations for the data acquisition and control
Security
Variety and Complexity activities on the RTUs, simulation of the computations
Simulation of Devices
performed on the SCADA servers, etc. SimEvents from
Controllers
Heterogeneous Data
Mathworks can also be applied to perform the simulation in
control system [13]. Distributed attack simulation in a network
Actuators Sensors
system [14] can be simulated based SimEvents.
Different Simulators of
Heterogeneous Domains
Process Here we take Matlab Simulink combining with SimEvents
as the example to explain the implementation details of security
Figure 1. Control Systems and Their Security Simulatin Issues simulation. All the security solutions for control system are
regarded as a sequence of security events. Fig. 3 shows the
main functional components of the overall architecture. As a

46
Discrete Event Systems (DES) simulation engine, SimEvents is The principle of security simulation is shown in Fig. 4. At
driven by an Event Calendar where all future security events to first, the feature parameters of the network are obtained from
occur are listed in ascending order of their scheduled time. the control networks. Then the network element model can be
SimEvents always processes the first security event in this list established, which can be described by the main features of the
and updates the DES state accordingly. When such an event control network. Next, the security proposals added to the
presents, the Cooperative Event Driver translates it into a network model to establish the security simulation model.
Simulink signal. Then the Data Exchange module passes the Usually, the network security schemes are proposed in one or
signal to Simulink so that it may trigger a time driven process several layers of the reference 7-layer model. The security
or update various model parameters. In contrast, as a time- schemes for end to end data are proposed in upper layer
driven process evolves under the control of Simulink, it may (application layer, presentation layer and session layer) in the
generate security events in the form of level-crossing points OSI reference network model. On the other hand, the security
(from above, from below, or either) that the Data Exchange schemes for end to end segments, packets, frames, and bits are
module appropriately translates so they may be processed by proposed in media layer (transport layer, network layer, data
SimEvents blocks. The most challenging aspect of coordinating link layer and physical layer). This security scheme can be
time-driven and event-driven dynamics is that of proper timing. based on the basic technologies, such as encryption, integrity
In the architecture of Fig. 3, the system “clock” is maintained check, machine learning, wireless communication technologies,
by Simulink and the Cooperative Event Driver is responsible etc. Security proposals are added into the network model by
for ensuring consistency between Simulink blocks and adding these basic technologies to corresponding layer.
SimEvents blocks which interact with the Security Event
Calendar. Here we take ISA100.11a for example to explain the
security solutions in the OSI reference network model of
control networks. ISA100.11a is an open wireless networking
technology standard developed by the International Society of
SimEvents State Security Event Automation (ISA). There is a two-layer network security for
Dynamics Calendar ISA100.11a. These levels are inherited from the security
policies supported by IEEE 802.15.4, the underlying wireless
technology on which ISA100.11a is based. Various levels of
Data Cooperative Matlab authentication and encryption can be enabled for both layers.
Exchange Event Driver The ISA100.11a standard uses state-of-the-art encryption based
on AES-128 block ciphers. In order to provide protection from
a variety of attacks, the ISA100.11a standard employs time
Simulink State Simulink Engine
stamps in its security by including it in the nonce needed for
Dynamics (Clock)
the AES-128 encryption engine. In ISA100.11a, symmetric
keys are used for data encryption and authentication, while
Figure 3. A Simulation Framework based on Matlab Simulink/SimEvents asymmetric keys can be used for the join process.

B. Network Simulation Tool Device Data Security

Thread 1 DLL
Node 1 Segments Events
In control system, the control and sensing data must be
transmitted based on the wired or wireless control networks.
Network simulators can be used to test the security designs of
the data and/or network in control systems. Device
Thread 2
Data
DLL
Security
OMNeT ++ Node 2 Segments Events
Discrete
Simulator
Application Security Topology, Protocol,
...

Application, Network
Link, etc. of Control
Presentation Security Network
Device Data Security
Thread n DLL
Session Security Node n Segments Events
Network Element
Model
Transport Security Figure 5. A Simulation Framework based on OMNeT++
Protection of End
Network Security to End Data, Network Network simulator can test the overhead and/or
Segments, Packets, Simulation Model
Frames, and Bits
performance of the security solutions in control system, such as
Data Link Security authentication protocols, data secrecy, secure routing, and
network based intrusion detection, etc. In the security studies in
Physical Security Results [12], [15], OMNeT++ [3] is applied as the network simulation
of control systems. A key benefit of such a tool is its ability to
Figure 4. Security Simulation based Network Simulator test the effects of attacks on a real control process (NXT

47
devices) using realistic SCADA protocols (Modbus/TCP) [6]. Then the whole control system can be simulated to verify its
Other network simulators, such as OPNET [4], .NET Remoting security.
[5], Modbus simulator [6], and are used in the security studies
in [18], [19], and [20]. GME Integration Model

In order to explain the network security simulation in detail,

here we take a security simulation method based on OMNeT++ Generate .m Receiver and
for example. A simulation framework based on OMNeT++ is Original Simulink Sender S-function code
illustrated in Fig. 5. Various kinds of device nodes can be Security Model
simulated in OMNeT++ which can support distributed
Generate Java Code for
simulation. The method in this case is abstract every node as a Representing Simulink
thread. Then the codes which perform the security actions are Input and Output Binding Federate
called as dynamic link library (DLL) into the corresponding
thread. The DLL code is compiled based on underlying drivers
of discrete event simulation of OMNeT++. The global HLA Run-Time Infrastructure (RTI)
variables of the DLL can be stored in the data segments.
Figure 7. Simulink Integration
C. Integration Simulation Tool
In security simulation for control systems, there are We take Simulink as the example of single simulator and
heterogeneous models for different domains: plant models, OMNeT++ as an example of network simulator for analyzing
network models, controller models, etc. If the whole control the principle of integration. As shown in Fig. 7, original
system of SCADA needs to be simulated, an overarching Simulink model can be integrated into HLA-RTI architecture
integration model is a must which connects and relates the through adding the input and output binding. For input and
heterogeneous domain models in a logically coherent output binding, .m Receiver and Sender S-function code should
framework. An underlying software infrastructure is needed be generated based generic modeling environment (GME).
that connects and relates the heterogeneous simulators in a Next, Java code should be generated for representing Simulink
logically and temporally coherent framework. federate. Thus the Simulink model can send and receive signal
flow via HLA-RTI.
The integration principle of OMNeT++ is shown in Fig. 8.
HLA Compliance Object Model Interface The most important challenges for OMNeT++ integration are
Rule Template Specification
time management and protocol mapping. Time policy controls
the rate at which a federate requests time advances from the
RTI and the timing of the receipt of RTI events. Protocol
Security
Simulator 1
Security
Simulator 2 ... Security
Simulator n mapping provides a set of protocols with HLA mapping. The
RTI Interface RTI Interface RTI Interface
problem of time management can be resolved through
Integration
replacing OMNeT++ schedule, scheduler-module interactions
can be established. On the other hand, the basic idea for
RTI protocol integration is to keep low level message inside
Control System Environments OMNeT++ and provide high level application layer interface
provided for HLA.
Figure 6. Security Simulation based Integration Simulator
Time Integration Protocol Integration
High level architecture (HLA) based on runtime
infrastructure (RTI) can be used for this kind of simulation.
The basic principle of integration simulation is shown in Fig. 6.
Replace OMNeT++ Keep Low Level Message
HLA is a public and common distributed framework for
Schedule inside OMNeT++
simulation, which is involved in IEEE 1516 standard. In HLA,
all the simulation components are called Federate. Then the
simulation environment based on different Federate is Establish Scheduler- High Level Interface for
Federation. There are three important parts in the HLA, which Mmodule Interaction HLA Mapping
are HLA compliance rule, object model template (OMT), and
interface specification. HLA compliance rule is a set of rules of
design and management for federation. Object model template HLA Run-Time Infrastructure (RTI)
(OMT) provides the presentation template between the real
events and event models, which act as the common standards Figure 8. OMNeT++ Integration
for interactions among federates. Interface specification defines
the service functions for RTI, which can perform the The C2WindTunnel (C2WT) [7] is an existing example,
interoperations and managements for different kinds of which has been constructed so that it can serve as an integration
simulations. Based on this integration method, different kinds test bed for computational experimentation. The basic
of security event simulation (i. e. single simulation, network framework of C2WT is also HLA-RTI. For constructing C2WT,
simulation, etc.) can be integrated in a uniform framework. HLA Run-Time Infrastructure (RTI) is used to integrate

48
Simulink, Colored Petri Net, network models (i. e. OMNeT tool can be established. The security studies for different
Discrete Event Simulation) and 3D visual sensor simulator. subsystems can be tested in this tool.
D. Realistic Test Bed Tool In the security studies in [12], [21], [22], RINSE [8],
Some test bed tools not only consist solely of the simulation Emulab [9], OPSAID [10], and DETER [11] are used to
model but also include modules to setup, run, rerun and analyze construct the realistic test bed.
experiments. This kind of tool uses some actual commercial
SCADA devices along with implementations of the software
modules performing the data processing (running on realistic iSSFNet Network
Simulator
hardware), emulations of the network, and real-time
simulations for the plant (running on dedicated, high-
performance hardware). Such a simulation is feasible and could Network Viewer
Database
be made highly realistic and scalable. Clients

Control System
Internet

Subsystem
Model 1
... Subsystem
Model i
Subsystem
Model i+1
... Subsystem
Model n
Data Server Simulator Database
Manager

Simulator Simulator Device Set Device Set

RINSE Backup
Instance

Interactions among Different Subsystems Figure 10. RINSE Architecture

Realistic Test Bed Tool

In order to analyze the details, we take RINSE for example.
As shown in Fig. 10, RINSE consists of five components: the
iSSFNet network simulator, the Simulator Database Manager, a
Test Security Studies at Each Subsystem database, the Data Server, and client-side Network Viewers.
Each simulation component connects independently to the
Figure 9. Realistic Test Bed Tool Simulator Database. On the user/player side, the Data Server
interfaces with client applications, which allows the user to
The principle of realistic test bed tool is shown in Fig. 9. monitor and control the simulated network. The Data Server
The control system usually can be divided into several performs authentication for each user, distributes definitions of
subsystem models. Different security schemes can be proposed the client’s view of the network (using the Domain Modeling
for these subsystems. Then some of the subsystems can be Language), and provides a simple way for the client
simulated in the corresponding simulators, and the other applications to access new data in the database through XML-
subsystem can be implemented in the real devices. Through the based remote procedure calls. The Data Server responds with
interactions among different subsystems, a realistic test bed new data for each client, extracted from the database.

TABLE I
COMPARISON OF SIMULATORS FOR CONTROL SYSTEM SECURITY
Real-time simulation
Classification Complexity Realistic capability Simulated security events Related examples
performance

User access control, secure actions,

Single Matlab Simulink,
simulator Low Middle Middle fault tolerance, sever node based
SimEvents, StateFlow
intrusion detection, etc.

Authentication protocols, data OMNET++,

Network
Low Middle Middle secrecy, secure routing, network OPNET, .NET Remoting,
simulator
based intrusion detection, etc. Modpoll Diagslave

Integrate different kinds of security

Integration
Middle High Low simulations on devices, data, and C2WT
simulator
networks

Realistic test Relate security schemes design for RINSE, Emulab, DETER,
bed tool High High High
devices, networks, data, etc. OPSAID

49
 IV. COMPARISON [2] MathWorks Simulink. [Online]: Available: [Link]
[3] OMNeT++. [Online]: Available:[Link]
Based on aforementioned classification and analysis, we
[4] OPNET network simulator. [Online]: Available: [Link]
compare the simulators and tools for control system security
[5] B. Vanhooff, D. Preuveneers, and Y. Berbers, “.NET Remoting and Web
studies from the view of complexity, realistic capability, real- Services: A Lightweight Bridge between the .NET Compact and Full
time simulation performance simulated security event, as Framework,” Journal of Object Technology, vol. 5, no. 3, pp. 59-81, Apr.
shown below and in Table I. 2006.
[6] Modpoll Modbus Master Simulator. [Online]:
Complexity: This factor indicates the complexity of the [Link]
simulation, including the software/hardware environment [7] A. Davis, “Developing SCADA Simulations with C2WindTunnel,”
establishment, measurements, etc. Single simulators and Master Dissertation, Vanderbilt University, May 2011.
network simulators are relative simple, because they are always [8] M. Liljenstam, J. Liu, D. Nicol, Y. Yuan, G. Yan, and C. Grier, “Rinse:
performed as software simulation in the computer. Integration The Real-Time Immersive Network Simulation Environment for
simulator is relative complex because of the interaction design Network Security Exercises,” in Proc. of IEEE PADS 2005, Jun. 2005.
between different simulators. Realistic test bed tools are the [9] Emulab. [Link]
most complex because they are usually involve the software [10] S. A. Hurd, J. E. Stamp, and A. R. Chavez, “OPSAID Initial Design and
and hardware design for the real systems. Testing Report,” Report, Sandia National Laboratories, Nov. 2007.
[11] The DETER Testbed. [Online]: [Link]
Realistic Capability: This feature denotes the level of [12] A. Giani, G. Karsai, T. Roosta, A. Shah, B. Sinopoli, and J. Wiley, “A
realistic of the simulation. Essentially, single simulators, Testbed for Secure and Robust SCADA Systems,” in Proc. of IEEE
network simulators, and integration simulators are software RTAS 2008, Jul. 2008.
simulators which only are performed in the computers. [13] J. Nutaro, P. T. Kuruganti, M. Shandar, L. Miller, and S. Mullen,
Realistic test bed tools can present well the physical truth of the “Integrated Modeling of the Electric Grid, Communications, and
control systems. Control,” International Journal of Energy Sector Management, vol. 2, no.
3, pp. 420-438, Mar. 2008.
Real-Time Simulation Performance: This factor indicates [14] G. Khazan, M. A. Azgomi, “A Distributed Attack Simulation for
the simulation capability of real-time performance. Basically, Quantitative Security Evaluation using SimEvents,” in Proc. of
the realistic test bed tool has the best simulation capability of IEEE/ACS AICCSA 2009, May, 2009.
real-time. On the other hand, because of the integration among [15] R. Chabukswar, B. Sinopoli, G. Karsai, A. Giani, H. Neema and A.
different simulators, the delay of the integration simulator is Davis, “Simulation of Network Attacks on SCADA Systems,” in Proc.
of TRUST SCS 2010, Apr. 2010.
relative longer than other simulators.
[16] C. Queiroz, A. Mahmood, J. Hu, Z. Tari, and X. Yu, “Building a
Simulated Security Event: Different kind of simulators SCADA Security Testbed,” in Proc. of NSS 2009, Oct. 2009.
and tools are used for different security event simulations. [17] S. Hu, Z. Zhao, Y. Zhang, and S. Wang, “A Novel Modbus RTU-based
Single simulators are usually used to simulate security schemes Communication System for Adjustable Speed Drives,” In Proc. of VPPC
2008, Spet. 2008.
in devices. Network simulators are usually used to do
[18] M. Mallouhi, Y. Al-Nashif, D. Cox, T. Chadaga, and S. Hariri, “A
simulation for security schemes for data and/or network. Testbed for Analyzing Security of SCADA Control Systems
Integration simulators can be applied for simulating the whole (TASSCS),” In Proc. of IEEE ISTG 2011, Dec. 2011.
control system including different kinds of devices. Realistic [19] B. Genge, I. N. Fovino, C. Siaterlis and M. Masera, “Analyzing Cyber-
test bed tools can provide a relative real environment to Physical Attacks on Networked Industrial Control Systems” in Proc. of
simulate all kinds of security selutions for control systems. 5th IFIP ICCIP 2011, Mar. 2011.
[20] J. Rrushi and R. Campbell, “Detecting Attacks in Power Plant
V. CONCLUSION Interfacing Substations through Probabilistic Validation of Attack-Effect
Bindings,” in Proc. of S4 2008, Jan. 2008.
In this paper, various properties of the tools and simulators
[21] C. M. Davis, J. E. Tate, H. Okhravi, C. Grier, T. J. Overbye, and D.
for control system security were discussed. Firstly, issues of Nicol, “SCADA Cyber Security Testbed Development,” in Proc. of
security simulation for control systems were presented. Then, NAPS 2006, Spet. 2006.
according to the important characteristics or the tools and [22] D. C. Bergman, D. Jin, D. M. Nicol, and T. Yardley, “The Virtual Power
simulators, we classified them into four types, which are single System Testbed and Inter-Testbed Integration,” in Proc. of CSET 2009,
simulation tool for devices, network simulation tool, Aug. 2009.
integration simulation tool, and realistic test bed tool. Next, we [23] N. Liu, J. Zhang, H. Zhang, and W. Liu, "Security Assessment for
analyzed the implementation principle and gave typical Communication Networks of Power Control Systems Using Attack
Graph and MCDM," IEEE Transactions on Power Delivery, vol. 25, no.
examples of each kind of simulator for security studies. Finally, 3, pp. 1492-1500, Jul. 2010.
according to the issues for control system security simulation, [24] R. E. Young, “Petroleum Refining Process Control and Real-Time
we compared the important features of the tools and simulators, Optimization,” IEEE Control System, vol. 26, no. 6, pp. 73-83, Dec.
including complexity, realistic capability, real-time simulation 2006.
performance and simulated security events. This paper provides [25] M. L. Jørgensen, “Analysis and Enhancement of Safety-Critical
a good reference for doing efficient simulations for security Communication for Railway Systems,” Master Dissertation, Aalborg
studies in control systems. University, Aug. 2008.

REFERENCES
[1] M. Brandle and M. Naedele, “Security for Process Control Systems: An
Overview,” IEEE Security and Privacy, vol. 6, no. 6, pp. 24-29,
Nov./Dec. 2008.

50

Powered by TCPDF ([Link])