dsgffd (1)
dsgffd (1)
Reference
Architecture for
Integrating OT and
Modern IT
Paul DeBeasi
31 July 2023
Reference Architecture for Integrating OT and
Modern IT
Published 31 July 2023 - ID G00792965 - 36 min read
By Analyst(s): Paul DeBeasi
Initiatives: Infrastructure for Technical Professionals
Overview
Key Findings
■ The hierarchical levels and point-to-point connections of the Purdue Model require
developers to integrate a disjointed set of incompatible, vendor-defined APIs. The
result is a proliferation of incompatible interfaces that make it difficult for
developers to discover, access and analyze business data trapped in disparate
operational technology (OT) systems.
■ The Purdue Model assumes that architects isolate OT systems from IT systems. The
IT-OT isolation creates zones of trust that protect OT assets. However, the Internet of
Things has blurred IT-OT boundaries, broken down zones of trust and exposed
attack surfaces.
Through 2025, 70% of companies will deploy cyber-physical systems protection platforms
as the first step in their asset-centric security journey.
Analysis
Data pipelines in industries such as manufacturing, energy and utilities have become
critical to supporting diverse, complex and mission-critical business processes. But many
organizations struggle to find, distribute and analyze their OT data. Even when OT
systems do share data, the data is often unstructured, fragmented and real-time. Pressure
to deliver faster results, higher quality and greater resiliency is driving organizations to
consider how they can integrate OT with IT.
Some organizations are attempting to address these issues. They are deploying modern
IT technologies, such as MQTT event brokers and security solutions that automate OT
asset discovery and protection. Some are embracing the Sparkplug B standard and a
hierarchical, ISA-95-inspired data model called the Unified Namespace. They are also
deploying platforms such as data hubs and Industrial Internet of Things (IIoT) platforms.
Most organizations are at the early stages of this industrial transformation, and thus have
limited experience making these changes.
■ Unified Namespace
The Purdue model originated as a generalized distributed control architecture. The model
designers never intended to create an industrial security architecture. They assumed that
OT systems would remain isolated from IT systems and thus remain secure. Designers
used firewalls to form trust zones between model layers. This approach became the gold
standard for industrial security.
However, the Internet of Things (IoT) changed all that. For example, modern sensors and
devices can generate data streams and send those streams directly to the cloud,
bypassing trust zones. As a result, the IT-OT boundaries have blurred. Industrial
architects are reluctantly coming to the realization that they must apply zero-trust
security principles to their OT assets.
The reference architecture described herein proposes a specific solution architecture. The
primary value of this architecture is that it improves the ability of industrial organizations
to discover, secure, share and analyze their OT data. The architecture integrates the
industrial edge (aka “the core”) with cloud computing (see Figure 2).
The industrial edge represents the physical plant. It contains industrial devices (e.g.,
robotic arms), OT systems (e.g., SCADA systems), and industrial networks (e.g., Modbus).
The industrial edge also includes IT systems (e.g., machine learning inference engines),
enterprise applications (e.g., ERP), and enterprise networks (e.g., Wi-Fi).
The cloud represents the applications running in the cloud. Some of these applications
(e.g., the IIoT platform and CPS-PP) interact with the industrial edge systems (e.g., the IIoT
gateway and CPS data collectors). Other applications analyze edge data that has been
transferred to the cloud (e.g., ML models and digital twins).
The industrial edge connects to the cloud over a secure connection using a variety of
technologies (IPsec, VPN, SD-WAN, etc.). The edge-to-cloud transport protocols also vary
considerably. They are dependent upon which edge and cloud applications are
communicating with each other and which protocols they use (HTTPS, MQTT, FTP, etc.).
■ Data Hub: The data hub accesses OT data at the edge and shares it with
applications in the cloud. The hub normalizes data formats and maintains data
flows between applications. Data hubs use a variety of methods to transfer data
from the edge to the cloud (Apache Kafka, MQTT messages, file transfer, etc.)
■ CPS Protection Platforms and Data Collectors: CPS-PPs discover and protect cyber-
physical assets. CPS data collectors send asset data to the CPS protection platform,
located in the cloud or at the edge.
MQTT Broker
The proliferation of event-centric architectures and stream processing technologies have
enabled new integration patterns that complement many of the familiar data integration
methods (e.g., extract transform and load). One of the more common patterns is the
event-centric integration pattern. This pattern forms the foundation of the industrial
reference architecture.
For example, in Figure 3, an edge node collects industrial data from a robotic arm
controller. The edge node creates an MQTT event message, copies the industrial data into
the MQTT payload and transmits the MQTT message to the MQTT broker using a MQTT
topic. The MQTT broker forwards the message to the five applications that subscribed to
that MQTT topic (refer to the MQTT Broker section for a description of topics). In this
figure, the edge node is the source/publisher, and the five applications are the
sinks/subscribers.
Compare Figures 1 and 3. In Figure 1, the SCADA system communicates with the data
historian over a point-to-point connection using a vendor-defined API. Every new SCADA-
to-device integration requires a new point-to-point connection. In Figure 3, the SCADA
system communicates with the data historian using a publish-subscribe protocol via the
MQTT broker. The SCADA system needs only one connection to the MQTT broker,
regardless of the number of interconnected devices. This approach is easier to scale and
maintain than the Purdue Model approach. For more information on event-centric
integration patterns, refer to:
A key component of the event-centric integration pattern is the event broker. The broker
moves messages from event producers to event consumers. The event broker decouples
producers and consumers in time, space and synchronization. The decoupling enables
technical professionals to add producers and consumers without impacting the rest of the
system, thus improving design flexibility and system scalability.
Note that 5.0 client-to-client features (e.g., user properties) require a 5.0 client for both
publisher and subscriber, whereas 5.0 client-to-broker features (e.g., enhanced
authentication) do not. MQTT brokers must support a mix of MQTT 3 and MQTT 5.0
clients to enable a graceful migration to MQTT 5.0. Check with your MQTT supplier to
ensure this is the case.
Gartner recommends placing the MQTT broker behind the OT firewall, because doing so
requires only one port to be open in the OT firewall, regardless of the number of publishers
and subscribers (see Figure 2). The broker enables communication between OT systems
and enterprise applications.
The broker uses access control technology to explicitly control which clients can
communicate with the broker, as part of a move to zero trust. MQTT 3.0 uses the simple
exchange of username/password for client authentication. MQTT 5.0 uses the new AUTH
packet to authenticate a client by presenting a challenge that the client must respond to.
This enables organizations to use security protocols such as Kerberos or salted challenge
response authentication mechanism (SCRAM).
Some hyperscaler IoT platform vendors have integrated event brokers into their IoT
gateway software (e.g., Amazon Web Services [AWS] IoT Greengrass, Microsoft Azure IoT
Edge). Data hub vendors have also embedded event brokers into their hubs (e.g.,
HighByte, Cogent). If you choose to deploy multiple event brokers from different vendors,
then you must ensure the brokers support the same messaging protocol, topic structure
(see Unified Namespace) and message encoding scheme (see Sparkplug B). For
additional event broker information, refer to Choosing Event Brokers: The Foundation of
Your Event-Driven Architecture.
■ MQTT Essentials: The Ultimate Guide to MQTT for Beginner and Experts
Data Hub
Data and analytics teams cannot achieve the project deployment speed they desire
because too many roles, too much complexity and constantly shifting requirements make
it difficult. In most organizations, this complexity is exacerbated by limited or inconsistent
coordination across the roles involved in building, deploying and maintaining industrial
data pipelines. DataOps techniques can address these challenges through a more agile,
collaborative and change-friendly approach to building and managing data pipelines.
DataOps is about a more efficient way of working when delivering data and analytics
solutions. Applying techniques adapted from the DevOps concepts (see Figure 5), which
many organizations have leveraged in implementing applications, better communication
and tighter collaboration results in faster deployments and greater effectiveness in
reacting to change postdeployment. With the increasing awareness of DataOps concepts
and terminology, organizations are looking for the best ways to introduce these ideas into
their operations teams.
For many, DataOps represents a massive shift in approach, raising substantial change
management concerns. Leaders who seek to increase their teams’ effectiveness must
identify effective ways to gradually introduce DataOps techniques into data and analytics
solution delivery methodologies. For more information on DataOps, refer to
DataOps practices rely upon having connections among systems, people and things as a
means of sharing data for insights, innovation and competitive advantage. Enterprises
often meet this need by two approaches: connecting applications and data sources via
point-to-point interfaces (e.g., the Purdue Model) and centralizing as much data as
possible in a single system (e.g., a cloud database). Both approaches eventually become
costly, inflexible and run into scalability issues as data volume grows.
Integration of a data hub (see Figure 6) enables improved data sharing and integration via
more consistent, scalable and well-governed data flow. A data hub can deliver benefits,
including:
Data hubs from vendors such as HighByte, Cogent and Cybus provide connections to a
variety of OT systems and protocols (e.g., OPC/UA, Modbus), cloud services (e.g.,
Amazon Kinesis, Azure Event Hubs, Google Cloud Pub/Sub), pub/sub protocols (e.g.,
MQTT, Sparkplug B) and a growling list of connectors. The data hub connector is the
method by which organizations provide integration with their existing systems. For
additional information on data hubs, refer to Data and Analytics Essentials: Data Hubs
and Implementing the Technical Architecture for Master Data Management.
Over the past few years, CPS-PPs have emerged to help security leaders inventory their OT
assets. The reason CPS-PPs have risen to such prominence in the OT world is because
they lead with asset discovery. CPS-PPs may not find all OT assets, due to air-gapped
equipment and firewalls. However, it is not unusual for organizations to end up with an
inventory that shows they have up to 75% more assets than they knew about. Not only do
CPS-PPs provide a front end to the security process, they are also becoming a back end by
capturing telemetry, utilization and operational data. This information has value to the
engineering team.
CPS platform features initially focused on asset discovery, visibility and network topology.
However, as the products evolved, they became CPS protection platforms. The platforms
increasingly interoperate with other security tools, such as SIEM, or security orchestration,
automation and response (SOAR) solutions.
Organizations deploy the CPS platform in the cloud (as in Figure 7) or as an on-premises
appliance. The CPS data collectors reside at the industrial edge, often behind the OT
firewall. The collectors perform passive and active monitoring of OT assets. The
collectors forward asset information to the CPS-PP, as illustrated by the orange arrow in
Figure 7. There are many different types of data collectors. They include passive
discovery methods (e.g., networking monitoring using Switched Port Analyzer ports),
operating system clients (e.g., Microsoft Defender for Endpoint), network infrastructure
polling (e.g., DHCP, DNS) and OT protocol scanning (e.g., Modbus).
Sample vendors include Armis, Claroty, Dragos, Forescout, Fortinet, Microsoft, Nozomi
Networks, Ordr, SCADAfence and Xage Security. For more information on CPS-PPs, see:
■ Isolation of the OT network from the IT network via an OT firewall. By default, port
8883 is open for event messages (e.g., MQTT).
■ Assurance that edge appliances (e.g., edge firewall, IIoT gateway, CPS data
collectors) are trusted devices via an X.509 certificate.
■ Encryption of MQTT messages via use of TLS. By default, port 8883 is open for
event messages. Refer to Section 5 of the MQTT specification for additional
information.
The IIoT platform communicates with the IIoT gateway to access historically siloed data
sources and improve insights across heterogeneous asset groups. The IIoT gateway
provides event processing, workload orchestration and OT system integration at the
edge. The IIoT platform deploys workloads (e.g., ML inference, stream analytics) onto the
IIoT gateway. The gateway runs these workloads and forwards data to the IIoT platform.
The orange arrow in Figure 8 illustrates data being forwarded from the gateway to the
IIoT platform. IIoT vendors usually provide the gateway software as a GitHub open-
source project (e.g., Azure IoT Edge, AWS IoT Greengrass, thin-edge.io).
Sample vendors include ABB, AWS, Envision Digital, Hitachi, Microsoft, Siemens and
Software AG. For additional information, refer to Architect IoT Using the Gartner
Reference Model, and Emerging Technologies: AI-Enabled IoT.
Sparkplug B
The MQTT standard does not define a topic structure, nor does it define a payload
encoding scheme. Sparkplug B is an open-source specification, managed by the Eclipse
Foundation, that addresses both needs to improve interoperability. In addition, the
specification makes use of MQTT’s native continuous sessions awareness capability. The
Eclipse Foundation released Sparkplug B version 3 in November 2022.
■ Report by Exception: Edge nodes only publish data when values change in the
devices managed by the edge node. This is much more efficient than having an
application poll the edge node to discover data value changes.
■ Birth and Death Messages: The NBIRTH message is the first message an edge node
publishes. It includes the definition and current value for every metric managed by
the edge node. This message enables applications to dynamically and efficiently
discover data. This is powerful because it eliminates the need for static device and
tag configuration. This is analogous to how a router dynamically discovers
networks, instead of requiring static network configuration. The NDEATH message is
the final message an edge node will publish prior to intentionally disconnecting from
the broker. The NDEATH message notifies applications that the edge node is offline,
and all metrics managed by the edge node are stale.
■ Google Protocol Buffer Encoding: Sparkplug B uses Google protocol buffers for data
encoding. Tests have shown that Google protocol buffer encoding can reduce
network bandwidth consumption by more than 75%, compared to Modbus. 5
Sparkplug B defines a topic structure for MQTT messages (see Figure 10) that includes
the following components:
■ Message_type: Indicates how to process the MQTT payload (e.g., NBIRTH, NDEATH,
DDATA, DCMD).
■ Device_id: Identifies the device connected (logically or physically) to the edge node.
This is an optional element.
Vendor support for Sparkplug B is still emerging. Some brokers are Sparkplug-aware,
whereas some are only Sparkplug-compliant. Organizations must deploy a Sparkplug-
aware broker to realize the benefits of continuous-state awareness and RBE. Sample
MQTT brokers with Sparkplug B support include AWS IoT Core, Azure IoT Hub, Cirrus
Chariot, EMQX, HiveMQ, Rabbit MQ and Solace Event Broker.
For example, consider the fictitious Acme Corporation. Acme is a multinational company
with three divisions: Acme Oil, Acme Gas, Acme Solar. In this example, we focus on the
Acme Oil division where they have three refineries: R1 (in Ontario), R2 (Tijuana), R3
(Louisiana). The organization defined the identifiers listed below for their company codes,
country codes, site codes and so on. (Bolded terms are those appearing in Figure 12.)
■ Company (AO, AG, AS): Acme Oil, Acme Gas, Acme Solar
■ Area (R1, R2, R3, LNG1/2, SA1/2): Refineries 1, 2 and 3, Liquid Natural Gas Plants 1
and 2, Solar Arrays 1 and 2
■ System (VA1, VA2, TA1, TA2, PA1, PA2): Valves 1 and 2, Tanks 1 and 2, Panels 1 and
2
Figure 12 illustrates their topic structure for the bold codes to identify a specific tag
(VA1013C) on a valve (V1) attached to a device (G2).
Figure 13 illustrates the publish and subscribe topics with the reference architecture. The
industrial application subscribes to end-node G2 messages with wildcards + and #. When
edge node G2 sends an NBIRTH message, the application automatically discovers all the
metrics, and their current states from G2. The application receives all three of the
messages published by G2. If the organization adds new sensors and tags to G2, then G2
will publish a new NBIRTH message. The industrial application receives this NBIRTH
message, thus enabling it to dynamically discover the new sensor/tag metrics, as well as
updated values for the previously reported sensors/tags.
■ Edge MQTT Broker: The edge Sparkplug-aware MQTT broker can process Sparkplug
and non-Sparkplug messages. Organizations should install a broker that is
compliant with MQTT 5.0 and Sparkplug 3.0.
■ Cloud MQTT Broker: The cloud Sparkplug-compliant MQTT broker cannot participate
in the Sparkplug protocol (e.g., it does not have the ability to store NBIRTH and
DBIRTH messages). But it can forward Sparkplug messages from publisher to
subscriber.
■ Edge Node: OT equipment that does not support MQTT will use their OT protocols to
communicate with an edge node. The edge node translates the OT protocol to
MQTT.
■ Data Hub: The hub consumes data from the MQTT broker as an MQTT subscriber,
and from OT systems using OT protocols. It transfers that data to the cloud using
various methods (Kafka streams, file transfer, database synchronization, etc.).
■ Industrial IoT: The IIoT gateway consumes messages from OT equipment using OT
protocols. It may also perform edge processing (e.g., stream analytics). The gateway
communicates with the IIoT platform by sending event messages to the cloud MQTT
broker. Most IIoT platforms support the MQTT protocol.
■ CPS Protection Platform: The CPS data collectors interoperate with the CPS
protection platform to discover and protect cyber-physical assets. Organizations
may deploy the CPS protection platform in the cloud or on-premises.
■ Edge and Cloud MQTT Broker: The edge and cloud Sparkplug-aware MQTT brokers
can process Sparkplug and non-Sparkplug messages. Organizations should install a
broker that is compliant with MQTT 5.0 and Sparkplug 3.0.
■ Data Hub: The hub consumes data from the MQTT broker using MQTT/Sparkplug,
and from OT equipment using OT protocols. It transfers that data to the cloud using
various methods (Kafka streams, file transfer, database synchronization, etc.).
■ Industrial IoT: The IIoT gateway consumes messages from OT equipment using OT
protocols. It may also perform edge processing (e.g., stream analytics). The gateway
communicates with the IIoT platform by sending event messages to the cloud MQTT
broker.
■ CPS Protection Platform: The CPS data collectors interoperate with the CPS
protection platform to discover and protect assets. Organizations may deploy the
CPS protection platform in the cloud or on-premises.
Strengths
This reference architecture has the following strengths:
■ Makes use of open-source standards MQTT and Sparkplug B, which define message
formats and publisher/subscriber behavior. Uses a widely supported data
serialization format (Google protocol buffers). These features improve
interoperability and vendor independence.
■ Provides continuous-state awareness and RBE, thus eliminating the need for
applications to poll devices to determine session state and current tag values.
■ Integrates CPS protection platforms that provide asset discovery, visibility, protection
and network topology. The platforms integrate with SIEM and SOAR solutions to
provide unified IT-OT security.
■ Integrates an IIoT platform that collects high-volume time series and/or low-latency
complex machine data from networked IoT endpoints. The IIoT platform also
orchestrates historically siloed data sources to enable better accessibility, and
improve insights and actions across a heterogeneous asset group.
Weaknesses
This architecture has the following weaknesses:
■ The reference architecture has not been widely adopted. There may be unforeseen
issues as industrial organizations perform large-scale deployments.
■ UNS requires organizations to define a unified naming hierarchy that the entire
organization uses to define, access and manage their industrial business data. For
many organizations, this may be a long and possibly contentious process, as it
requires collaboration across many different roles and teams.
■ MQTT and Sparkplug B are optimized for telemetry data publishing (one publisher to
many subscribers). However, they do not support transaction exchanges (e.g.,
command/response exchange to load a new process manufacturing recipe into a
machine). MQTT and Sparkplug B do not indicate that a subscriber received the
message or that a subscriber acted upon the message. (Setting QoS=2 only
indicates that the broker received the message from the publisher.) Hence,
organizations must use a mechanism other than MQTT and Sparkplug B for
command/response exchanges.
■ The [group ID + edge node ID] couplet must be unique within the global namespace.
A message containing a [group ID + edge node ID] must only identify a single broker
destination. This may become a problem as different parts of the organization
independently define their group and edge node IDs, and inadvertently use duplicate
IDs. One solution is to define a pool of globally unique group IDs that are assigned to
various teams and independently managed by those teams. Each team can then
independently define their edge node IDs. This is like creating a pool of
organizational unique identifiers for the first three bytes of a MAC address.
■ Some people may perceive this reference architecture as too technical and lacking
alignment with business processes. Thus, technical professionals may not be able
to deploy the architecture.
Guidance
I&O technical professionals should take the following actions:
Evidence
This research used the anonymized information drawn from Gartner client inquiry and
from IT-OT technical documents shared by Gartner clients.
1
The Industrial Internet Reference Architecture, Industry IoT Consortium.
2
Reference Architectural Model Industrie 4.0, VDI/VDE Society Measurement and
Automatic Control (GMA).
3
OASIS Message Queueing Telemetry Transport (MQTT) TC, OASIS Open.
4
The NIS2 Directive: A High Common Level of Cybersecurity in the EU, European
Parliament.
5
Efficient IIoT Communications: A Comparison of MQTT, OPC-UA, HTTP, and Modbus,
Cirrus Link.
6
Sparkplug Version 3.0, Sparkplug.
© 2023 Gartner, Inc. and/or its affiliates. All rights reserved. Gartner is a registered trademark of
Gartner, Inc. and its affiliates. This publication may not be reproduced or distributed in any form
without Gartner's prior written permission. It consists of the opinions of Gartner's research
organization, which should not be construed as statements of fact. While the information contained in
this publication has been obtained from sources believed to be reliable, Gartner disclaims all warranties
as to the accuracy, completeness or adequacy of such information. Although Gartner research may
address legal and financial issues, Gartner does not provide legal or investment advice and its research
should not be construed or used as such. Your access and use of this publication are governed by
Gartner's Usage Policy. Gartner prides itself on its reputation for independence and objectivity. Its
research is produced independently by its research organization without input or influence from any
third party. For further information, see "Guiding Principles on Independence and Objectivity." Gartner
research may not be used as input into or for the training or development of generative artificial
intelligence, machine learning, algorithms, software, or related technologies.
Roadmap Webinar
Craft a Cloud Strategy to Develop the Infrastructure
Optimize Value Workforce of the Future
Maximize the benefits of the cloud with Overcome skills shortages through a
a strategy that clarifies the cloud’s role culture of development in infrastructure
in delivering IT-driven business value. organizations.
Already a client?
Get access to even more resources in your client portal. Log In
Connect With Us
Get actionable, objective insight that drives smarter decisions and
stronger performance on your mission-critical priorities. Contact us
to become a client:
Become a Client
© 2023 Gartner, Inc. and/or its affiliates. All rights reserved. CM_GTS_2656055