Internet of Things 23 (2023) 100854

Contents lists available at ScienceDirect

Internet of Things
journal homepage: www.sciencedirect.com/journal/internet-of-things

Microservices based architectures for IoT systems -

State-of-the-art review
Hassaan Siddiqui a, b, Ferhat Khendek a, *, Maria Toeroe b
a
Electrical and Computer Engineering, Concordia University, Montreal, Canada
b
Ericsson Inc., Montreal, Canada

A R T I C L E I N F O A B S T R A C T

Keywords: The Internet of Things (IoT) enables connectivity among physical world objects to empower
IoT multitudes of applications in different domains. This type of large-scale connectivity and distri­
Microservices bution comes with several functional and non-functional challenges. These challenges may be
Reliability
addressed by software architectures and techniques. Microservices is one of the latest software
Availability
Taxonomy
architecture paradigms that aims at addressing many of the challenges. In an effort to explore
Survey microservice architecture’s capability to address IoT challenges, this paper presents a review of
the current state-of-the-art on the microservices based architecture for IoT systems. In this paper,
we analyze how the microservices based architecture is currently used to improve non-functional
characteristics of IoT systems, specifically reliability and availability. This is the first state-of-the-
art review on microservices based IoT systems, it highlights strengths, weaknesses and oppor­
tunities for the microservices based architecture for IoT systems. To conduct this study a set of
research questions and a taxonomy for microservices based IoT systems have been defined. The
answers to the research questions provide an overall picture of the state of the art while the
mapping of the surveyed papers to the defined taxonomy provides a more detailed picture about
what has been done so far. The analysis shows that microservices based architecture has strong
potential for IoT systems, specifically for improving reliability and availability. However, several
weaknesses in the current solutions call for more research for handling properly reliability and
availability of IoT systems.

Introduction

The Internet of Things (IoT) paradigm aims at a global infrastructure of connected physical world objects for several application
domains such as, connected vehicles, smart city, and smart manufacturing. IoT systems consist of heterogenous devices, such as
sensors, software applications, as well as communication networks for connecting them. The IoT reference model recommended by
ITU-T, shown in Fig 1, includes four layers with management and security capabilities associated with each layer [1]. In this reference
model, the application layer consists of the IoT software applications, the service support and application support layer contains
generic and specific support capabilities for IoT applications such as data processing and storage. The network layer consists of
networking and transport capabilities, while the device layer contains IoT devices and gateways for interactions through the
communication networks. The most common characteristics associated with IoT systems are heterogeneity, large-scale connectivity,

* Corresponding author.
E-mail address: [email protected] (F. Khendek).

https://doi.org/10.1016/j.iot.2023.100854

Available online 27 June 2023

2542-6605/© 2023 Elsevier B.V. All rights reserved.
H. Siddiqui et al. Internet of Things 23 (2023) 100854

complexity, dynamic changes, massive data and intelligence [1,2]. These types of large-scale, complex and dynamic systems come with
various challenges. Some of these challenges are related to the systems’ features and functionalities, we refer to them as functional
challenges. Other challenges are related to the performance (throughput or latency for instance), reliability, availability, security,
scalability, portability, etc., we refer to them as non-functional challenges. Researchers have been investigating several software ar­
chitectures and techniques to tackle these challenges.
The traditional monolithic software architecture presents several weaknesses with respect to scalability, deployability, adapt­
ability, portability and availability. For instance, one cannot deploy or scale up/down only a portion of a monolithic application, the
whole application must be deployed or scaled up/down. Moreover, adapting new technologies, frameworks or languages requires full
application recompilation and redeployment. From the availability perspective, service recovery with restart requires full and heavy
application restart. These limitations are not acceptable for the dynamic and the distributed nature of IoT systems.
The Microservices Architecture (MA), on the other hand, is a recent software architecture paradigm that aims at solving these
limitations of traditional monolithic software by decomposing the whole software into independently deployable and scalable smaller
pieces called services or microservices [3]. The idea behind this decomposition is to decouple the different functions of the system.
Each microservice runs some independent business logic. Microservices communicate through lightweight mechanisms and are
independently deployable; thus, enable scalability. Some of the characteristics of the microservices architecture include componen­
tization, smart communication endpoints, decentralization, and infrastructure automation.
Non-functional Requirements (NFRs) and Quality of Services (QoS) parameters such as availability, reliability, interoperability,
latency, delay, are important characteristics of any distributed system. Satisfying NFRs and ensuring QoS for large-scale, complex, and
distributed systems, like IoT systems, is not a trivial task. The characteristics of MA make it suitable for IoT systems and for satisfying
NFRs and QoS at each layer of the IoT architecture [4,5].
The state-of-the-art study reported in this paper is a review conducted to analyze and evaluate how MA is used to address non-
functional requirements of IoT systems. To conduct this study, we set two goals. The first one is to have an overall picture of the
usage of MA for IoT systems, the challenges and techniques used, using two research questions. The second goal is to understand and
evaluate in more detail each research paper by mapping it to a taxonomy for MA based IoT systems we have devised. The taxonomy
and the mapping help in characterizing the variety of solutions, their strengths and weaknesses, and therefore derive conclusions about
the current state-of-the-art for the usage of MA for IoT systems.
The rest of the paper is organized as follows. Section 2 presents the research protocol used to conduct this study of the state-of-the-
art. In Section 3, a taxonomy for MA based IoT systems is presented. Research findings, including answers to research questions, and
classification of papers as per the proposed taxonomy, are presented in Section 4. Key observations, open challenges and future di­
rections are discussed in Section 5. Finally, Section 6 concludes this paper.

Research protocol

The aim of this study of the state-of-the-art is to analyze the use of MA for IoT systems. The reviewed research papers were selected
through a four-step process, as shown in Fig 2. In the first step, four databases, i.e. IEEE, ACM, Springer, and Elsevier Science Direct,
were searched using a search string that included keywords “IoT” and “microservices”. This search resulted in a total of 115 research
papers. In the second step, duplicate papers were removed. In the third step, exclusion criteria, see Table 1, were applied to abstracts
and conclusions, and research papers that met these criteria were filtered out. In the fourth step, the same exclusion criteria were
applied to the full texts and research papers that met the criteria were filtered out. At the end of this process, 29 research papers were
selected for thorough review. During the review, seven more research papers were added through snowballing. Final results are shown
in Table 2. In total, 36 papers [6–41] were evaluated in detail in this study.

Fig 1. IoT Reference Model by ITU-T [1].

2
H. Siddiqui et al. Internet of Things 23 (2023) 100854

Fig 2. Paper Filtering and Evaluation process.

Table 1
Exclusion criteria for paper selection.
Exclusion Criteria

1 Studies on microservices for IoT with insufficient information (light and vision papers).
2 Studies on microservices for IoT with focus on Networking or Network architecture.
3 Studies about IoT that do not really focus on microservices.

Taxonomy

To understand MA based IoT systems we propose a taxonomy that defines the domain, the sub-domains, the features, the concepts
and their relationships. This classification will be used later to understand and assess the surveyed research papers, their focus and
proposed solutions. The proposed taxonomy is shown in Fig 3, hereafter we discuss the different aspects.

IoT application domain

IoT provides an end-to-end (E2E) connectivity infrastructure which benefits all industrial domains that rely on sensors data. These
industrial domains include, but are not limited to, Smart City, Smart Grid, Industrial Internet of Things (IIoT), Internet of Vehicles
(IoV), and Healthcare. While designing MA based system for IoT, it is important to consider and understand the IoT application
domain, since there are domain specific challenges, NFRs and QoS requirements.

Deployment

A microservices based system can be deployed on either a cloud platform, an on-premise server, or an edge/fog device, where an
individual microservice instance could be hosted on either a Virtual Machine (VM) or a container. Since IoT systems are distributed by
nature, it is also possible that some microservice instances of the same application are deployed on a cloud platform and some
microservice instances are deployed on an edge/fog device, for performance reasons for instance.

Architecture & communication

The application layer of an IoT platform, as shown in Fig 1, runs software applications to serve the business purpose, for example,
an enterprise application for manufacturing industry or a data analytics application to analyze data from IoT sensors. In most cases, the

Table 2
Paper Filtering throughout the selection process.
Source Number of papers

Step 1 Step 2 Step 3 Step 4 Snowballing

IEEE 24 24 18 13 +7
ACM 6 6 6 3
Springer 58 57 7 6
Science Direct 27 26 15 7
Total 115 113 46 29 36

3
H. Siddiqui et al. Internet of Things 23 (2023) 100854

Fig 3. Taxonomy for microservices based IoT systems.

data from IoT sensors are routed to the IoT application through a middleware.
The IoT middleware provides the necessary abstraction as there are many heterogeneous devices and applications involved in an
IoT deployment. There are various types of middleware used for IoT deployments including Publish/Subscribe based, web services
based, and event streaming based middleware [42]. The Publish/Subscribe and event streaming based middleware follow the
Peer-to-Peer (P2P) communication model, whereas the web services based middleware follow the client-service communication
model.
Moreover, selecting an appropriate messaging technique is an essential part of MA based applications, as different instances of
microservices within an application communicate through API calls instead of function calls. Furthermore, the distinction between
stateful and stateless applications is also important. A stateful application keeps track of its execution state, whereas a stateless
application does not keep track of its execution state.

Non-Functional requirements

For heterogenous and distributed IoT systems, ensuring that the NFRs are met is a challenging task. IoT systems rely on real-time
data from sensors, and the complex and heterogenous nature of IoT systems makes it very challenging to ensure E2E availability,

4
H. Siddiqui et al. Internet of Things 23 (2023) 100854

reliability, and interoperability.

Reliability is one of the most important NFRs for IoT systems [5,30]. ISO [43] defines Reliability as “the degree to which a system,
product or component performs specified functions under specified conditions for a specified period of time”. Reliability is composed
of the following sub-characteristics; (1) Maturity, (2) Availability, (3) Fault-tolerance, and (4) Recoverability. Where, Availability is
defined as “the degree to which a system, product or component is operational and accessible when required for use”. Therefore, for
reliable IoT systems, (high) availability and fault-tolerance are a must.
Furthermore, maintaining and upgrading MA based applications is very different from performing these activities on monolithic
applications. Physical distribution of IoT systems requires that the maintenance and upgrade activities are performed remotely, which
presents numerous challenges [14]. Therefore, the proposed taxonomy takes into account six NFRs which are important for MA based
IoT systems: 1) Availability; 2) Reliability; 3) Scalability; 4) Interoperability; 5) Upgradability; and 6) Maintainability.

Availability and reliability for MA based IoT systems

The sub-taxonomy shown in Fig 4 presents some mechanisms that can be used to achieve high availability and reliability for MA
based applications, including redundancy, load balancing, service recovery, failure detection and failure prevention. It also covers the
metrics that are used to evaluate availability and reliability of a system. These metrics are grouped into two categories: failure metrics
and detection metrics.
Failure metrics: include Mean Time Between Failure (MTBF), Mean Time to Repair (MTTR), and recovery time. Recovery time is
different from MTTR since recovery of a microservice can be done using a redundant microservice which will not require repairing a
failed microservice [44].
Detection metric: includes failure detection time which refers to the time required to detect a failure since the time it happened.

Research findings

To evaluate the state of the art, we first map the research papers into the proposed taxonomy and evaluate each paper as per
proposed taxonomy to understand the approaches proposed, the domains, the tackled issues, the concepts adopted and used. We then
provide an overall picture of the domain by answering two high level questions, before summarizing some most prominent solutions.

Classification of state-of-the-art research

Each paper is mapped into the proposed taxonomy. The following sub-sections present the taxonomy mapping in detail.

IoT application domain

IoT application domains that are discussed in the literature are shown in Fig 5. It can be observed that 47% of papers do not belong

Fig 4. Sub-Taxonomy for Availability & Reliability.

5
H. Siddiqui et al. Internet of Things 23 (2023) 100854

to any specific IoT application domain, instead they discuss IoT in general. IIoT, which includes Industry 4.0, is the most discussed IoT
application domain whereas Smart City is the second IoT application domain discussed in the context of MA. Also, it can be noted that
MA is not yet explored for connected vehicles and IoV domains.

Non-Functional requirements
Our analysis shows that out of 36 papers, 21 papers discuss NFRs. Classification of papers according to the handled NFRs is pre­
sented in Table 3. It can be observed from Table 3 that the most discussed NFR in the literature is reliability, whereas maintainability
and upgradability are the least discussed.
Table 4 presents a classification of the papers according to the evaluation metrics used for NFR. It can be observed that for
evaluating availability & reliability, only 1 paper used MTBF and MTTR, and one paper used recovery time, whereas none of the papers
used failure detection time. Also, most of the metrics used to evaluate availability or reliability are indirect, for example, throughput,
latency. Furthermore, Aquino et al. [6] came up with a new metric called Availability to collaborate which they define as “the fraction of
time the system is active responding to requests from its peers”. This definition is very generic, and no further details were provided to
show how this metric is measured. This metric looks similar to Service availability as defined in [45] as "level of a provided service in
terms of the proportion of time it is operational and ready to deliver".
For scalability, the most used metrics are Load versus Response time and CPU versus Processing time. It can also be observed that
maintainability was not evaluated in any of the papers.

Research questions

Two research questions have been formulated to provide an overall picture of the state of the art at a higher level of abstraction.
These research questions will also help understand the challenges addressed by MA, as IoT systems have their own challenges including
NFRs.

• RQ1: What are the challenges addressed by MA for IoT systems?

• RQ2: What are the techniques proposed for handling NFRs using MA for IoT systems?

RQ1: what are the challenges addressed by MA for IoT systems?

Our analysis shows that the main challenges addressed by MA for IoT systems are: IoT device integration, heterogeneity and
interoperability, fault-tolerance and reliability, scalability, and system deployment and configuration. Section 4.3 presents brief
overviews of papers that use MA to handle these challenges of IoT systems.

RQ2: what are the techniques proposed for handling NFRs using MA for IoT systems?
Our analysis shows that for improving availability, the prominent techniques are reactive architecture, circuit braker pattern,
orchestration, and machine learning. Whereas for interoperability, messaging techniques and protocols have been used. Furthermore,
for scalability, orchestration and load balancing techniques have been proposed. Table 5 summarizes our findings about the techniques
proposed for NFRs using MA for IoT systems.

Overview of prominent solutions for NFR handling

In this section, a brief overview of some prominent papers is presented. These papers use different NFR handling techniques with

Fig 5. IoT Application Domains discussed in Literature.

6
H. Siddiqui et al. Internet of Things 23 (2023) 100854

Table 3
Classification of papers according to handled Non-Functional Requirements.
NFR Number of Papers Papers’ Reference

Availability 6 [8,12,15,22,29,30]
Reliability 14 [6,9,12-15,22-24,26-30]
Scalability 8 [8,11,14,21-24,38]
Interoperability 4 [7,15,19,22]
Upgradability 2 [14,35]
Maintainability 2 [14,22]

Table 4
Classification of papers according to metrics for NFR.
NFRs Metrics Number of Papers Papers’ Reference

Availability & Reliability MTBF 1 [29]

MTTR 1 [29]
Recovery time 1 [9]
Failure detection time 0 –
Server Response time 1 [28]
Service start time 1 [27]
Throughput 2 [12,28]
Latency 2 [12,27]
Task execution time 1 [9]
Availability to collaborate 1 [6]
Scalability Load versus Service instances 1 [11]
Load versus Response time 2 [11,21]
Load versus Throughput 1 [21]
CPU versus Processing time 2 [23,24]
Service deployment time 1 [8]
Service destruction time 1 [8]
Throughput 1 [38]
Interoperability Latency per protocol 1 [7]
Service registry time 1 [7]
Service discovery time 1 [7]
Upgradability Compilation time 1 [35]
Startup time 1 [35]
Maintainability – – –

MA, such as fault-tolerance, load balancing, orchestration, machine learning, etc. These techniques when used with MA, have a
positive impact on NFRs including availability, interoperability and scalability.

Availability & reliability

Power and Kotonya, 2018 [26] proposed a microservices based framework that provide fault-tolerance support to IoT systems.
Their proposed framework consists of four microservices, out of which two microservices are used for reactive and proactive
fault-tolerance support. One microservice provides reactive fault-tolerance through complex event processing, whereas another
microservice provides proactive fault-tolerance through Machine Learning. For error recovery, they propose blocking erroneous IoT
devices at the entry point microservice so that erroneous data do not propagate through the system, while the system can employ
backward or forward error recovery mechanisms. They used RESTful architecture for communication between microservices and
presented REST interfaces for data transmission between microservices using JSON format. They also provided an early evaluation of

Table 5
Techniques proposed for NFRs.
NFR Proposed techniques

Availability & Reliability • Watchdog microservice [9].

• Circuit braker microservice [6,12,29,30].
• Reactive microservices [12,29,30].
• Orchestration for microservice recovery [27].
• Complex event processing [26].
• Machine learning for proactive fault-tolerance [26].
• Distributed message brokers [14,15].
Interoperability • Multiple messaging protocols (RESTful & Pub/Sub) [15,7,22].
• Artificial Neural Network (ANN) for protocol selection [7].
Scalability • Orchestration with Blue/Green deployment strategy [8].
• Load balancing [21,11,38].

7
H. Siddiqui et al. Internet of Things 23 (2023) 100854

their proposed framework however the experiment is of very basic nature and does not validate the proposed framework.
Falas et al. [14] identified several key challenges for development of IoT service based systems, which are: (1) Adaptability,
flexibility and scalability; (2) Maintainability and updates; (3) Network connectivity; (4) Dynamic data stream processing services; (5)
Security; and (6) Power efficiency. They also presented implementation of an IoT edge device based on IoT-A model [46]. This
implementation uses microservices architecture integrated with an altered version of pub/sub message broker, addressing the iden­
tified challenges, in which each service handles a separate use case as a workflow. In their proposed approach, a distributed version of
message broker is used where interconnectivity between each of the services is conducted via a dedicated message broker instead of
one central message broker. They suggested that this approach increases the reliability of the IoT system as the failure of a single
service only affects a single use case/workflow; and in case of a service failure, all undelivered messages will remain in the message
queue and can be consumed later by a new instance of the service. They conducted an experiment with four communication scenarios,
which differed in the location of the message broker (RabbitMQ) and the data source, in which data transfer efficiency and average
message rate were measured. They concluded that the proposed architecture provides solutions to the key IoT challenges. However,
their experiment does not validate that the proposed framework increases reliability of the IoT system.
Celesti et al. [9] proposed a watchdog service for monitoring the status of containerized microservices, to improve the reliability of
microservices in IoT devices. They proposed that each IoT device should have two additional containers running: (1) reliability service;
and (2) local database, to track status of deployed microservices. Each IoT device can have either active microservices or passive
microservices running in containers. A passive microservice is for replication purposes which is used in case of a microservice failure in
a nearby IoT device. When a microservice fails in a device, it is detected by other neighbor devices and then the watchdog service can
either repair the broken microservice or replace it with a passive (replica) microservice. Through the experiments, they concluded that
a failed microservice is replaced with a replica in an acceptable timeframe, and that Docker container engine introduces an acceptable
overhead during the microservice replacement process.
Rufino et al. [27] proposed an orchestration solution for IIoT applications based on Docker and microservices. The proposed ar­
chitecture is composed of three layers, sensing layer, mediation layer, and enterprise layer. Each layer has containerized microservices
running and each service depicts a different use case. They explained that resilience and fault-tolerance can be achieved through
independent services where a single microservice failure can have little impact on system performance, and orchestration rules that
ensure service recovery in case of failure, can improve reliability. They performed an experiment which shows that a failed micro­
service is recovered in an acceptable timeframe.
Data Pool (Gateway) proposed by García-Ortiz et al. [15] also ensures fault-tolerance and high availability through the broker
clustering mechanism of the RabbitMQ broker to form a logical federated broker cluster on different physical devices.

Scalability
Most of the architectures presented in the literature that focus on scalability requirements only discuss the design and imple­
mentation of the scalable architecture, but do not validate the feasibility of their proposed approach [11,47,48]. Also, validating the
actual scalability of a complex platform, like smart city, is very challenging as it requires strong infrastructure and comprehensive
datasets [11].
Del Esposte et al. presented a microservices based smart city platform, InterSCity, with focus on scalability in [21] and evaluated
scalability of this platform in [11] using a smart city simulator InterSCSimulator [49]. The InterSCity platform ensures scalability
through a set of six microservices, with design principles including decentralized data management, i.e., each microservice has its own
database, synchronous and asynchronous communication methods, and stateless services. The scalability evaluation in [11] consid­
ered a smart parking scenario and concluded that the InterSCity platform is capable of handling varying workload by horizontal
scaling, and it can handle more than one million complex requests during approximately 3 h. Furthermore, the authors in [11]
proposed the use of Kubernetes and Docker to achieve scalability.
Tserpes [38] presented a microservices based solution for IoT stream processing data pipelines. This solution incorporates load
balancing within the application logic using microservices architecture to achieve scalability and proposes adding an infrastructure
aware stream microservice for the load balancer. When high load is identified, more stream microservices are spawned in the
workflow. Although the proposed approach offers scalability, the presented experiment does not validate the proposed approach.

Interoperability
Koundourakis & Petrakis [19] presented iXen, a microservices based IoT platform, with motive to provide better security and
interoperability. This platform uses several RESTful microservices communicating with each other over HTTP. They also performed
performance evaluation of the proposed architecture using a set of experiments where the system response time was evaluated against
varying number of requests and concluded that the system is able to perform as expected under heavy load. However, no validation
was presented that the proposed architecture provides interoperability.
García-Ortiz et al. [15] presented a gateway for IoT device integration. This gateway is based on MA and ensures several
non-functional requirements including interoperability, resilience, safety, security, and scalability. This gateway is capable of trans­
lating messages from one IoT communication protocol to another using RabbitMQ message broker, and uses Internet Protocol for
Smart Objects (IPSO) [50] for message payload to ensure interoperability. Furthermore, for fault-tolerance and high availability,
broker clustering mechanism of RabbitMQ broker is used to form a logical federated broker cluster on different physical devices.
Although the architecture presented by the authors offers interoperability and fault-tolerance, no case study has been presented to
validate the proposed approach.

8
H. Siddiqui et al. Internet of Things 23 (2023) 100854

Discussion

Through the evaluation of the selected papers and their taxonomy mapping, we observed that most of the papers discuss the
benefits of MA but not the challenges they present. MA comes with its own set of challenges that should be considered. There is a lack of
validation of the research using realistic settings and appropriate evaluation metrics. Also, MA is not yet explored in all domains. Some
domains, e.g., connected vehicles and IoV, may present several unique challenges where MA can be helpful, specially from the
availability and reliability perspective.
More important, our review and analysis show that end-to-end availability and reliability have not been addressed in the literature.
End-to-end availability differs from availability of one or some components of a system. Indeed, if even one component of the system is
error-prone, it can affect the availability of the whole system. For high-availability for instance, each component of the system must be
highly-available to achieve end-to-end high-availability for the system. Solutions for maintainability and upgradability are also
needed, since maintaining and upgrading a large-scale distributed system can be very challenging, especially when availability should
be taken into account. Usually, each component of a large-scale system is developed and maintained by a different team, and each team
may have different availability requirements, availability measurements and service level agreements (SLAs), which can affect the end-
to-end availability of the whole system. Based on our evaluation and analysis, one important research question can be formulated for
future investigations:

• How end-to-end availability and reliability can be achieved for IoT systems leveraging MA and possibly other techniques and
architectures such as, Edge/Fog nodes and cloud platforms?

Conclusions

In this paper, we reviewed 36 papers about microservices based IoT systems. We devised a taxonomy to gain a deeper under­
standing of microservices based IoT systems, and a sub-taxonomy for availability and reliability for microservices based IoT systems.
The review was primarily based on a mapping to the devised taxonomy and answers to two research questions. We have also presented
an overview of the prominent papers that use different NFR handling techniques with microservice architecture. Our analysis shows
that microservice architecture has strong potential for IoT systems specifically for improving NFRs. Furthermore, along with its
benefits, we also observed that microservice architecture also presents some challenges, such as design, operational complexities and
communication. However, most of the papers discuss only the benefits of microservices but not the challenges they present. We
observed that most of the solutions focus on reliability whereas maintainability and upgradability have the least presented solutions.
Finally, a comprehensive solution for end-to-end availability and reliability of IoT systems is lacking.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to
influence the work reported in this paper.

Data availability

No data was used for the research described in the article.

Acknowledgements

This work has been partially supported by the Natural Sciences and Engineering Research Council of Canada (NSERC) and Ericsson.

References

[1] Overview of the Internet of Things, Y.2060, International Telecommunication Union, Jun. 15, 2012. Accessed: May 11, 2021. [Online]Available, http://handle.
itu.int/11.1002/1000/11559.
[2] H.F. Atlam, R.J. Walters, G.B. Wills, Internet of things: state-of-the-art, challenges, applications, and open issues, Int. J. Intell. Comput. Res. 9 (3) (Sep. 2018)
928–938, https://doi.org/10.20533/ijicr.2042.4655.2018.0112.
[3] J. Lewis and M. Fowler, “Microservices,” Mar. 25, 2014. https://martinfowler.com/articles/microservices.html (accessed May 17, 2021).
[4] B.E. Khalyly, A. Belangour, M. Banane, A. Erraissi, A comparative study of microservices-based IoT platforms, Int. J. Adv. Comput. Sci. Appl. IJACSA 11 (8)
(2020), 8, https://doi.org/10.14569/IJACSA.2020.0110850. Art. no.
[5] G. White, V. Nallur, S. Clarke, Quality of service approaches in IoT: a systematic mapping, J. Syst. Softw. 132 (Oct. 2017) 186–203, https://doi.org/10.1016/j.
jss.2017.05.125.
[6] G. Aquino, R. Queiroz, G. Merrett, B. Al-Hashimi, The circuit breaker pattern targeted to future IoT applications, in: S. Yangui, I. Bouassida Rodriguez, K. Drira,
Z. Tari (Eds.), Service-Oriented Computing, Springer International Publishing, Cham, 2019, pp. 390–396, https://doi.org/10.1007/978-3-030-33702-5_30, in
Lecture Notes in Computer Science.
[7] A. Benayache, A. Bilami, S. Barkat, P. Lorenz, H. Taleb, MsM: a microservice middleware for smart WSN-based IoT application, J. Netw. Comput. Appl. 144 (Oct.
2019) 138–154, https://doi.org/10.1016/j.jnca.2019.06.015.
[8] A. Buzachis, A. Galletta, A. Celesti, L. Carnevale, M. Villari, Towards osmotic computing: a blue-green strategy for the fast re-deployment of microservices, in:
Proceedings of the 2019 IEEE Symposium on Computers and Communications (ISCC), Jun. 2019, pp. 1–6, https://doi.org/10.1109/ISCC47284.2019.8969621.

9
H. Siddiqui et al. Internet of Things 23 (2023) 100854

[9] A. Celesti, L. Carnevale, A. Galletta, M. Fazio, M. Villari, A watchdog service making container-based micro-services reliable in IoT clouds, in: Proceedings of the
2017 IEEE 5th International Conference on Future Internet of Things and Cloud (FiCloud), IEEE, Aug. 2017, pp. 372–378, https://doi.org/10.1109/
FiCloud.2017.57.
[10] M. Ciavotta, M. Alge, S. Menato, D. Rovere, P. Pedrazzoli, A Microservice-based Middleware for the Digital Factory, in: Proc. 27th Int. Conf. Flex. Autom. Intell.
Manuf. FAIM2017 27-30 June 2017 Modena Italy 11, Jan. 2017, pp. 931–938, https://doi.org/10.1016/j.promfg.2017.07.197.
[11] A. de M. Del Esposte, et al., Design and evaluation of a scalable smart city software platform with large-scale simulations, Future Gener. Comput. Syst. 93 (Apr.
2019) 427–441, https://doi.org/10.1016/j.future.2018.10.026.
[12] C.J.L. de Santana, B. de Mello Alencar, C.V.S. Prazeres, Reactive microservices for the internet of things: a case study in fog computing, in: Proceedings of the
34th ACM/SIGAPP Symposium on Applied Computing, New York, NY, USA, ACM, Apr. 2019, pp. 1243–1251, https://doi.org/10.1145/3297280.3297402, in
SAC ’19.
[13] J. Dobaj, J. Iber, M. Krisper, C. Kreiner, A microservice architecture for the industrial internet-of-things, in: Proceedings of the 23rd European Conference on
Pattern Languages of Programs, New York, NY, USA, Association for Computing Machinery, Jul. 2018, pp. 1–15, https://doi.org/10.1145/3282308.3282320, in
EuroPLoP ’18.
[14] Ł. Falas, P. Schauer, P. Świątek, R. Trzaska, Practical implementation of internet of things service systems architecture, in: Proceedings of the 2017 25th
International Conference on Systems Engineering (ICSEng), Aug. 2017, pp. 291–298, https://doi.org/10.1109/ICSEng.2017.57.
[15] J.C. García-Ortiz, D. Todolí-Ferrandis, J. Vera-Pérez, S. Santonja-Climent, V. Sempere-Payá, Design of a micro-service based data pool for device integration to
speed up digitalization, in: Proceedings of the 2019 27th Telecommunications Forum (℡FOR), IEEE, Nov. 2019, pp. 1–4, https://doi.org/10.1109/
℡FOR48224.2019.8971346.
[16] X. He, F. Deng, Research on architecture of internet of things platform based on service mesh, in: Proceedings of the 2020 12th International Conference on
Measuring Technology and Mechatronics Automation (ICMTMA), Feb. 2020, pp. 755–759, https://doi.org/10.1109/ICMTMA50254.2020.00164.
[17] J. Innerbichler, S. Gonul, V. Damjanovic-Behrendt, B. Mandler, F. Strohmeier, NIMBLE collaborative platform: microservice architectural approach to federated
IoT, in: Proceedings of the 2017 Global Internet of Things Summit (GIoTS), Jun. 2017, pp. 1–6, https://doi.org/10.1109/GIOTS.2017.8016216.
[18] K. Khanda, D. Salikhov, K. Gusmanov, M. Mazzara, N. Mavridis, Microservice-based IoT for smart buildings, in: Proceedings of the 2017 31st International
Conference on Advanced Information Networking and Applications Workshops (WAINA), Mar. 2017, pp. 302–308, https://doi.org/10.1109/WAINA.2017.77.
[19] X. Koundourakis, E.G.M. Petrakis, iXen: context-driven service oriented architecture for the internet of things in the cloud, in: Proceedings of the s11th
International Conference on Ambient Systems, Networks and Technologies (ANT) /The 3rd International Conference on Emerging Data and Industry 4.0
(EDI40) / Affiliated Workshops, Jan. 2020, pp. 145–152, https://doi.org/10.1016/j.procs.2020.03.019.
[20] P. Krivic, P. Skocir, M. Kusek, G. Jezic, Microservices as agents in IoT systems, in: G. Jezic, M. Kusek, Y.-H.J. Chen-Burger, R.J. Howlett, L.C. Jain (Eds.), Agent
and Multi-Agent Systems: Technology and Applications, Springer International Publishing, Cham, 2017, pp. 22–31, https://doi.org/10.1007/978-3-319-59394-
4_3, in Smart Innovation, Systems and Technologies.
[21] A.M. Del Esposte, F. Kon, F.M. Costa, N. Lago, InterSCity: a scalable microservice-based open source platform for smart cities, in: Proceedings of the 6th
International Conference on Smart Cities and Green ICT Systems, Setubal, PRT, SCITEPRESS - Science and Technology Publications, Lda, Apr. 2017, pp. 35–46,
https://doi.org/10.5220/0006306200350046. SMARTGREENS 2017.
[22] H.K. Ngankam, et al., An IoT architecture of microservices for ambient assisted living environments to promote aging in smart cities, in: J. Pagán, M. Mokhtari,
H. Aloulou, B. Abdulrazak, M.F. Cabrera (Eds.), How AI Impacts Urban Living and Public Health, Springer International Publishing, Cham, 2019, pp. 154–167,
https://doi.org/10.1007/978-3-030-32785-9_14, in Lecture Notes in Computer Science.
[23] K. Ogawa, K. Kanai, K. Nakamura, H. Kanemitsu, J. Katto, H. Nakazato, IoT device virtualization for efficient resource utilization in smart city IoT platforsm, in:
2019 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops), IEEE, Mar. 2019, pp. 419–422, https://
doi.org/10.1109/PERCOMW.2019.8730806.
[24] K. Ogawa, et al., Performance evaluations of IoT device virtualization for efficient resource utilization, in: Proceedings of the 2019 Global IoT Summit (GIoTS),
Jun. 2019, pp. 1–6, https://doi.org/10.1109/GIOTS.2019.8766410.
[25] E.G.M. Petrakis, S. Sotiriadis, T. Soultanopoulos, P.T. Renta, R. Buyya, N. Bessis, Internet of Things as a Service (iTaaS): challenges and solutions for
management of sensor data on the cloud and the fog, Internet Things 3–4 (Oct. 2018) 156–174, https://doi.org/10.1016/j.iot.2018.09.009.
[26] A. Power, G. Kotonya, A microservices architecture for reactive and proactive fault tolerance in IoT systems, in: Proceedings of the 2018 IEEE 19th International
Symposium on “A World of Wireless, Mobile and Multimedia Networks” (WoWMoM), Jun. 2018, pp. 588–599, https://doi.org/10.1109/
WoWMoM.2018.8449789.
[27] J. Rufino, M. Alam, J. Ferreira, A. Rehman, K.F. Tsang, Orchestration of containerized microservices for IIoT using Docker, in: Proceedings of the 2017 IEEE
International Conference on Industrial Technology (ICIT), Mar. 2017, pp. 1532–1536, https://doi.org/10.1109/ICIT.2017.7915594.
[28] D.D. Sánchez-Gallegos, et al., On the Continuous Processing of Health Data in Edge-Fog-Cloud Computing by Using Micro/Nanoservice Composition, IEEE
Access 8 (2020) 120255–120281, https://doi.org/10.1109/ACCESS.2020.3006037.
[29] C. Santana, L. Andrade, F.C. Delicato, C. Prazeres, Increasing the availability of IoT applications with reactive microservices, Serv. Oriented Comput. Appl. (Nov.
2020), https://doi.org/10.1007/s11761-020-00308-8.
[30] C. Santana, L. Andrade, B. Mello, E. Batista, J.V. Sampaio, C. Prazeres, A reliable architecture based on reactive microservices for IoT applications, in:
Proceedings of the 25th Brazillian Symposium on Multimedia and the Web - WebMedia ’19, Rio de Janeiro, Brazil, ACM Press, 2019, pp. 15–19, https://doi.org/
10.1145/3323503.3345027.
[31] P. Schauer, Ł. Falas, P. Świątek, IoT service based systems building methodology, in: Proceedings of the 2018 26th International Conference on Systems
Engineering (ICSEng), Dec. 2018, pp. 1–9, https://doi.org/10.1109/ICSENG.2018.8638213.
[32] G. Selvakumar, L.S. Jayashree, Agile supply chain management enabled by the internet of things and microservices, in: L.A. Kumar, L.S. Jayashree,
R. Manimegalai (Eds.), Proceedings of the International Conference on Artificial Intelligence, Smart Grid and Smart City Applications, Cham, Springer
International Publishing, 2020, pp. 449–456, https://doi.org/10.1007/978-3-030-24051-6_43.
[33] W. Sriborrirux, P. Laortum, Healthcare center IoT edge gateway based on containerized microservices, in: Proceedings of the 2020 4th International Conference
on Intelligent Systems, Metaheuristics & Swarm Intelligence, New York, NY, USA, Association for Computing Machinery, Mar. 2020, pp. 24–29, https://doi.org/
10.1145/3396474.3396495, in ISMSI ’20.
[34] P. Sridhara, N. Kamath, S. Srinivas, A microservices-based smart IoT gateway system, in: S.C. Satapathy, V. Bhateja, S. Das (Eds.), SmartIntelligent Computing
and Applications, Springer, Singapore, 2019, pp. 619–630, https://doi.org/10.1007/978-981-13-1927-3_65, in Smart Innovation, Systems and Technologies.
[35] C. Sun, J. Wang, J. Guo, Z. Wang, L. Duan, et al., A reconfigurable microservice-based migration technique for IoT systems, in: S. Yangui, A. Bouguettaya,
X. Xue, N. Faci, W. Gaaloul, Q. Yu, et al. (Eds.), Proceedings of the Service-Oriented Computing – ICSOC 2019 Workshops, Cham, Springer International
Publishing, 2020, pp. 142–155, https://doi.org/10.1007/978-3-030-45989-5_12, in Lecture Notes in Computer Science.
[36] K. Thramboulidis, D.C. Vachtsevanou, A. Solanos, Cyber-physical microservices: an IoT-based framework for manufacturing systems, in: Proceedings of the
2018 IEEE Industrial Cyber-Physical Systems (ICPS), May 2018, pp. 232–239, https://doi.org/10.1109/ICPHYS.2018.8387665.
[37] K. Thramboulidis, D.C. Vachtsevanou, I. Kontou, CPuS-IoT: a cyber-physical microservice and IoT-based framework for manufacturing assembly systems, Annu.
Rev. Control 47 (Jan. 2019) 237–248, https://doi.org/10.1016/j.arcontrol.2019.03.005.
[38] K. Tserpes, Stream-MSA: a microservices’ methodology for the creation of short, fast-paced, stream processing pipelines, ICT Express 5 (2) (Jun. 2019) 146–149,
https://doi.org/10.1016/j.icte.2019.04.001.
[39] F.J. Valente, J.P. Morijo, K.C.T. Vivaldini, L.C. Trevelin, Fog-based data fusion for heterogeneous iot sensor networks: a real implementation, in: Proceedings of
the 2019 15th International Conference on Network and Service Management (CNSM), IEEE, Oct. 2019, pp. 1–5, https://doi.org/10.23919/
CNSM46954.2019.9012744.
[40] Y. Wang, B. Cheng, M. Niu, An end-user microservice-based lightweight architecture for IoT, in: Proceedings of the 2018 14th International Computer
Engineering Conference (ICENCO), Dec. 2018, pp. 68–72, https://doi.org/10.1109/ICENCO.2018.8636150.

10
H. Siddiqui et al. Internet of Things 23 (2023) 100854

[41] H. Chegini, A. Mahanti, A Framework of Automation on context-aware Internet of Things (IoT) systems, in: Proceedings of the 12th IEEE/ACM International
Conference on Utility and Cloud Computing Companion, New York, NY, USA, Association for Computing Machinery, Dec. 2019, pp. 157–162, https://doi.org/
10.1145/3368235.3368848. UCC ’19 Companion.
[42] M. Díaz, C. Martín, B. Rubio, State-of-the-art, challenges, and open issues in the integration of Internet of things and cloud computing, J. Netw. Comput. Appl.
67 (May 2016) 99–117, https://doi.org/10.1016/j.jnca.2016.01.010.
[43] “Reliability, ” ISO 25010. https://iso25000.com/index.php/en/iso-25000-standards/iso-25010/62-reliability (accessed Aug. 22, 2021).
[44] M. Nabi, M. Toeroe, F. Khendek, Availability in the cloud: state of the art, J. Netw. Comput. Appl. 60 (Jan. 2016) 54–67, https://doi.org/10.1016/j.
jnca.2015.11.014.
[45] M. Toeroe, F. Tam, Service Availability: Principles and Practice, Wiley, 2012.
[46] M.B. Nec et al., “Internet of Things – Architecture IoT-A Deliverable D1.5 – Final architectural reference model for the IoT v3.0,” 2013.
[47] E.F.Z. Santana, A.P. Chaves, M.A. Gerosa, F. Kon, D.S. Milojicic, Software platforms for smart cities: concepts, requirements, challenges, and a unified reference
architecture, ACM Comput. Surv. 50 (6) (Nov. 2017), https://doi.org/10.1145/3124391, 78:1-78:37.
[48] L. Sanchez, et al., SmartSantander: the meeting point between future Internet research and experimentation and the smart cities, in: Proceedings of the 2011
Future Network Mobile Summit, Jun. 2011, pp. 1–8.
[49] E.F.Z. Santana, N. Lago, F. Kon, D.S. Milojicic, InterSCSimulator: large-scale traffic simulation in smart cities using erlang, in: G.P. Dimuro, L. Antunes (Eds.),
Multi-Agent Based Simulation XVIII, Springer International Publishing, Cham, 2018, pp. 211–227, https://doi.org/10.1007/978-3-319-91587-6_15, in Lecture
Notes in Computer Science.
[50] J. Jimenez, M. Koster, and H. Tschofenig, “IPSO Smart Objects,” in IoT Semantic Interoperability Workshop, Internet Architecture Board, 2016. [Online].
Available: https://www.iab.org/wp-content/IAB-uploads/2016/03/ipso-paper.pdf.

11