1082 IEEE INTERNET OF THINGS JOURNAL, VOL. 4, NO.

4, AUGUST 2017

Experimental Characterization of Mobile

IoT Application Latency
Carlos Pereira, António Pinto, Duarte Ferreira, and Ana Aguiar, Member, IEEE

Abstract—The Internet of Things (IoT) emerges as a myriad standards foster the emergence of Internet of Things (IoT)
of devices and services that interact to build complex dis- applications, by providing interoperability among devices
tributed applications. Interoperability and standardization are and services outside application silos. Currently, many of
imperative for the realization of this vision. Machine-to-machine
(M2M) communications standards can be the middleware that these applications exploit the massive use of smartphones
glues together the IoT. However, standards are highly complex as well as their connectivity and sensing capabilities, e.g.,
and require a large amount of interpretation, deployments are for monitoring health-related information or intelligent
currently scarce, and performance evaluations simplistic or spec- mobility [2]. In mobile M2M communications, smartphones
ulative. In this paper, we focus on the experimental evaluation of are also envisioned to play the important role of M2M
latency in IoT service composition with mobile gateways (GWs).
We measure latency between system components and quantify gateways (GWs), acting as proxy for nearby devices with
application protocol overheads to assess the capabilities and constrained resources and limited connectivity [1].
limitations of a standard M2M middleware. We designed and Recent forecast [3] estimates that 6.4 billion connected
implemented a mobile e-health use case on top of ETSI M2M things will be in use worldwide in 2016 reaching 20.8 billion
and openEHR standards. We ran a pilot remote monitoring ten by 2020, and IoT supported total services spending of $235
people for three weeks, collecting nearly 480 h of data. Our
results show that while the latency added by a broker lies around billion in 2016. Interoperability could guarantee that devices
25 ms, the cellular network often exceeds 1 s, becoming a problem can be integrated with infrastructures and services, and that
for interactive applications. Moreover, we observe that latencies services can be composed into complex applications in which
between a smartphone GW and cloud hosted services vary largely each stakeholder focuses on his specific know-how. Mobile
depending on the user mobility, and on the promotion delay of e-health is a perfect use case because available solutions tend
the used wireless network.
to be proprietary. This leads to closed and inefficient vertical
Index Terms—Cellular networks, e-health, Internet of Things silos that have difficulty in scaling and cause dispersion due
(IoT), machine-to-machine (M2M) communications, mobile to the impossibility to share resources [4], [5]. Interoperability
gateways (GWs), smartphones, system performance, wireless
networks. and standardization are key for general recognition and accep-
tance [6]–[8].
In this paper, as a storyboard, we envision that individu-
als are advised by a primary healthcare unit to have their
I. I NTRODUCTION lifestyle remotely monitored using a wearable system that
monitors their heart rate and mobility [9]. This storyboard
ACHINE-TO-MACHINE (M2M) communications
M refer to mechanisms, algorithms, and technologies that
enable networked devices, wired and/or wireless, and services
is implemented as an interoperable mobile e-health applica-
tion, and the application overview is presented in Fig. 1.
Users follow a daily routine: every morning, before leav-
to seamlessly exchange data and control information without
ing home for school or work, they put on a Bluetooth (BT)
explicit human intervention [1]. M2M communications’
wearable device and start an application on their smartphone.
The smartphone continuously uploads the sensor data using
Manuscript received October 28, 2016; revised February 17, 2017; accepted
March 23, 2017. Date of publication March 30, 2017; date of current version a cellular data plan to a service which extracts several heart
August 9, 2017. This work is a result of the project NanoSTIMA (NORTE- rate and mobility indicators and verifies whether daily and
01-0145-FEDER-000016). This work builds upon software and datasets that weekly physical recommendations are met. This service also
resulted from I-City for Future Health (NORTE-07-0124-FEDER-000068),
FP7 - Future Cities (FP7-REGPOT-2012-2013-1), and direct funding by PT formats data according to the semantics required for the cor-
Inovação - Altice Labs. Research was carried out in the scope of R&D rect interpretation by an open electronic health record (EHR)
Unit 50008 (UID/EEA/50008/2013), and financed by the applicable finan- interface service that runs in the healthcare unit. The pro-
cial framework: Norte Portugal Regional Operational Programme (NORTE
2020), through Portugal 2020 and the European Regional Development Fund, cessed data is sent to the openEHR interface service which
FCT/MEC through national funds and when applicable co-funded by FEDER is responsible for storing and making the data available to
PT2020 partnership agreement. (Corresponding author: Carlos Pereira.) care takers. Medical personnel can analyze the users’ lifestyle
C. Pereira, A. Pinto, and A. Aguiar are with the Faculty of Engineering,
Instituto de Telecomunicações, University of Porto, Porto 4200-465, Portugal information through their EHR. Communications between ser-
(e-mail: [email protected]; [email protected]; [email protected]). vices and devices use M2M middleware to implement a
D. Ferreira is with the Faculty of Medicine, Center for Research in Health publish-subscribe communication model via a message broker.
Technologies and Information Systems, University of Porto, Porto 4200-465,
Portugal (e-mail: [email protected]). Latency is a critical metric of service composition
Digital Object Identifier 10.1109/JIOT.2017.2689682 performance [10], and, thus, M2M middleware performance.
2327-4662 c 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
PEREIRA et al.: EXPERIMENTAL CHARACTERIZATION OF MOBILE IoT APPLICATION LATENCY 1083

capture the inherent problems of real-world deployments, like

the link quality or mobility. On the other hand, the devel-
opment of robust applications for real conditions is also a
cumbersome task [16].
We summarize our contributions as follows.
1) We detail the design and implementation of an IoT
e-health application (remote monitoring) as an IoT ser-
vice composition on top of standard technologies ETSI
M2M and openEHR.
2) We quantify application protocol overheads.
3) We measure latency between system components using
nearly 480 h of data collected from a three week mobile
pilot with ten users. We show that the latency between
a smartphone GW and the openEHR interface service
can exceed 1 s when using cellular networks, and we
Fig. 1. Overview of the mobile e-health application. Arrows represent user quantify the impact of the middleware. Among other
data flow.
findings, we show that there is a correlation between the
distance traveled by each user and the latency measured
between its GW and the broker.
Critical applications respond to event alarms and emer- 4) We find that the largest part of the E2E latency lies
gency situations, which depend on timely data transfer, and between the GW and the broker, even for different wire-
many other envisioned applications are of interactive nature. less networks. Its magnitude is greatly dependent of the
Interactivity might not only be user-related, where users need GW’s network interface state at the moment when the
to acknowledge or validate data, but also service/machine- application schedules transmissions due to the promo-
related in the form of alarm, prevention, or advisory systems tion delay. An idle state requires time for allocation of
related to physiological data in a remote actuation scenario. network resources to the interface.
Furthermore, future micro- or even nano-size wearable devices This paper is structured as follows. In Section II, we sur-
can bring additional use case scenarios (and, thus, applica- vey related work. We present background on ETSI M2M
tions) where subsecond time requirements might also need to and openEHR standards, system architecture, system setup
be guaranteed. and implementation in Section III. We discuss the chal-
As data usefulness decays after a deadline, M2M mid- lenges concerning latency measurements and how we address
dleware must guarantee not only interoperability but also them in Section IV. Section V presents the performance
predictable and measurable end-to-end (E2E) quality of ser- evaluation and results. We compile important observations
vice. However, this aspect has received little attention so far, for researchers in Section VI, and we conclude this paper
likely due to lack of available implementations of such plat- in Section VII.
forms and applications on top of them. We aim to understand
if standardized M2M communications are able to provide
time-requirement guarantees in a context of an IoT service II. R ELATED W ORK
composition using mobile networks. Services are a main building block of IoT. Recently, sev-
In this paper, we report the design and implementation of a eral IoT services appeared using M2M as a key enabler
mobile e-health application on a standard M2M middleware, to connect devices. Perhaps the most known examples for
and experimentally evaluate its latency performance for service IoT/M2M are e-health [17]–[20], intelligent transportation
composition in a three week mobile pilot with ten participants. systems [21], [22], smart grids [23], smart home [24], [25],
Our framework relies on ETSI M2M communications [11] as and smart city [26]. E-health aims to monitor and track
scalable and reliable infrastructure for interconnecting devices patients/users by means of sensors, where personal data is col-
and services, and openEHR [12] for storing and exposing lected and remotely analyzed by medical or processing centers.
health records. To the best of our knowledge, providing this Intelligent transportation systems or vehicular telematics, aim
use case on standard interoperable systems is also novel, thus for efficiency and safety of the transportation systems. They
we dedicate some space to describe our implementation and rely on devices in the vehicle and the monitoring and security
to characterize application data. We quantitatively evaluate the systems. The aim of smart grids is to leverage telemetry and
E2E latency as well as latency between system components increase the energy efficiency of house and buildings, while
and response times of the system which acquire a special fla- connecting billions of metering devices that collect and mon-
vor under dynamic and volatile environments characterized by itor the energy consumption, allowing its management. Smart
mobile networks. Although there have been attempts to mea- home appliances can be monitored and actuated upon using
sure and characterize M2M traffic in cellular networks [13], smart home services. Heating devices, televisions, etc., can be
recent works regarding latency in M2M systems only include shut on or off according to the users will. Finally, smart city
estimations [14], or generic methodologies for measurement is a concept from the service ubiquity to improve the quality
between two hosts [15]. They fail to predict, reproduce, and of life in the city.
1084 IEEE INTERNET OF THINGS JOURNAL, VOL. 4, NO. 4, AUGUST 2017

E-health services include, among others, ambient assisted for exhibition on users’ EHR. Implementation details and the
living for aging and incapacitated individuals [27], [28], detec- pilot deployment are presented in Section III-D.
tion of allergies and fatal adverse drug reaction [29], or
children’s health awareness by tracking their daily activi-
ties [30]. Other e-health concept frameworks for collection A. Background on Standards
and interoperation exist [31]–[33]. In [34]–[38] GWs are used ETSI [11] developed and defined a set of standards for
to interact with personal clinical devices, collecting, and trans- M2M communication middleware to enable the deployment
mitting signals. For example, in [38] an intelligent medicine of interoperable IoT applications, agnostic of the underlying
box acts as a GW, with several connection possibilities and networks and technologies. Transparency and interoperability
capable of connecting to various wearable sensors. It sends are attained by sharing common network resources and build-
data to a cloud, enabling clinical diagnosis, while the GW ing blocks, claiming at the same time to simplify development
itself can analyze and display all collected data. and deployment.
The combination of smartphones and the ubiquity and ETSI M2M provides a generic set of service capabili-
availability of cellular networks are driving IoT applications. ties (SCs): communication management, application manage-
Mobile phones have acquired powerful capabilities, in terms ment, service and device discovery and registration, device
of connectivity, battery, memory, or processing, and they management, data processing, security, etc. The architec-
are becoming an important platform for the proliferation of ture [11] settles on current network domain standards, but
health-related applications [39]. Several applications have been extends them with M2M applications and generic service capa-
proposed for tracking health-related information, e.g., heart bility layers (SCLs). SCs are functions shared by all entities
rate, blood pressure, blood oxygen saturation, stress levels, in the M2M ecosystem, and communications are made using
detection of asthma, chronic obstructive pulmonary disease, defined interfaces.
cystic fibrosis, coughing, allergic rhinitis, nose-related symp- Information is represented by resources, following a
toms of the respiratory tract, melanoma, and the analysis of RESTful architecture style [46], with the four basic CRUD
wounds in advanced diabetes patients [40], or for encourag- methods to handle resources: 1) CREATE; 2) RETRIEVE;
ing physical activity [41]. For example, the eCAALYX mobile 3) UPDATE; and 4) DELETE; plus two more additional meth-
application [39] aims at building remote monitoring for senior ods: 1) EXECUTE for executing a management command and
citizens with multiple chronic diseases. The mobile applica- 2) NOTIFY for reporting a notification about a change of a
tion acts as intermediary between the wearable health sensors resource to the subscribers of that resource. Any resource can
and the healthcare facility’s Internet site by transmitting the be manipulated by these methods. Thus, different applications
health-related measurements and position, or potential alerts. can access the same resources through the same reduced set of
Those applications keep using proprietary and closed solu- operations, something that cannot be achieved when dedicated
tions, neglecting the possible positive effects that standard- infrastructures are used. In REST, resources have a particular
ization could bring to the commercialization of such appli- state during time (they are mutable over time) and are uniquely
cations. Current services and applications abstract standards, addressable using a universal resource identifier. Furthermore,
either M2M or health-related. Although some provide E2E manipulation of data and data exchange between entities are
interoperation, the lack of using known standards reduce the all stateless. In ETSI M2M, each SCL hosts resources in a
possibilities for integration with other systems. hierarchical tree structure, where information is maintained.
Performance evaluation of latency in e-health applications ETSI M2M supports two communication models:
using cellular networks exist in terms of simulations [42], [43], 1) request-response and 2) publish-subscribe. Sensing
but the systems are limited and use very simplistic models. and remote monitoring makes up a large batch of IoT
On the other side, real testbeds implementation also exist, but applications. They can be more efficiently implemented with
performance is left out [44], [45]. Real performance of M2M, the publish-subscribe model than with the request-response
in a context of IoT applications, in terms of quality of service model that would forcedly be used for polling. Time and
and latency evaluation, to the best of our knowledge, is not space decoupling allows greater scalability and flexibility
present in literature. Works involving latency performance in for the IoT than polling-based communications, and allows
M2M middleware only include estimations for a specific set a more dynamic topology [47]. The standard defines how
of examples [14] or methodologies for measurement between subscriptions are made and allows subscriptions at different
two hosts [15], and fail to capture realism of measurements. levels of the hierarchical resource model. Subscriptions are
represented and can be operated upon as resources under the
resource tree.
III. S YSTEM D ESIGN OpenEHR [12] is a nonproprietary standard architecture
In this section, we describe the design and implementation in health informatics aiming at interoperability in e-health.
of the system for providing the mobile e-health applica- OpenEHR allows the standardization of EHR architecture fol-
tion. Next, we summarize some important concepts of M2M lowing a multilevel modeling approach which separates infor-
communications and openEHR. We model our storyboard mation from knowledge [48]. For that, openEHR Foundation
as a service choreography and map it into standard M2M published a set of specifications [49] that define a health infor-
middleware in Section III-B. Section III-C summarizes the mation reference model, archetypes, and a query language. In
information that is collected and processed from participants openEHR, an archetype is the model for the capture of health
PEREIRA et al.: EXPERIMENTAL CHARACTERIZATION OF MOBILE IoT APPLICATION LATENCY 1085

Fig. 3. Message sequence during the pilot.

Fig. 2. System design. Arrows represent user data flow.

the respective resource in the NSCL.1 As data processor

information [50]. Example archetypes can be found in [51]. and openEHR services do not possess M2M capabilities,
OpenEHR mandates openness in terms of data, models, and we enable them with M2M capabilities with the use of
APIs for systems and components. Being independent of soft- NA libraries (NAlib). M2M GWs communicate with the
ware applications and technology changes [52], it provides NSCL using the mId interface, while the data processor and
interoperability between health information systems, avoiding the openEHR communicate with the NSCL using the mIa
dispersion and limitations of proprietary technical solutions, interface [11]. Access control is verified on resource access
and allowing flows between distributed systems. by the NSCL, and TLS is used to ensure privacy and security.
The data processor subscribes the raw sensor data at the
NSCL, and is notified of all data published by GWs. Once
B. Mapping Storyboard to M2M per day, the data processor processes the data from each user,
The storyboard was mapped and deployed into an ETSI formats it as required by the openEHR, and publishes it at the
M2M ecosystem, as depicted in Fig. 2. Wearable devices, like NSCL. The NSCL then notifies the openEHR. Fig. 3 shows
the chest band heart rate monitor (Zephyr HxM BT [53]) used the normal message sequence between the entities during the
in this paper, usually do not possess M2M capabilities and pilot.
do not use standardized interfaces. Thus, they cannot “play” Initially, every entity registers itself at the NSCL. Each
directly in the M2M ecosystem. These devices are termed M2M GW registers itself under its own SCL using its
legacy devices. They need to connect first to M2M GWs that smartphone’s serial number for unique identification,
act as proxies or concentrators on their behalf. Users’ smart- /scls/<SerialNumber>/, below the SCL base at the NSCL at
phones can act as M2M GWs, collecting data from sensors, https://193.136.29.19/m2m. M2M GWs publish data to /scls/
embedded, or connected via BT, or another body area network <SerialNumber>/applications/BT-ZEPHYR/containers/DATA/
technology. The M2M GW run two software components, an contentInstances/. Data processor and openEHR register
M2M gateway application and an M2M gateway SCL, that are themselves directly at the applications resource collection
required to manage sensors and data on the M2M device and below the SCL base at /applications/DataProcessor/ and
GW domain. The M2M GWs periodically send/publish data /applications/openEHR/, respectively. Data processor pub-
to the network SCL (NSCL), whose main function is to act as lishes each user data at http://applications/DataProcessor/
a broker.
M2M network applications (NAs) are services in the M2M
1 If necessary, M2M NAs are also able to send commands to M2M GWs,
network domain that run the application logic using M2M
e.g., alerts or behavior commands, following the approach presented in [54],
capabilities. Every NA that is interested in receiving data but we do not use this functionality in our use case. For efficiency reasons
exposed through the M2M middleware must subscribe to M2M GWs can be set to transmit only after receiving subscriptions.
1086 IEEE INTERNET OF THINGS JOURNAL, VOL. 4, NO. 4, AUGUST 2017

http://containers/<user>/contentInstances/, where <user> using IP instead of domain names to reduce possible delays
corresponds to an integer value between 0 and 9. of using DNS.

IV. M EASURING L ATENCIES

C. User Data
Latency or one-way delay (OWD) [58] refers to the amount
Data processor receives notifications from NSCL contain- of time for data to travel one way from one point to
ing the raw sensor data. It processes and formats the data to another [58]. Latency measurements can be achieved by non-
the correct archetype elements to be published at the NSCL intrusive (or passive) or intrusive (or active) measurements.
and forwarded to the openEHR. This way, the templates at In this paper, we use nonintrusive measurements, i.e., we use
the openEHR are filled with the most relevant information, real network traffic to measure latencies. The use of latency is
and users’ physical activity and heart rate condition can be advisable for complex networks due to the existence of asym-
observed by medical personnel. Archetypes are a specification metric paths or different routing or queueing policies [58].
for a single clinical concept. The specification are expressed However, latency measurements require comparable times-
in archetype definition language which is an ISO standard. tamps through internally synchronized clocks, and access to all
Example archetypes can be found in [51]. Although archetypes system components to correlate sending and reception infor-
can be defined by healthcare units or other medical personnel, mation. We did not have this kind of synchronization at the
for evaluation purposes, we have defined and implemented two GWs, and we did not have access to the openEHR machine.
types: 1) one heart rate and 2) one mobility. So, comparing all timestamps to estimate one-way delays was
Every parameter of the archetypes must include the units, a challenge. In the rest of this section, we describe how we
the type of the data (e.g., float, integer, etc.), limits, and measured latency for the different segments given the different
expected normal limits for detection of critical situations. possibilities and accesses.
Besides the two archetypes, data processor includes some
additional data necessary for the correct interpretation of A. Synchronization
openEHR. With these two archetypes, clinical personnel, or
any other service provider, can obtain a detail information The most common method for clock synchronization over a
of the user’s heart rate information, mobility, and general distributed network is network time protocol (NTP) [59]. NTP
lifestyle. is a well-known protocol used to synchronize the clock of a
client to a reference time source, and it allows up to sub-
millisecond synchronization [60]–[62]. We use timestamps
D. Pilot Setup and Implementation with millisecond precision, so an NTP precision near 1 ms
Along three weeks ten volunteers used a Moto 2g [55] suffices. Data processor, NSCL, and openEHR were synchro-
smartphones running Android 4.4.4 KitKat [56] and a chest nized using NTP. However, the M2M GWs were not, so we
band heart rate monitor, Zephyr HxM BT, following the pro- had to find an alternative way to estimate the latency between
cedure presented in the storyboard. The users were willing to them and the NSCL.
follow the procedure during the week days, i.e., from Monday An option to estimate latency between points that are
to Friday, preferably during 8 h. The users signed an informed not synchronized is using the half of the round-trip
consent for the collection of personal data. Mapping between time (RTT) [63] measurements, assuming that paths are sym-
users and smartphones were not disclosed, and therefore, we metric. RTT is measured as the time interval between sending
can consider it as pseudoanonymous collection. a packet and receiving its acknowledgment at the same point,
All communications use HTTP over TCP/IP. Publications, and thus, it includes processing times at the remote location.
subscriptions, and notifications are made with a POST method Recent measurements show that halving RTT can introduce
which is followed by an acknowledgment message from the tens of milliseconds of delay asymmetry (larger than our
receiver. We estimate that the use of CoAP [57] instead would precision), and that asymmetries are higher for commercial
lead to a decrease of 28% of transmitted application protocol networks than education and research networks [64]. Echo
headers, without considering retransmissions. request and reply mechanisms (ping) of the Internet control
M2M GWs periodically transmitted M2M data using a message protocol [65] are an alternative commonly used to
UMTS or GPRS cellular network selected automatically estimate RTT network latencies. Another possible solution is
according to the network availability and state. Although pro- to use TCP ping which uses the SYN/ACK or SYN/RST
viding lower data rates, GPRS is a solution commonly adopted mechanisms in the TCP handshake. These techniques pro-
by telcos for M2M traffic, and a majority of M2M devices use vide an estimate of network distance and latency, but use only
it [13]. Whenever an M2M GW experienced periods with- very short control packets being intrusive (though with negli-
out connectivity, it would continue to collect sensor data and gible impact). Thus, we halve the RTT to estimate the latency
attempt to publish at the defined transmission period. between GW and NSCL during the pilot.
NSCL is installed in a machine running CentOS 7 in our
department facilities. The data processor runs in a machine B. Timestamping
hosted at a commercial cloud provider with Ubuntu 14 LTS, Table I shows the timestamping points for our measure-
and openEHR is installed in a machine at the nearest medical ments. We acquired timestamps at kernel- and application-
school, less than 2 km away. All communications are made levels at NSCL and data processor since we had full
PEREIRA et al.: EXPERIMENTAL CHARACTERIZATION OF MOBILE IoT APPLICATION LATENCY 1087

TABLE I
P OINTS AND L EVELS W HERE T IMESTAMPS W ERE ACQUIRED
D URING THE E XCHANGING OF DATA PACKETS
B ETWEEN THE D IFFERENT E NTITIES

access to them. Due to limited access we could not acquire

kernel-level timestamps at openEHR and we did not acquire
timestamps at kernel-level at M2M GWs either due to the need
for root.
Timestamps measured at the kernel-level are the clos-
est time we can acquire before the packet is passed to
(or after the packet is received from) the network device.
Timestamp measurements at the kernel-level were acquired
using TCPDUMP [66] that uses libpcap [66]. We acquired
timestamps at application-level in all entities before data was
passed to (or received from) the M2M libraries. These libraries
lay between the application and the kernel. Timestamp mea-
surements at the application-level were acquired using the
currentTimeMillis Java method [67]. While E2E latency mea-
surements tend to choose application-level timestamp-based Fig. 4. Latency measurement between entities presented. Dashed lines rep-
measurements, pure network latency/delay measurements resent transmissions or receptions at kernel-level between entities which is
tend to choose kernel-based measurements [15]. Performing the closest to the network times.
application-level measurements allows us to include the inher-
ent delay of the application protocol libraries, e.g., accessing
or using libraries, queuing processes, etc. Interactions between
the application-level and the kernel-level are performed on a
per-packet basis and, therefore, we can acquire the timestamps
of each packet at each level and quantify the time differences
between levels.
For latency measurements between application and kernel-
levels at the same entity, we calculate the difference of
time between the timestamp of the HTTP data sent at the
application-level and the timestamp of the last TCP segment
with HTTP data sent at the kernel-level. For latency mea-
surements between kernel and application-levels at the same
entity, we calculate the difference of time between the times-
Fig. 5. Comparison between the time each GW was running and the actual
tamp of the first TCP segment with HTTP data received at the active time when it was collecting sensor data and transmitting.
kernel-level and the timestamp of the HTTP data received at
the application-level.
Table II depicts the average latencies with 95% confi- users for a total of 1146 h, from which 479 h were actual
dence intervals between system components presented in the data collection and transmission (i.e., active time), corre-
following section, and Fig. 4 shows how we calculate them. sponding to 42% of the total. During measurements, users
experienced some sensor device malfunctioning which caused
V. R ESULTS the time differences between operation and actual collec-
tion and transmission. In some cases, the chest band heart
In this section, we present a characterization and
rate monitor indicated full battery when it was not the
quantification of the data and latency times, as the most rel-
case, leading to short or even null periods of operation.
evant results obtained from the pilot measurements. We did
The very low active time for M2M GW7 is explained due
not observe significant changes on the memory or CPU usage
to the user leaving its application running without the BT
during the pilot on any device, and thus, we do not present
device being connected. Figs. 5 and 6 show that M2M
those results. We discuss the results at the end of the section.
GWs with larger active times transmit more information, as
expected.
A. Quantification of Application Protocol Overheads In total, M2M GWs received approximately 6.3 MB of
M2M GWs produced 22 759 publications. Fig. 5 shows HTTP data and transmitted approximately 65.6 MB of HTTP
the amount of hours monitored by each GW. We monitored data, as shown in Fig. 6. From the latter, 43.3 MB were the
1088 IEEE INTERNET OF THINGS JOURNAL, VOL. 4, NO. 4, AUGUST 2017

TABLE II
N OTATIONS AND E QUATIONS OF L ATENCIES M EASURED B ETWEEN S YSTEM C OMPONENTS

Fig. 7. CDF of the average application-level latency between the entities

that compose the E2E latency.
Fig. 6. Amount of HTTP data sent and received at each GW.

M2M middleware. E2E latency can be split into the laten-

actual compressed sensor data, corresponding to 430 MB of cies between neighbor system components. Fig. 7 shows the
uncompressed data. The total HTTP protocol headers trans- latency splits between the entities that compose the E2E
mitted during the measurements corresponded to 3.2 MB of latency. We decomposed the E2E in three parts: 1) the latency
the total transmitted data. The M2M resource structure paths between M2M GW and NSCL; 2) the latency between NSCL
transmitted were 19.1 MB which corresponded to 29% of the and data processor; and 3) the latency between data processor
total transmitted data. This shows that protocol headers are and openEHR
only a small part of the total overhead. On the other hand, E2E GW,NSCL NSCL,DP DP,EHR
resources are a large part. If we had used CoAP we estimate OWDA = OWDA + OWDA + OWDA .
approximately 1 MB savings of application protocol headers
and a saving of 1.5% of the total transmitted data which is The average application-level latency between an M2M GW
not as relevant as one would expect. However, as the resource i publication and the respective acknowledgment from NSCL
paths were large and the HTTP protocol headers were rela- is 0.9671 s. The average application-level latency of a notifi-
tively small, the use of CoAP would not provide significative cation from NSCL to data processor is 0.0138 s. The average
advantages. application-level latency of a publication from data proces-
sor to openEHR is 0.3130 s. The last value is obtained by
averaging the time differences between the application layer
B. Characterization of the E2E Latency timestamps of the reception of the k notification at openEHR
We start by looking into the E2E latency experienced by and of the transmission of the k publication at data processor.
data produced at a GW that would be processed at the This latency shows that message passing between services can
data processor and then delivered to the openEHR using the be a large component in the user-perceived latency.
PEREIRA et al.: EXPERIMENTAL CHARACTERIZATION OF MOBILE IoT APPLICATION LATENCY 1089

In our system, a ping probing between data processor and

NSCL had an RTT average of 5.7 ms, and a ping probing
between NSCL and openEHR had an RTT average of 2.0
DP,EHR
ms. We see that the latency between services OWDA ,
which includes the latency impact of the broker and the appli-
cations, extends the latency estimated by ping by 8000%. This
shows that, although ping is an invaluable tool for having a
first impression of network latencies, it is not representative
of actual application-level latencies.
The latencies that compose the E2E latency fol-
low a normal distribution, verified using one-sample
Kolmogorov–Smirnov [68], [69] and Lilliefors [70] normal-
Fig. 8. Average application-level latency between M2M GW publications
ity tests. Thus, the average application layer E2E latency is and reception at NSCL and the traveled distance per day for each GW.
the sum of the average of these latencies
E2E
OWDA = 0.9671 + 0.0138 + 0.3130 = 1.2939 s.
D. Impact of Mobility
Latency between M2M GWs and NSCL makes up for 75% of Mobility is inherent to any mobile communication system.
the total E2E latency. Mobility introduces handovers, increase of packet loss, and
link failure. Fig. 8 shows the average application-level latency
C. Quantification of the Impact of the Broker between M2M GW publications and reception at NSCL and
The sum of the average latencies between kernel- and the traveled distance per measurement day for each GW.
application-levels and between application- and kernel-levels Average latency varied between close to half second and one
of NSCL, plus the time for processing the subscriptions, gives and half seconds. We observe a similar trend between the
us an estimate of the NSCL forwarding latency, i.e., the latency magnitude of the traveled distance and the average latency
introduced by NSCL receiving and forwarding data experienced for each GW. We obtain 0.67 (n = 10, p < 0.04,
NSCL NSCL NSCL NSCL 95% CI: 0.068 − 0.914) for Pearson’s correlation coeffi-
OWDForward = OWDK,A + OWDProc + OWDA,K
cient [71] between traveled distance and latency values. On
NSCL the other hand, M2M GW4 and M2M GW7 were located
where OWDProc is the average latency of the NSCL
application for processing subscriptions (publications and 300 km apart from the rest of the GWs during measurements.
notifications). We observe that the average latency between But their average experienced latency does not differ from the
kernel- and application-levels of NSCL when receiving pub- global average, though they follow the same trend of mobility.
lications on the mId interface is one order of magnitude These results indicate that the average latency values
smaller than when receiving publications on the mIa interface. between M2M GWs and NSCL: 1) depend on the M2M
Therefore, we quantify the impact of the broker taking into GW mobility and 2) are independent of the physical distance
consideration the two situations in separate. If we consider between GW j and the NSCL for the considered distances.
publications received only on the mId interface We assume that this observation is mainly due to the selection
NSCL NSCLmId NSCL NSCLmIa
of GPRS and/or vertical handovers introduced by the increase
mId
OWDForward = OWDK,A + OWDProc + OWDA,K of mobility; however, we do not have enough data to confirm
= 0.0106 + 0.0068 + 0.0074 this assumption.
= 0.0248 s
E. Quantification of the Impact of the Promotion Delay
while for publications received only on the mIa interface
We study the impact on the E2E latency of the GW’s
NSCLmIa NSCLmIa NSCL NSCLmIa
OWDForward = OWDK,A + OWDProc + OWDA,K network interface state at the moment when the applica-
= 0.1399 + 0.0068 + 0.0074 tion schedules transmission by quantifying the impact of the
promotion delay of the access network, under controlled mea-
= 0.1541s.
surements. The promotion delay is the time needed for the
Application-level processing and forwarding at the NSCL network to allocate network resources to the GW’s network
alone introduces on average at least 25 ms of latency for publi- interface. Although other works [72], [73] have characterized
cations received on the mId interface, but can be ≈520% larger promotion delay, we cannot apply it to our measurements as
for publications received on the mIa interface. Considerable different networks carriers adopt different state machine mod-
differences between forwarding latencies in different interfaces els with varied parameters. To complement our results, we also
can occur, and they depend of specific broker implementations. quantify latencies between M2M GW and NSCL as well as
Thus, the impact of the broker forwarding latency should be expectable E2E latencies for different access networks. Thus,
always measured separately in different interfaces to provide we performed additional measurements for different access
realistic performance guarantees. Additionally, we see that ser- networks using a single GW and following a different pro-
vice composition via broker may add significant latency to the cedure for transmissions, depicted in Fig. 9. Now, the M2M
application response time, even if the broker is not overloaded. GW is completely static and it is not used for any personal
1090 IEEE INTERNET OF THINGS JOURNAL, VOL. 4, NO. 4, AUGUST 2017

TABLE III
M EASURED P ROMOTION D ELAY FOR W I -F I , GPRS, UMTS,
AND LTE, IN A S TATIC AND C ONTROLLED E NVIRONMENT

Fig. 9. Message sequence to measure the impact of the GW’s network TABLE IV
interface state at the average application-level RTT between the M2M GW AVERAGE A PPLICATION -L EVEL L ATENCY B ETWEEN THE M2M GW AND
and NSCL in a static and controlled environment. NSCL W HEN THE GW’ S I NTERFACE S TATE I S ON AND I DLE FOR W I -F I ,
GPRS, UMTS, AND LTE, IN A S TATIC AND C ONTROLLED
E NVIRONMENT. R IGHT C OLUMN S HOWS THE E XPECTABLE
E2E L ATENCY FOR THE S AME S CENARIOS BY I NCLUDING
THE L ATENCY B ETWEEN THE R EST OF THE E NTITIES
T HAT C OMPOSE THE A PPLICATION

Fig. 10. Average application-level RTT between the M2M GW and NSCL
for the first and the subsequent four publications of each sequence for Wi-Fi,
GPRS, UMTS, and LTE, in a static and controlled environment.

use which eliminates any background traffic. The stationari-

ness reduces possible random delays originated from mobility.
The M2M GW transmits periodically a set of five publications, with [75]–[77] that argue that network access in 2G and 3G
each one transmitted only after the reception of the acknowl- dominates overall E2E latency.
edgment of the previous publication, followed by a period of Keeping the interface state on and eliminating the promotion
time without any transmissions to guarantee the NIC returns delay allows to reduce latency for all technologies. Table III
to an idle state. shows the measured promotion delay. These values are in con-
We measured using a Wi-Fi connection to a high-speed formity with previous works which obtained promotion delay
fixed backbone, a 2G GSM/GPRS cellular network, a 3G values of approximately 0.080 s for Wi-Fi, 1 s for GPRS, 2 s
UMTS cellular network, and a 4G LTE cellular network, all for UMTS, and 0.260 s for LTE [72], [73], respectively. While
of the same Internet provider, for almost 2 h, and we acquired our results differ in magnitude for GPRS and UMTS, the
the application-level RTT between the M2M GW and NSCL. ratio between them is similar to previous work. Nevertheless,
All cellular networks had two GWs between the base station this confirms that it is not advisable to reuse network time
and the Portuguese Internet eXchange (GigaPIX), the location parameters across different networks.
where different IP networks exchange traffic. The average latency between M2M GW and NSCL, when
The first publication of each set is affected by the promo- the GW’s interface state is on and idle at the moment when
tion delay. The remaining publications (2–5) of each set are the application schedules transmission, for the different tech-
not affected by the promotion delay. The results are shown nologies is presented in Table IV. Overall, E2E latencies using
in Fig. 10. Overall, we can observe there is a significant dif- Wi-Fi stay below 0.5 s for both scenarios, and E2E latencies
ference between the average RTT of the publications affected using UMTS and LTE are below 0.5 s only for an on state.
by the promotion delay and the average RTT of the publica- E2E latencies using GPRS and UMTS are clearly above 1 s
tions not affected by the promotion delay for all technologies. for an idle state. Furthermore, the access network does not
The average RTT using GPRS was the highest for the publi- dominate the E2E latency in any case when using Wi-Fi.
cations affected by the promotion delay, followed by UMTS, Applications with large transmission periods or aperiodic
and finally, LTE and Wi-Fi. Wi-Fi and LTE offer much lower transmissions, like monitoring/sensing or event-based applica-
average RTT (almost 1 s less) than 2G and 3G cellular tions, do not drive the network interface to be constantly on
networks. We can observe that the TCP three-way handshake, and suffer promotion delay at every network access. Therefore,
as expected [74], is not responsible for the high average RTT, they will experience latency values between the M2M GW
since it would affect all technologies in a similar way. In any and NSCL similar to the average latency measured for the
case, alternatives to the TCP three-way handshake could be publications affected by promotion delay. For these type of
the use of UDP and CoAP. These findings are in conformity applications the access network will dominate the E2E latency.
PEREIRA et al.: EXPERIMENTAL CHARACTERIZATION OF MOBILE IoT APPLICATION LATENCY 1091

On the other hand, applications with small transmission peri- when radio devices have the network interface idle, they must
ods can drive the network interface to be constantly on and synchronize and negotiate radio resources with the nearby
do not suffer promotion delay. Thus, they will experience radio tower. Even though large transmission periods are associ-
latency values between the M2M GW and NSCL similar to ated with a lower battery consumption, time requirements can
the average latency measured for the publications not affected be compromised due to latency in the access network intro-
by promotion delay. In this case, the dominant part of the duced by the promotion delay. Moreover, the extent of the
E2E latency is the latency between NSCL, data processor, and promotion delay depends on the technology used. In case time
openEHR, i.e., the latency between services, when considering requirements need to be guaranteed to, for example, detect or
low latency technologies as LTE, Wi-Fi, or even UMTS. react to alarm situations, the solution can be to maintain the
The network access plays an important role even for appli- interface on by using TCP SYN.
cations with small transmission periods when considering high Further latency occurs at the core network as packets must
latency technologies, such as GPRS. Nevertheless, the average flow from the radio tower to the packet GW, enhanced by
latency values measured for the publications without promo- the restricted routing topology of cellular networks observed
tion delay using GPRS approximate the ones measured for the in [77]. This advises locating content servers inside cellular
publications with promotion delay using LTE. Furthermore, networks or as close as possible to the packet GW. However,
they are considerably smaller than the average latency values in an open ecosystem, there is no desire to guarantee that
for publications with promotion delay measured using UMTS. services to be composed will be located inside a single cellu-
lar operator’s network. An alternative may be the availability
of multiple peering points in the cellular backbone, with the
F. Discussion inherent costs and challenges.
As M2M standards are usually a one-size-fits-all and tel- LTE replaces the two-layered radio access network architec-
cos need to guarantee average service, latency serves as a ture of 3G into a single-layered architecture reducing overall
performance indicator. We showed that it is possible to provide latency [73]. Future IoT applications for mobile scenarios
an IoT service composition on top of standards in a context of should adopt lower latency cellular networks, such as LTE, to
mobility; however, the high E2E latency observed can disrupt provide services with real-time requirements. However, recent
user experience. We observed average E2E latency for the IoT LTE measurements showed that it is less energy efficient dur-
service composition during the mobile pilot of roughly 1.3 s. ing idle state and for transferring smaller amounts of data [73].
This value falls near estimations obtained in [14], where E2E Nevertheless, in scenarios of heterogeneous networks, the
latency between an M2M GW and a single M2M applica- exploitation of low latency and power efficiency tradeoffs
tion using an LTE network could reach up to 1.5 s. Thus, offered by most wireless networks will be mandatory. Thus, it
for interactive applications that involve a response from a ser- is advisable to enhance the networking API of mobile devices
vice on the network domain to a mobile device, the expected with the possibility to choose among available connectivity
response time could be twice this much, with impact on the opportunities, and/or to request specific service guarantees,
user experience. High average application-level RTT between exploring service level awareness that are based on different
M2M GWs and NSCL additionally means that devices will traffic modes [82].
have their network interface powered on more time, as it does Finally, the paradigm shift envisioned by the Tactile
not return to an idle state between transmission and reception Internet, providing ultrareliable and ultraresponsive connec-
of data. Ultimately, this leads to higher battery consump- tivity, will force a transition from LTE to 5G communications
tion [13], [73]. We also showed that ping is not an admissible to achieve the desired RTT in the order of 1 ms [83]–[86].
tool for estimating latency between services. This paradigm shift may become a key enabler of real-time
We identified the access network of the GW as the interactive systems and allow the development of new and
main bottleneck. According to our results, E2E latencies innovative IoT applications with smaller time requirements.
between a smartphone GW and cloud hosted services vary
largely and can exceed 1 s. Also, they depend on the GW’s
network interface state due to promotion delay and mobility.
Nevertheless, further research with a larger number of users
should be conducted to better characterize the impact of mobil- VI. I MPORTANT O BSERVATIONS
ity on the application latency. Critical applications are latency In this section, we present important observations from
sensitive and demand low latency. If we consider that certain e- designing and implementing this service composition that we
health applications may tolerate up to 1 s of E2E latency, such believe to be useful for other researchers and practitioners
as for ECG data [78]–[80], then, according to Table IV, we when developing IoT applications.
just cannot use GPRS or UMTS for infrequent transmissions • Standardization: M2M standardization foster the emer-
that may lead to idle states. However, e-health applications gence of IoT applications by providing interoperability, as
with stricter E2E latency requirements, such as 500 ms [81], discussed previously. Standardization and interoperability
can face latency problems if any type of cellular networks is allowed us to create M2M libraries for use in differ-
to be used. ent machines, while saving development and deployment
IoT applications should carefully coordinate the network time. In this concrete case, the NAlib at openEHR was
access for latency performance. In every transmission, deployed by a developer which had no familiarity with
1092 IEEE INTERNET OF THINGS JOURNAL, VOL. 4, NO. 4, AUGUST 2017

M2M standards; however, the NALib and its function- requirements and that this may be a tradeoff with functionality/
alities allowed a fast deployment and integration of this complexity. Finally, we highlight some lines along which the
NA. current developments in networking may contribute to address
• Access Limitations: We faced several access limitations some of the problems identified. Future work should focus
to different parts of the system, e.g., having access for on devising algorithms to schedule transmissions in order to
enabling services, such as NTP or TCPDUMP, or opening accommodate the tradeoff between battery consumption and
TCP ports for receiving notifications. In IoT systems, it is fulfilling deadlines, in light of the findings in this paper.
unlikely that anyone will have full access to all systems. Algorithms should explore the tradeoffs between latency and
So, performance evaluation of IoT systems must design response time obtained when the network interface is on or
experiments to deal with the limitations of each system. idle and the resource usage associated with the changes in the
• Application Design: Developers must be aware that data network interface state.
collection from a smartphone’s BT or internal sensors
can have a significant impact on battery life [87]. Even ACKNOWLEDGMENT
the choice between using RAM and hard drive to store
The authors would like to thank the volunteers for making
sensor data can have impact on battery consumption [2].
this pilot possible and their helpful feedback, and the anony-
Therefore, application developers need to pay close atten-
mous reviewers for their careful reading of this paper and their
tion to resource usage optimizations. The constrained
insightful comments and suggestions.
nature of devices and networks introduces tradeoffs, such
as transmission frequency and battery consumption and
the specific application time-requirements. R EFERENCES
• Pilot Study Design: During measurements, users experi- [1] C. Pereira and A. Aguiar, “Towards efficient mobile M2M commu-
enced some sensor device malfunctioning which caused nications: Survey and open challenges,” Sensors, vol. 14, no. 10,
pp. 19582–19608, Oct. 2014.
the time differences between operation and actual col- [2] C. Pereira et al., “Smartphones as M2M gateways in smart cities IoT
lection and transmission which ultimately led to short applications,” in Proc. 23rd Int. Conf. Telecommun. (ICT), Thessaloniki,
or even null periods of operation. A previous study that Greece, 2016, pp. 1–7.
[3] Gartner. (2015). Press Release: Gartner Says 6.4 Billion Connected
assessed the feasibility of continuous stress measure- “Things” Will Be in Use in 2016, Up 30 Percent From 2015. [Online].
ment using wireless physiological sensors [88] observed Available: http://www.gartner.com/newsroom/id/3165317
a significant learning effect after the first week of [4] G. Wu, S. Talwar, K. Johnsson, N. Himayat, and K. D. Johnson, “M2M:
From mobile to embedded Internet,” IEEE Commun. Mag., vol. 49, no. 4,
use. However, two differences to this paper are evi- pp. 36–43, Apr. 2011.
dent. In that study, users had to sporadically interact [5] S. Umer, M. Afzal, M. Hussain, K. Latif, and H. F. Ahmad,
with the smartphone, and were monetarily compensated. “Autonomous mapping of HL7 rim and relational database schema,”
Inf. Syst. Front., vol. 14, no. 1, pp. 5–18, Mar. 2012.
Our application is designed to be nonintrusive and the [6] A. Soceanu, A. Egner, and F. Moldoveanu, “Towards interoperability of
M2M middleware intended for reduced or no human ehealth system networked components,” in Proc. 19th Int. Conf. Control
intervention. Therefore, it was easy to “forget” the appli- Syst. Comput. Sci. (CSCS), Bucharest, Romania, 2013, pp. 147–154.
[7] G. L. Kreps and L. Neuhauser, “New directions in eHealth communi-
cation. Moreover, all participants were volunteers and cation: Opportunities and challenges,” Patient Edu. Counsel., vol. 78,
did not receive any form of compensation. For future no. 3, pp. 329–336, Mar. 2010.
implementations we advise researchers to add control [8] R. J. Cruz-Correia et al., “Reviewing the integration of patient data: How
systems are evolving in practice to meet patient needs,” BMC Med. Inf.
mechanisms to assure data quality locally and pro- Decis. Making, vol. 7, no. 1, p. 14, Jun. 2007.
vide feedback/notification of possible malfunctioning for [9] C. Pereira, S. Frade, P. Brandão, R. Correia, and A. Aguiar, “Integrating
users. Finally, some form of user interaction or incentives data and network standards into an interoperable e-health solution,” in
for participation increases the engagement of participants Proc. IEEE 16th Int. Conf. e-Health Netw. Appl. Services (Healthcom),
Natal, Brazil, 2014, pp. 99–104.
or users. [10] Y. Li and C. Lin, “QoS-aware service composition for workflow-
based data-intensive applications,” in Proc. IEEE Int. Conf. Web
Services (ICWS), Washington, DC, USA, 2011, pp. 452–459.
VII. C ONCLUSION [11] ETSI TS 102 690 V2.1.1 (2013-10) Machine-to-Machine
Communications (M2M); Functional Architecture, ETSI,
We presented an experimental characterization of latency Sophia Antipolis, France, 2013.
and application overheads in an IoT application of service [12] openEHR. Accessed on Oct. 15, 2016. [Online]. Available:
http://www.openehr.org/
composition with mobile GWs. For that, we designed and [13] M. Z. Shafiq, L. Ji, A. X. Liu, J. Pang, and J. Wang, “Large-scale mea-
implemented a mobile e-health use case, combining ETSI surement and characterization of cellular machine-to-machine traffic,”
M2M communications and openEHR, that monitored ten peo- IEEE/ACM Trans. Netw., vol. 21, no. 6, pp. 1960–1973, Dec. 2013.
[14] N. Nikaein and S. Krea, “Latency for real-time machine-to-machine
ple for three weeks. We observed that M2M resource structure communication in LTE-based system architecture,” in Proc. 11th Eur.
paths contribute a lot to the overhead, but protocol headers Wireless Conf. Sustain. Wireless Technol. (Eur. Wireless), Vienna,
only a small part. Further, latency between the smartphone Austria, Apr. 2011, pp. 1–6.
[15] J. Fabini and T. Zseby, “M2M communication delay challenges:
and broker contributes to a large portion of the E2E latency, Application and measurement perspectives,” in Proc. IEEE Int. Instrum.
and depends on the mobility. We also verified that the access Meas. Technol. Conf. (I2MTC), Pisa, Italy, 2015, pp. 1859–1864.
network makes up most of that latency due to the promo- [16] G. Z. Papadopoulos, K. Kritsis, A. Gallais, P. Chatzimisios, and
T. Noel, “Performance evaluation methods in ad hoc and wireless sen-
tion delay. We conclude that M2M middleware should provide sor networks: A literature study,” IEEE Commun. Mag., vol. 54, no. 1,
means to support low latency for applications with real-time pp. 122–128, Jan. 2016.
PEREIRA et al.: EXPERIMENTAL CHARACTERIZATION OF MOBILE IoT APPLICATION LATENCY 1093

[17] Z. Fan, R. Haines, and P. Kulkarni, “M2m communications for e-health [40] S. M. R. Islam, D. Kwak, M. H. Kabir, M. Hossain, and K.-S. Kwak,
and smart grid: An industry and standard perspective,” IEEE Wireless “The Internet of Things for health care: A comprehensive survey,” IEEE
Commun., vol. 21, no. 1, pp. 62–69, Feb. 2014. Access, vol. 3, pp. 678–708, Jun. 2015.
[18] K.-C. Chen, “Machine-to-machine communications for healthcare,” [41] S. Consolvo et al., “Flowers or a robot army? Encouraging awareness
J. Comput. Sci. Eng., vol. 6, no. 2, pp. 119–126, Jun. 2012. & activity with personal, mobile displays,” in Proc. 10th Int. Conf.
[19] ETSI TR 102 732 V1.1.1 (2013-09) Machine-to-Machine Ubiquitous Comput., Seoul, South Korea, 2008, pp. 54–63.
Communications (M2M); Use Cases of M2M Applications for [42] D. Vouyioukas, I. Maglogiannis, and D. Komnakos, “Emergency
eHealth, ETSI, Sophia Antipolis, France, 2013. m-health services through high-speed 3G systems: Simulation and
[20] E. Kartsakli et al., “A survey on M2M systems for mhealth: performance evaluation,” Simulation, vol. 83, no. 4, pp. 329–345,
A wireless communications perspective,” Sensors, vol. 14, no. 10, Apr. 2007.
pp. 18009–18052, Sep. 2014. [43] K. Kang, Q. Wang, J. Hur, K.-J. Park, and L. Sha, “Medical-grade quality
[21] M. Gerla, E.-K. Lee, G. Pau, and U. Lee, “Internet of vehicles: From of service for real-time mobile healthcare,” Computer, vol. 48, no. 2,
intelligent grid to autonomous cars and vehicular clouds,” in Proc. IEEE pp. 41–49, Feb. 2015.
World Forum Internet Things (WF IoT), Seoul, South Korea, 2014, [44] G. J. Mandellos, M. N. Koukias, I. S. Styliadis, and
pp. 241–246. D. K. Lymberopoulos, “e-SCP-ECG+ protocol: An expansion on
SCP-ECG protocol for health telemonitoring—Pilot implementation,”
[22] L. Yongfu, S. Dihua, L. Weining, and Z. Xuebo, “A service-oriented
Int. J. Telemed. Appl., vol. 2010, no. 3, pp. 1–17, Mar. 2010.
architecture for the transportation cyber-physical systems,” in Proc. 31st
[45] S.-J. Jung, R. Myllylä, and W.-Y. Chung, “Wireless machine-to-machine
Chin. Control Conf. (CCC), Hefei, China, 2012, pp. 7674–7678.
healthcare solution using android mobile devices in global networks,”
[23] Y. Yan, Y. Qian, H. Sharif, and D. Tipper, “A survey on smart grid com-
IEEE Sensors J., vol. 13, no. 5, pp. 1419–1424, May 2013.
munication infrastructures: Motivations, requirements and challenges,”
[46] R. T. Fielding, “Architectural styles and the design of network-based
IEEE Commun. Surveys Tuts., vol. 15, no. 1, pp. 5–20, 1st Quart., 2013.
software architectures,” Ph.D. dissertation, Dept. Inf. Comput. Sci., Univ.
[24] M. R. Alam, M. B. I. Reaz, and M. A. M. Ali, “A review of smart California, Irvine, CA, USA, 2000.
homes—Past, present, and future,” IEEE Trans. Syst., Man, Cybern. C, [47] E. G. Davis, A. Calveras, and I. Demirkol, “Improving packet delivery
Appl. Rev., vol. 42, no. 6, pp. 1190–1203, Nov. 2012. performance of publish/subscribe protocols in wireless sensor networks,”
[25] N. Komninos, E. Philippou, and A. Pitsillides, “Survey in smart grid Sensors, vol. 13, no. 1, pp. 648–680, Jan. 2013.
and smart home security: Issues, challenges and countermeasures,” IEEE [48] T. Beale and S. Heard. (2008). OpenEHR Architecture Overview.
Commun. Surveys Tuts., vol. 16, no. 4, pp. 1933–1954, 4th Quart., 2014. [Online]. Available: http://www.openehr.org/releases/1.0.2/architecture/
[26] A. Caragliu, C. Del Bo, and P. Nijkamp, “Smart cities in Europe,” in overview.pdf
Proc. 3rd Central Eur. Conf. Regional Sci., 2009, pp. 49–59. [49] openEHR Specifications. Accessed on Oct. 15, 2016. [Online]. Available:
[27] M. S. Shahamabadi, B. B. M. Ali, P. Varahram, and A. J. Jara, http://www.openehr.org/ programs/specification/releases/currentbaseline
“A network mobility solution based on 6lowpan hospital wireless sen- [50] H. Leslie. (2012). Introduction to Archetypes and Archetype
sor network (NEMO-HWSN),” in Proc. 7th Int. Conf. Innov. Mobile Classes. [Online]. Available: https://openehr.atlassian.net/wiki/display/
Internet Services Ubiquitous Comput. (IMIS), Taichung, Taiwan, 2013, healthmod/Introduction+to+Archetypes+and+Archetype+classes
pp. 433–438. [51] openEHR. Clinical Knowledge Manager. Accessed on Oct. 15, 2016.
[28] F. Palumbo et al., “Sensor network infrastructure for a home care [Online]. Available: http://www.openehr.org/ckm/
monitoring system,” Sensors, vol. 14, no. 3, pp. 3833–3860, Feb. 2014. [52] E. Hovenga, Health Informatics: An Overview (Studies in Health
[29] A. J. Jara et al., “A pharmaceutical intelligent information system Technology and Informatics). Amsterdam, The Netherlands: IOS Press,
to detect allergies and adverse drugs reactions based on Internet of 2010.
Things,” in Proc. 8th IEEE Int. Conf. Pervasive Comput. Commun. [53] Zephyr HxM BT. Accessed on Oct. 15, 2016. [Online]. Available:
Workshops (PERCOM Workshops), Mannheim, Germany, 2010, http://zephyranywhere.com/ products/hxm-bluetooth-heart-rate-monitor/
pp. 809–812. [54] ETSI TR 102 966 V1.1.1 (2014-02) Machine-to-Machine
[30] M. Vazquez-Briseno, C. Navarro-Cota, J. I. Nieto-Hipolito, Communications (M2M); Interworking Between the M2M Architecture
E. Jimenez-Garcia, and J. D. Sanchez-Lopez, “A proposal for using and M2M Area Network Technologies, ETSI, Sophia Antipolis, France,
the Internet of Things concept to increase children’s health awareness,” 2014.
in Proc. 22nd Int. Conf. Elect. Commun. Comput. (CONIELECOMP), [55] G. Arena. Motorola Moto G (2nd Gen) Specifications.
2012, pp. 168–172. Accessed on Oct. 15, 2016. [Online]. Available:
[31] B. Xu et al., “Ubiquitous data accessing method in IoT-based informa- http://www.gsmarena.com/motorola_moto_g_ (2nd_gen)-6647.php
tion system for emergency medical services,” IEEE Trans. Ind. Informat., [56] Google. Android KitKat 4.4. Accessed on Oct. 15, 2016. [Online].
vol. 10, no. 2, pp. 1578–1586, May 2014. Available: https:// www.android.com/versions/kit-kat-4-4/
[32] R. Alberts, T. Fogwill, A. Botra, and M. Cretty, “An integrative ICT [57] Z. Shelby, K. Hartke, and C. Bormann, “The constrained application
platform for ehealth,” in Proc. IST Africa Conf., May 2014, pp. 1–8. protocol (CoAP),” IETF, Fremont, CA, USA, RFC 7252, 2014. [Online].
Available: https://tools.ietf.org/html/rfc7252
[33] C. Mazurek and M. Stroinski, “Innovative ICT platform for emerging
[58] G. Almes, S. Kalidindi, and M. Zekauskas, “A one-way delay metric for
ehealth services: Towards overcoming technical and social barriers and
IPPM,” IETF, Fremont, CA, USA, RFC 2679, 1999. [Online]. Available:
solving grand challenges in medicine,” in Proc. 2nd Int. Conf. eHealth
https://tools.ietf.org/html/rfc2679
Telemed. Soc. Med. (ETELEMED), 2010, pp. 33–38.
[59] D. Mills, “Network time protocol (version 3): Specification, implimenta-
[34] A. J. Jara, M. A. Zamora, and A. F. Skarmeta, “Knowledge acquisition
tion and anlysis,” IETF, Fremont, CA, USA, RFC 1305, 1992. [Online].
and management architecture for mobile and personal health environ-
Available: https://tools.ietf.org/html/rfc1305
ments based on the Internet of Things,” in Proc. IEEE 11th Int. Conf.
[60] V. Smotlacha, “One-way delay measurement using NTP,” in Proc.
Trust Security Privacy Comput. Commun. (TrustCom), Liverpool, U.K.,
TERENA Netw. Conf., 2003, pp. 1–2.
2012, pp. 1811–1818.
[61] D. L. Mills, “Internet time synchronization: The network time protocol,”
[35] S. K. Datta, C. Bonnet, and N. Nikaein, “An IoT gateway centric archi- IEEE Trans. Commun., vol. 39, no. 10, pp. 1482–1493, Oct. 1991.
tecture to provide novel M2M services,” in Proc. IEEE World Forum [62] L. De Vito, S. Rapuano, and L. Tomaciello, “One-way delay measure-
Internet Things (WF IoT), Seoul, South Korea, 2014, pp. 514–519. ment: State of the art,” IEEE Trans. Instrum. Meas., vol. 57, no. 12,
[36] M. J. Morón, R. Luque, and E. Casilari, “On the capability of smart- pp. 2742–2750, Dec. 2008.
phones to perform as communication gateways in medical wireless [63] G. Almes, S. Kalidindi, and M. Zekauskas, “A round-trip delay met-
personal area networks,” Sensors, vol. 14, no. 1, pp. 575–594, Jan. 2014. ric for IPPM,” IETF, Fremont, CA, USA, RFC 2681, 1999. [Online].
[37] L. Catarinucci et al., “An IoT-aware architecture for smart healthcare Available: https://tools.ietf.org/html/rfc2681
systems,” IEEE Internet Things J., vol. 2, no. 6, pp. 515–526, Dec. 2015. [64] A. Pathak, H. Pucha, Y. Zhang, Y. C. Hu, and Z. M. Mao, “A measure-
[38] G. Yang et al., “A health-IoT platform based on the integration of intel- ment study of Internet delay asymmetry,” in Passive and Active Network
ligent packaging, unobtrusive bio-sensor, and intelligent medicine box,” Measurement (LNCS 4979), M. Claypool and S. Uhlig, Eds. Heidelberg,
IEEE Trans. Ind. Informat., vol. 10, no. 4, pp. 2180–2191, Nov. 2014. Germany: Springer, 2008, pp. 182–191.
[39] M. N. Boulos, S. Wheeler, C. Tavares, and R. Jones, “How smart- [65] J. Postel, “Internet control message protocol,” IETF, Fremont, CA, USA,
phones are changing the face of mobile and participatory healthcare: RFC 792, 1981. [Online]. Available: https://tools.ietf.org/html/rfc792
An overview, with example from eCAALYX,” BioMed. Eng. OnLine, [66] S. McCanne, C. Leres, and V. Jacobson. TCPdump & Libpcap. Accessed
vol. 10, no. 1, p. 24, 2011. on Oct. 15, 2016. [Online]. Available: http://www.tcpdump.org
1094 IEEE INTERNET OF THINGS JOURNAL, VOL. 4, NO. 4, AUGUST 2017

[67] Java Platform, Standard Edition 7 API Specification—CurrentTimeMillis Carlos Pereira received the M.Sc. degree in electri-
Method, Oracle, Redwood City, CA, USA, 2016. [Online]. Available: cal and computer engineering from the University of
http : / / docs.oracle.com / javase / 7 / docs / api / java / lang / System.html # Porto, Porto, Portugal, in 2011, where he is currently
currentTimeMillis() pursuing the Ph.D. degree in electrical and computer
[68] A. N. Kolmogorov, “Sulla Determinazione Empirica di una Legge engineering.
di Distribuzione,” Giornale dell’Istituto Italiano degli Attuari, vol. 4, His current research interests include machine-
pp. 83–91, 1933. to-machine communications, heterogeneous wireless
[69] N. Smirnov, “Table for estimating the goodness of fit of empirical systems, and vehicular networks.
distributions,” Ann. Math. Stat., vol. 19, no. 2, pp. 279–281, Jun. 1948.
[70] H. W. Lilliefors, “On the Kolmogorov–Smirnov test for normality with
mean and variance unknown,” J. Amer. Stat. Assoc., vol. 62, no. 318,
pp. 399–402, Jun. 1967.
[71] K. Pearson, “Notes on regression and inheritance in the case of two
parents,” Proc. Roy. Soc. Lond., vol. 58, nos. 347–352, pp. 240–242,
Jan. 1895.
[72] F. Qian et al., “Characterizing radio resource allocation for 3G
networks,” in Proc. 10th ACM SIGCOMM Conf. Internet Meas. (IMC), António Pinto received the M.Sc. degree in network
Melbourne, VIC, Australia, 2010, pp. 137–150. and information systems from the University of
[73] J. Huang et al., “A close examination of performance and power char- Porto, Porto, Portugal, in 2016.
acteristics of 4G LTE networks,” in Proc. 10th Int. Conf. Mobile Syst. He is currently pursuing a career in industry.
Appl. Services (MobiSys), Ambleside, U.K., 2012, pp. 225–238. His current research interests include in machine-
[74] A. Singla, B. Chandrasekaran, P. B. Godfrey, and B. Maggs, to-machine communications and wireless systems.
“The Internet at the speed of light,” in Proc. 13th ACM Workshop Hot
Topics Netw. (HotNets XIII), Los Angeles, CA, USA, 2014, pp. 1–7.
[75] J. Huang et al., “An in-depth study of LTE: Effect of network protocol
and application behavior on performance,” ACM SIGCOMM Comput.
Commun. Rev., vol. 43, no. 4, pp. 363–374, Oct. 2013.
[76] I. Grigorik, High Performance Browser Networking: What Every Web
Developer Should Know About Networking and Web Performance.
Sebastopol, CA, USA: O’Reilly Media, 2013.
[77] Q. Xu et al., “Cellular data network infrastructure characterization and
implication on mobile content placement,” in Proc. ACM SIGMETRICS
Joint Int. Conf. Meas. Model. Comput. Syst. (SIGMETRICS), San Jose,
CA, USA, 2011, pp. 317–328.
[78] J. R. Gallego et al., “Performance analysis of multiplexed medical data Duarte Ferreira received the M.Sc. degree in infor-
transmission for mobile emergency care over the UMTS channel,” IEEE matics systems and network engineering from the
Trans. Inf. Technol. Biomed., vol. 9, no. 1, pp. 13–22, Mar. 2005. University of Porto, Porto, Portugal, in 2010, where
[79] L. Skorin-Kapov and M. Matijasevic, “Analysis of QoS requirements for he is currently pursuing the Ph.D. degree in indus-
e-health services and mapping to evolved packet system QoS classes,” trial engineering and management.
Int. J. Telemed. Appl., vol. 2010, pp. 1–18, Jan. 2010. He was a Software Developer in management and
[80] ITU-T Recommendation Y.1541, Network Performance Objectives for e-commerce area for six years. His current research
IP-Based Services, ITU-T, Geneva, Switzerland, 2002. interests include health informatics, data analysis,
[81] M. Shirvanimoghaddam, Y. Li, M. Dohler, B. Vucetic, and S. Feng, and decision support systems.
“Probabilistic rateless multiple access for machine-to-machine com-
munication,” IEEE Trans. Wireless Commun., vol. 14, no. 12,
pp. 6815–6826, Dec. 2015.
[82] “Service level awareness and open multi-service Internetworking—
Principles and potentials of an evolved Internet ecosystem,”
NetWorld2020 Expert Group, White Paper, 2016. [Online]. Available:
http://networld2020.eu/wp-content/uploads/2016/03/NetWorld2020_
WhPaper_Service-Level-Awareness_Final-Draft_0.9.5.pdf
[83] G. P. Fettweis, “The tactile Internet: Applications and challenges,” IEEE Ana Aguiar (S’02–M’09) received the Electrical
Veh. Technol. Mag., vol. 9, no. 1, pp. 64–70, Mar. 2014. and Computer Engineering degree from the
[84] M. Maier, M. Chowdhury, B. P. Rimal, and D. P. Van, “The tactile University of Porto, Porto, Portugal, in 1998, and
Internet: Vision, recent progress, and open challenges,” IEEE Commun. the Ph.D. degree in telecommunication networks
Mag., vol. 54, no. 5, pp. 138–145, May 2016. from the Technical University of Berlin, Berlin,
[85] M. Simsek, A. Aijaz, M. Dohler, J. Sachs, and G. Fettweis, “5G-enabled Germany, in 2008.
tactile Internet,” IEEE J. Sel. Areas Commun., vol. 34, no. 3, Since 2009, she has been an Assistant Professor
pp. 460–473, Mar. 2016. with the Faculty of Engineering, University of
[86] A. Aijaz, M. Dohler, A. H. Aghvami, V. Friderikos, and M. Frodigh, Porto. She contributes to several interdisciplinary
“Realizing the tactile Internet: Haptic communications over next gener- projects in the fields of smart cities, intelligent
ation 5G cellular networks,” IEEE Wireless Commun., to be published, transportation systems, and well being (stress).
doi: 10.1109/MWC.2016.1500157RP. She began her career as an RF Engineer for cellular operators and she
[87] C. Pereira et al., “IoT interoperability for actuating applications was with Fraunhofer Portugal AICOS, Porto, where she was involved with
through standardised M2M communications,” in Proc. IEEE 17th Int. service-oriented architectures and wireless technologies applied to ambient
Symp. World Wireless Mobile Multimedia Netw. (WoWMoM), Coimbra, assisted living. She has authored several papers published and presented in/at
Portugal, 2016, pp. 1–6. IEEE and ACM journals and conferences. Her current research interests
[88] M. M. Rahman et al., “Are we there yet? Feasibility of continuous stress include wireless networked systems, specifically mobile ad-hoc networks,
assessment via wireless physiological sensors,” in Proc. 5th ACM Conf. crowdsensing, and machine-to-machine communications and Internet of
Bioinformat. Comput. Biol. Health Informat. (BCB), Newport Beach, Things.
CA, USA, 2014, pp. 479–488. Dr. Aguiar is a Reviewer for IEEE and ACM conferences and journals.