Learning Automata Based Energy Efficient and Reliable Data Delivery Routing Mechanism in Wireless Sensor Networks
Learning Automata Based Energy Efficient and Reliable Data Delivery Routing Mechanism in Wireless Sensor Networks
a r t i c l e i n f o a b s t r a c t
Article history: Wireless Sensor Networks (WSNs) emerged as major data gathering paradigm due to its wide variety of
Received 18 October 2020 applications. Achieving energy efficient reliable data delivery is an important concern in WSNs. In a net-
Revised 15 April 2021 work, packet loss reduces the reliable data transmission rate. Retransmissions are required to increase
Accepted 16 April 2021
the reliable data transmission rate. However, number of retransmissions also increase the network
Available online 30 April 2021
energy consumptions and packet delivery delay. Packet loss rate can be reduced by selecting a best suc-
cessor node in a routing path. In this work, a Learning Automata based Routing mechanism is proposed
Keywords:
for wireless sensor networks to achieve energy efficiency and reliable data delivery. In this paper, learn-
A-star algorithm
Energy efficient routing path
ing automata has been adopted to calculate next node’s selection probability in a routing path using
Learning automata node’s score, quality of link and previous selection probability. Further, an energy efficient reliable rout-
Lifetime ing mechanism is proposed using combination of learning automata and A-star search algorithm. The
Wireless sensor networks proposed mechanism determines energy efficient and reliable routing paths in WSNs by favoring highest
residual energy, good quality of link , more free buffer space and less distance. Finally, simulation results
indicate that the proposed algorithm reduces the energy consumptions, data delivery delay, data trans-
missions and increases the network lifetime.
Ó 2021 The Authors. Published by Elsevier B.V. on behalf of King Saud University. This is an open access
article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
1. Introduction (Misra et al., 2014). Therefore, a sensor node may drop the packets
due to poor link quality, no free buffer (to accommodate a packet)
Wireless Sensor Networks (WSNs) emerged as major informa- and residual energy at node. Reliable data delivery and energy effi-
tion gathering and delivery paradigm due to its several applica- ciency are the major issues in the wireless sensor networks and
tions including health monitoring of nuclear plants and bridges, that are mainly depended on packet dropping rate. The sensor
floods alerts reporting, environment monitoring, fire detection in nodes can retransmit the dropped packets to achieve the reliable
forests and military surveillance (Akyildiz et al., 2002; Akyildiz data delivery (Mahmood et al., 2015; Lai et al., 2017). However,
and Kasimoglu, 2004; Rault et al., 2014). Usually, sensor nodes col- packet retransmission increases node’s energy consumption and
lect the data about physical world and relay to the sink node also packet delivery delay. Therefore, packet retransmission is
(where data processing will be done) using multi-hop communica- not an efficient approach in the time sensitive applications, such
tion. Sensor nodes are having limited resources, such as battery as reporting alerts related to earthquakes and tsunami, chemical
capacity, memory, computation and communication. Further, sen- explosions and forests fire (Akyildiz and Kasimoglu, 2004;
sor network will be deployed randomly in a dynamic environment Alshawi et al., 2012; Shen et al., 2014; Gungor and Hancke, 2009;
Elshrkawey et al., 2018). The process, in which selecting a node
⇑ Corresponding author. as a successor node in a routing path, takes a key part in the design
E-mail addresses: [email protected] (S. Gudla), [email protected] of energy efficient routing protocol. The packet dropping rate may
(N.R. Kuda). be reduced by selecting a successor node which has highest resid-
Peer review under responsibility of King Saud University. ual energy, free available buffer, good link quality and less dis-
tance. In this work, learning automata based routing mechanism
has been proposed. Learning automata is adopted to find the best
neighboring node by considering the parameters, such as energy,
Production and hosting by Elsevier quality of a link, buffer and distance. Further, A-Star heuristic
https://doi.org/10.1016/j.jksuci.2021.04.006
1319-1578/Ó 2021 The Authors. Published by Elsevier B.V. on behalf of King Saud University.
This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
S. Gudla and N.R. Kuda Journal of King Saud University – Computer and Information Sciences 34 (2022) 5759–5765
search algorithm has been applied to determine the energy effi- no free buffer to accommodate a packet and unavailability of a
cient routing paths in the WSNs. node due to the lack of residual energy. Reliable data delivery
The major objectives of the paper are as follows: can be achieved using retransmission of dropped packet. But,
retransmissions leads to more energy consumptions and also
Design of a learning automata based energy efficient and reli- increase the packet delivery delay (Liu et al., 2017; Wu et al.,
able data delivery routing algorithm to improve the energy effi- 2014). Therefore, retransmission mechanism is not suggestible
ciency and reliable data delivery for a WSN using A-star for the resource constrained networks such as wireless sensor net-
algorithm. works. Hence, there should be an efficient mechanism to select a
Design of a learning automata based mechanism to find a best node as successor node in a route so that packet dropping rate
node as successor in a routing path. can be reduced. In this work, Learning Automata is adopted to find
Simulation results are discussed to indicate the performance of the next hop node using resources of a node such as, residual
the proposed algorithm. energy, buffer, link quality, degree and distance. Further, A-star
search algorithm has been applied to find out the routing paths
2. Related work and motivation in a network.
In this section, literature on energy efficient routing algorithms, 3. Learning automata based energy efficient routing
energy estimation in wireless sensor networks are presented. In mechanism using A-star algorithm
Bhardwaj et al. (2001), upper bounds on lifetime of sensor net-
works have been derived. Lee and Lee (2010) have discussed the In this section, a network model has been introduced. A-star
upper bound on the network lifetime for a cluster-based wireless algorithm is explained for finding the energy efficient and reliable
sensor network. Alshawi et al. (2012) have presented a fuzzy based data delivery paths. Further, Learning automata is illustrated for
routing protocol for WSNs to enhance the network lifetime using estimating the node’s selection probability.
A-star algorithm. In Alshawi et al. (2012), remaining energy and
traffic load are considered as parameters for finding the node cost 3.1. Network model
in fuzzy approach. In Alshawi et al. (2012), the authors are not con-
sidered the link quality and buffer capacity while determining the A wireless sensor network has been represented as a graph
routing paths. Li et al. (2011) have proposed a routing algorithm GðV [ S; EÞ (refer Fig. 1), where V ¼ fv 1 ; v 2 ; v 3 . . . v n g is a set of n
using A-star search algorithm in a wireless sensor network. In Li sensor nodes, E ¼ fe1 ; e2 ; e3 . . . en g is a set of communication links
et al. (2011), greedy forwarding has been used to conserve the and S is a sink node where data processing can be done. In the
energy. In Li et al. (2011), routing paths are identified by consider- WSN (as shown in Fig. 1), every node calculates selection probabil-
ing the only distance parameter. Residual energy, buffer and link ities (pi ) for its neighbor nodes using learning automata model by
quality are the important parameters which are not been consid- taking the nodes’ score, link quality and distance. A node’s score is
ered in Li et al. (2011). Alkadhmawee and Lu (2016) have proposed estimated using link quality, buffer and residual energy (Yousefi
a routing protocol using fuzzy and A-star search algorithm to find et al., 2009). Further, A-star search algorithm is adopted to deter-
the optimal routes. In Alkadhmawee and Lu (2016), residual energy mines the energy efficient reliable routing paths using estimated
and traffic are considered as input parameters in the fuzzy logic selection probability value (pi ). In a data collection round, every
system. In Yao et al. (2010), Yao et al. have proposed an improved node in the network transmits one packet to the sink node through
A-star algorithm to find the effective and accurate feasible paths in multi-hop communication.
an unknown environment with obstacles. Risma Septiana and
Setiawan (2014) have proposed a routing protocol using A-star 3.2. Learning automata based energy efficient routing algorithm for
algorithm to obtain optimal routing paths in a WSN. In Ullah WSNs using A-star algorithm
et al. (2021), Ullah et al. have been proposed an energy-efficient
and reliable routing mechanism to enhance the stability period A-star search algorithm is one of the most popular graphic
in Wireless Body Area Networks. In our work, A -star algorithm searching algorithms. It is an efficient heuristic algorithm for find-
and learning automata has been adopted to identifying the optimal ing low cost paths. A-star algorithm is a best intelligent search
routing paths by considering parameters such as node’s energy,
link quality, distance and available buffer space. Thathachar and
Sastry (2002) have presented a variety of learning automata mod-
els for different applications. In Misra et al. (2017), a fault tolerant
routing mechanism is presented for autonomous unmanned vehi-
cle based networks using learning automata. Pantazis et al. (2013)
illustrated different energy efficient routing protocols in wireless
sensor networks. In Rault et al. (2014), Rault et al. have presented
various energy conservation schemes. Jakllari and Eidenbenz
(2012) have computed a routing path cost based on number of link
layer transmissions and retransmissions in sensor networks. How-
ever, Learning Automata has been used to estimate the node’s
selection probability by considering node’s residual energy, buffer
and link quality and previous selection probability.
2.1. Motivation
5760
S. Gudla and N.R. Kuda Journal of King Saud University – Computer and Information Sciences 34 (2022) 5759–5765
algorithm which has the merits of breadth-first algorithm and priority queue which find out the next node with largest evaluation
depth first search algorithm. A-star algorithm uses local informa- function value (f ðnÞ). The CLOSE list has a list of nodes which are
tion and some heuristic information to choose the shortest path. already visited. Initially, the OPEN list contains the starting node.
A-star algorithm searches all routes leading from the starting node In the first iteration, top (i.e. a node with largest evaluation function
until it finds the minimal cost path to the goal (Alshawi et al., 2012; value (f ðnÞ)) of the priority list will be taken and checks whether it
Li et al., 2011; Passino and Antsaklis, 1994). It finds the paths using is destination node (sink node). If so, the algorithm is completed.
an evaluation function which determines the order of visiting Otherwise, it computes the evaluation function of all neighboring
nodes in the network. The evaluation function is provided as: nodes and add them to the OPEN list. After completion of A-star
algorithm, it will find the path if a path exists. If it does not find a
f ðnÞ ¼ g ðnÞ þ hðnÞ ð1Þ
path, then algorithm guarantees that no such path exists (Alshawi
where g ðnÞ is the actual cost from the source node (i.e. a sensor et al., 2012; Li et al., 2011). The sensor network GðV [ S; EÞ, OPEN_-
node) to node n (an intermediate node), hðnÞ is an estimated cost List and CLOSE_List are given as input to the proposed mechanism
of the optimal path from node n to the destination node (i.e. sink which has been presented as Algorithm 1. Every node in the net-
node). In this work, hðnÞ is taken as lower bound of the actual dis- work executes the algorithm for reliable and energy efficient paths.
tance (shortest physical distance) from node n to the sink node (Li Assume Node-i executing the proposed mechanism. OPEN_List has
et al., 2011; Passino and Antsaklis, 1994). been initialized with Node-i and its f value(estimated using Eq.
(2)) . If the OPEN_List is NULL, then path searching is failed and
Algorithm 1. Learning Automata based Energy Efficient algorithm exits (line 1–3). POP() operation always returns a node
Routing Using A-Star Algorithm in WSNs with highest f value which is Node-j and it is pushed in to CLOSE_-
List (line 4–5). If Node-j is destination node (i.e. sink node), then
Input:GðV [ S; EÞ, OPEN_List= {Node-i}, CLOSE_List= {NULL}
path successfully finished (line 6–8). Otherwise, search will be con-
Output: Searches an energy efficient path
tinue by finding its neighbors (adjacent nodes). Here, Find Neigh-
1: If OPEN_List==NULL
bors() function returns the neighbors. If there are no neighbors
2: Path searching is failed and Exit
then goto step 1 (line 9–10). For each neighbor Node-k, compute
3: end if
the f value using Eq. (2). If the Node-k not in the both list (OPEN_List
4: Node-j = POP(OPEN_List) //Returns a node which has
and CLOSE_List), then push it into the OPEN_List. If the Node-k is
largest f value
either in OPEN_List or CLOSE_List, then f value of Node-j is adjusted
5: CLOSE_List = CLOSE_List [ Node-j
to larger value (line 12-19). OPEN_List is a priority queue data struc-
6: if Node-j==Sink_Node
ture and it always sort the nodes from large to small according to
7: Path search is successfully finished and reconstruct the
nodes’ f value. The algorithm runs till either OPEN_List becomes
path, Exit
null or destination node encountered. The flow diagram of the pro-
8: end if
posed mechanism has been presented in Fig. 2.
9: if Find_Neighbors(Node-j)== NULL
10: Goto Step-1
11: else 3.4. Learning automata approach
12: for k ¼ 1 to d // d is the number of neighbors of Node-j
13: Compute f ðkÞ for each node-k using Eq. (2) Assume that, 5-tuples h A; a; P; b; Li is representing a proposed
14: if (k R (OPEN_List ^ CLOSE_List)) learning automata model. A set of automaton can be represented
15: OPEN_List = OPEN_List [ Node-k as A ¼ fa1 ; a2 ; . . . ; ad g and each automata is placed at every node
16: else (where d is degree of a node). A series of actions can be represented
17: Choose the shortest path based on f value of as a ¼ fa1 ; a2 ; . . . ; ad g, where a indicates a set of d neighbor nodes.
Node-j Actions are performed on a random environment. The behavior of a
18: end if neighbor node is depends on residual energy, buffer, degree and
19: end for distance. The automata performs the repeated actions and chooses
20: Sort the nodes in the OPEN_List from large to small a specific action from the action set. Then, the environment pro-
according f value. duces a response (in terms of either reward or penalty) to specific
21: end if action (Mothku and Rout, 2019). The selection probability of neigh-
22: Goto Step-1 bor node is denoted by P and is computed using Eq. (4). Further, a
set of responses is represented by b after performing a finite set of
actions and L indicates the learning mechanism (which updates the
probability value of neighbor node). Here, reward is selection of
best neighboring node which has more residual energy, degree,
3.3. Implementation of A-star algorithm
buffer and less distance.
Each node calculates selection probability for its neighbors using
Every node in the network executes A-star algorithm to find out
learning automata. Score (Sj ) of a node-j indicates the node’s
the routes to the sink node. In the algorithm, a node is explored
resourcefulness i.e. available resources with node-j, such as residual
based on the evaluation function which is given as:
energy (Ej ), buffer (Bj ), degree (Dj ) and distance ( D b j ) (Yousefi et al.,
f ðnÞ ¼ PðnÞ þ ð1=SDðnÞÞ ð2Þ 2009; Mothku and Rout, 2019). A node’s score is estimated by
where PðnÞ selection probability of node n and it is calculated using bj
S ¼ W energy Ej þ W degree Dj þ W buffer Bj þ W distance D ð3Þ
Eq. (4) and it takes value ½0 . . . 1. Learning automata approach has
been adopted to compute optimal cost for node n (Selection proba-
where W energy ; W degree ; W buffer and W distance are the weights of residual
bility of a node n) by considering the node’s score, link quality and
energy, degree, available buffer and distance, respectively. Further,
distance. SDðnÞ is shortest distance from node n to the sink node.
probability of selection of node-j P ij is calculated using score Sj
Further, a node will be chosen as optimal node that has largest
f ðnÞ. A-star search algorithm has two lists: an OPEN list and a CLOSE (Eq. (3)), quality of link Lij and previous selection probability value
list. The OPEN list keeps track of the nodes in it and it works as a Pold
ij ) and is given by
5761
S. Gudla and N.R. Kuda Journal of King Saud University – Computer and Information Sciences 34 (2022) 5759–5765
5762
S. Gudla and N.R. Kuda Journal of King Saud University – Computer and Information Sciences 34 (2022) 5759–5765
Table 1
Simulation parameters.
Parameter Value
Sensor nodes 50 to 500 nodes
Transmission range 30 m
Initial energy of a node 20– 25 K
Es ¼ a 3 a3 ¼ 50 109 Joules=bit
Er ¼ a12 a12 ¼ 0:787 106 Joules=bit
Et ¼ a11 þ a2 d
n
a11 ¼ 0:937 106 Joules=bit
a2 ¼ 10 1012 Joules=bit=m2
d ¼ 85 m
Fig. 4. Network energy consumption (in one data collection round).
5763
S. Gudla and N.R. Kuda Journal of King Saud University – Computer and Information Sciences 34 (2022) 5759–5765
5. Conclusion
Energy efficiency and reliable data delivery are the main chal-
lenges in WSNs. In this paper, learning automata based energy effi-
cient and reliable data delivery routing mechanism is designed to
improve the reliable data delivery and energy efficiency for WSNs
using A-star search algorithm. Learning automata is adopted to
choose the best node in a routing path by considering the node’s
residual energy, quality of link and available free buffer as main
parameters which cause the dropping packets. In this work, A-
star search algorithm is adopted to identify the routing paths.
The proposed approach reduces the packet dropping rate and
achieved less average delivery delay of a packet and energy con-
sumption. Simulation results are shown that the proposed mecha-
nism reduces the network energy consumption upto 45%. Further,
lifetime has been improved significantly with the proposed mech-
anism. As a future research work, it may be extended by consider-
Fig. 6. The lifetime (first node died data collection round) of the Network. ing the node mobility and energy harvesting nodes in the network.
5764
S. Gudla and N.R. Kuda Journal of King Saud University – Computer and Information Sciences 34 (2022) 5759–5765
References Misra, S., Kar, P., Roy, A., Obaidat, M.S., 2014. Existence of dumb nodes in stationary
wireless sensor networks. J. Syst. Softw. 91, 135–146.
Misra, S., Krishna, P., Saritha, V., Agarwal, H., Vasilakos, A.V., Obaidat, M.S., 2017.
Akyildiz, I.F., Kasimoglu, I.H., 2004. Wireless sensor and actor networks: research
Learning automata-based fault-tolerant system for dynamic autonomous
challenges. Ad Hoc Netw. 2 (4), 351–367.
unmanned vehicular networks. IEEE Syst. J. 11 (4), 2929–2938.
Akyildiz, I.F., Su, W., Sankarasubramaniam, Y., Cayirci, E., 2002. Wireless sensor
Mothku, S.K., Rout, R.R., 2019. Markov decision process and network coding for
networks: a survey. Comput. Netw. 38 (2), 393–422.
reliable data transmission in wireless sensor and actor networks. Pervasive
Alkadhmawee, A.A., Lu, S. 2016. Prolonging the Network Lifetime Based on LPA-Star
Mob. Comput. 56, 29–44.
Algorithm and Fuzzy Logic in Wireless Sensor Network, in Proc. 12th World
Network Simulator 3. url: https://www.nsnam.org, Accessed: 2020-07-15.
Congress on Intelligent Control and Automation (WCICA).
Pantazis, N.A., Nikolidakis, S.A., Vergados, D.D., 2013. Energy-efficient routing
Alshawi, I.S., Yan, L., Pan, W., Luo, B., 2012. Lifetime enhancement in wireless sensor
protocols in wireless sensor networks: a survey. IEEE Commun. Surv. Tutorials
networks using fuzzy approach and a-star algorithm. IEEE Sens. J. 12 (10),
15 (2), 551–591.
3010–3018.
Passino, K., Antsaklis, P., 1994. A metric space approach to the specification of the
Bhardwaj, M., Garnett, T., Chandrakasan, A.P., 2001. Upper bounds on the lifetime of
heuristic function for the A* algorithm. IEEE Trans. Syst. Man Cybern. 24 (1),
wireless sensor networks. IEEE Int. Conf. Commun. (ICC), 785–790.
159–166.
Elshrkawey, M., Elsherif, S.M., Wahed, M.E., 2018. An enhancement approach for
Rault, T., Bouabdallah, A., Challal, Y., 2014. Energy efficiency in wireless sensor
reducing the energy consumption in wireless sensor networks. J. King Saud
networks: A top-down survey. Comput. Netw. 67, 104–122.
Univ. Comput. Inf. Sci. 30 (2), 259–269.
Risma Septiana, I.S., and Setiawan, N.A. 2014 Evaluation Function Effectiveness in
Gungor, V.C., Hancke, G.P., 2009. Industrial wireless sensor networks : challenges,
Wireless Sensor Network Routing using A-star Algorithm, in Proc. 4th
design principles, and technical approaches. IEEE Trans. Industr. Electron. 56
International Conference on Cyber and IT Service Management.
(10), 4258–4265.
Shen, W., Zhang, T., Barac, F., Gidlund, M., 2014. PriorityMAC: A priority-enhanced
Imon, S.K.A., Khan, A., Francesco, M.D., Das, S.K., 2015. Energy-efficient randomized
MAC protocol for critical traffic in industrial wireless sensor and actuator
switching for maximizing lifetime in tree-based wireless sensor networks. IEEE/
networks. IEEE Trans. Industr. Inf. 10 (1), 824–835.
ACM Trans. Netw. 23 (5), 1401–1415.
Thathachar, M.A.L., Sastry, P.S., 2002. Varieties of learning automata: an overview.
Jakllari, G., Eidenbenz, S., 2012. Link positions matter: A noncommutative routing
IEEE Trans. Syst. Man Cybern. 32 (6), 711–722.
metric for wireless mesh networks. IEEE Trans. Mob. Comput. 11 (1), 61–72.
Ullah, F., Zahid Khan, M., Faisal, M., Rehman, H.U., Abbas, S., Mubarek, F.S., 2021. An
Lai, X., Ji, X., Zhou, X., Chen, L., 2017. Energy efficient link-delay aware routing in
energy efficient and reliable routing scheme to enhance the stability period in
wireless sensor networks. IEEE Sens. J. 18 (2), 837–848.
wireless body area networks. Comput. Commun. 165, 20–32. https://doi.org/
Lee, S., Lee, H.S., 2010. Analysis of network lifetime in clustere-based sensor
10.1016/j.comcom.2020.10.017. url:https://www.sciencedirect.com/science/
networks. IEEE Commun. Lett. 14 (10), 900–902.
article/pii/S0140366420319654.
Li, X., Hong, S., Fang, K., 2011. WSNHA-GAHR: a greedy and A* heuristic routing
Wu, C., Ji, Y., Xu, J., Ohzahata, S., Kato, T., 2014. Coded packets over lossy links: A
algorithm for wireless sensor networks in home automation. IET Commun. 5
redundancy-based mechanism for reliable and fast data collection in sensor
(13), 1797–1805.
networks. Comput. Netw. 70, 179–191.
Liu, Y., Liu, A., Li, Y., Li, Z., Choi, Y., Sekiya, H., Li, J. 2017 APMD: A fast data
Yao, J., Lin, C., Xie, X., Wang, A.J., Hung, C.C. 2010 Path Planning for Virtual Human
transmission protocol with reliability guarantee for pervasive sensing data
Motion Using Improved A* Star Algorithm, in Proc. IEEE Seventh International
communication, Pervasive Mob. Comput. 41, 413–435.
Conference on Information Technology, pp. 1154–1158.
Mahmood, M.A., Seah, W.K., Welch, I., 2015. Reliability in wireless sensor networks:
Yousefi, H., Dabirmoghaddam, A., Mizanian, K., 2009. Score based reliable routing in
A survey and challenges ahead. Comput. Netw. 79, 166–187.
wireless sensor networks. IEEE Conf. Inf. Netw., 1–5
5765