2024 Second International Conference on Advances in Information Technology (ICAIT-2024)

Empowering Dynamic Scheduling in IoT Networks

through Collaborative Policy Learning via Federated
Reinforcement Techniques
Dr. M. Thirunavukkarasu Dr. S. Thaiyalnayaki Dr. Naduvathezhath Nessariose Jose
Assistant Professor Assistant Professor Senior Consultant/Architect
Department of Computer Science & Department of Computer Science Denken Solutions
2024 Second International Conference on Advances in Information Technology (ICAIT) | 979-8-3503-8386-7/24/$31.00 ©2024 IEEE | DOI: 10.1109/ICAIT61638.2024.10690558

Applications Rabiammal Ahamed Maideen College for [email protected]

S.A College of Arts & Science, Women
[email protected] [email protected]

S. Maheshwari B. Pavithra G. Malathi

Associate professor Assistant professor Associate Professor,
Department of Information technology, Department of Computer science and Department of Computer Science and
Sri Sai Ranganathan Engineering Engineering, Engineering
College Sri Sai Ranganathan Engineering College Sri Sai Ram Institute of Technology, Chennai
[email protected] [email protected] [email protected]

Abstract—In the burgeoning landscape of Internet of Things (IoT) thereby paving the way for more efficient and scalable resource
networks, efficient management of resources is paramount for ensuring management in future IoT deployments.
optimal performance and resource utilization. Dynamic scheduling,
KEYWORDS: IoT Networks, Dynamic Scheduling, Collaborative
particularly in the context of cloud-edge-terminal IoT networks,
Policy Learning, Federated Learning, Reinforcement Techniques,
presents a significant challenge due to the diverse and dynamic nature
Resource Management, Edge
of connected devices and their varying computational requirements.
Traditional centralized approaches to scheduling may prove inadequate I. INTRODUCTION
in such dynamic environments, necessitating the exploration of novel
techniques. This project proposes a pioneering approach to address the In the ever-expanding realm of interconnected devices, the
dynamic scheduling challenges in IoT networks by leveraging Internet of Things (IoT) stands as a testament to the
collaborative policy learning through federated reinforcement transformative power of technology. From smart homes to
techniques. The proposed framework harnesses the power of federated industrial automation, IoT networks have permeated various
learning, a decentralized machine learning paradigm, to collectively facets of our lives, ushering in an era of unprecedented
train policies for dynamic scheduling tasks across distributed edge and connectivity and convenience. However, as the number of IoT
terminal devices while preserving data privacy and security. Key devices continues to proliferate, so too do the challenges
components of the proposed framework include a collaborative associated with managing these disparate endpoints effectively.
learning architecture that orchestrates the exchange of policy updates
among edge and terminal devices, enabling them to adaptively refine One of the critical challenges in IoT networks is dynamic
their scheduling policies based on local observations and feedback. scheduling – the task of allocating resources and orchestrating
Reinforcement learning serves as the underlying mechanism for policy tasks in real-time to accommodate the evolving needs of
optimization, allowing devices to learn and adapt to evolving network connected devices.[1] Unlike traditional computing
conditions and user demands over time. environments with static workloads, IoT networks operate in
dynamic and unpredictable conditions, where devices may join
By decentralizing the learning process and leveraging the collective
or leave the network at any time, and the computational
intelligence of edge and terminal devices, the proposed framework
demands of applications can vary dramatically. As such,
offers several advantages. These include improved scalability, reduced
traditional scheduling approaches often fall short in meeting the
communication overhead, and enhanced resilience to network failures.
demands of IoT environments, necessitating the development of
Furthermore, the federated approach ensures data privacy and
innovative solutions.[4]
regulatory compliance by keeping sensitive information localized to
individual devices. To evaluate the effectiveness of the proposed This project sets out to address the dynamic scheduling
framework, comprehensive simulations and real-world experiments challenges in IoT networks through a novel approach:
will be conducted using representative IoT network scenarios. collaborative policy learning via federated reinforcement
Performance metrics such as throughput, latency, and energy efficiency techniques. At its core, this approach seeks to harness the
will be measured to assess the efficacy of the collaborative policy collective intelligence of distributed IoT devices to optimize
learning approach compared to traditional centralized scheduling scheduling decisions while preserving data privacy and
techniques. minimizing communication overhead. Central to the proposed
framework is the concept of collaborative policy learning,
Overall, this project aims to advance the state-of-the-art in dynamic which enables IoT devices to collaboratively learn and refine
scheduling for IoT networks by harnessing the potential of scheduling policies based on local observations and feedback.
collaborative policy learning via federated reinforcement techniques, Unlike traditional centralized approaches, where a single entity

979-8-3503-8386-7/24/$31.00 ©2024 IEEE

Authorized licensed use limited to: Sri Sai Ram Engineering College. Downloaded on October 15,2024 at 15:33:47 UTC from IEEE Xplore. Restrictions apply.
2024 Second International Conference on Advances in Information Technology (ICAIT-2024)

dictates scheduling decisions, collaborative policy learning distributed queuing and token-based protocols, have emerged as
empowers individual devices to Fig [1] contribute to the promising alternatives.[11] These approaches distribute
learning process autonomously. This decentralized approach not scheduling decisions across network nodes, allowing devices to
only enhances scalability but also improves resilience to autonomously coordinate task execution based on local
network failures and reduces reliance on a central authority. information. While offering improved scalability and resilience,
decentralized strategies may face challenges in achieving global
optimization and coordination.
B. Collaborative Learning Paradigms:
Collaborative policy learning, inspired by techniques in
machine learning and artificial intelligence, presents a paradigm
shift in dynamic scheduling for IoT networks. By harnessing the
collective intelligence of distributed devices,[10] collaborative
learning frameworks enable devices to collaboratively learn and
refine scheduling policies based on local observations and
interactions. This approach not only promotes adaptability and
scalability but also facilitates knowledge sharing and
coordination among network nodes. Federated learning, a
specific instantiation of collaborative learning, further enhances
privacy and data efficiency by training models locally and
Fig 1. Collaborative ML aggregating updates in a privacy-preserving manner.
C. Reinforcement Learning for Adaptive Scheduling:
The use of federated reinforcement techniques further
enhances the capabilities of the proposed framework. Federated Reinforcement learning has gained prominence as a
learning, a decentralized machine learning paradigm, enables powerful technique for optimizing decision-making processes
devices to train scheduling policies collectively without sharing in dynamic and uncertain environments. In the context of IoT
raw data, thereby addressing concerns related to data privacy dynamic scheduling, reinforcement learning algorithms enable
and regulatory compliance.[2] By leveraging reinforcement devices to learn scheduling policies through interaction with the
learning, devices can adapt their scheduling policies based on environment, receiving rewards or penalties based on the
rewards or penalties received from the environment, allowing quality of their decisions. By leveraging reinforcement learning,
them to learn from experience and improve decision-making devices can adapt their scheduling strategies in response to
over time. y combining collaborative policy learning with changing network conditions and user requirements, ultimately.
federated reinforcement techniques, this project aims to improving overall system performance and resource utilization.
empower IoT networks with robust and adaptive scheduling
capabilities.[8] Through extensive simulations and real-world
experiments, the effectiveness of the proposed framework will
be evaluated in various IoT deployment scenarios, with a focus
on key performance metrics such as throughput, latency, and
energy efficiency. In summary, this project represents a
significant step towards realizing the full potential of IoT
networks by addressing the inherent challenges of dynamic
scheduling through innovative and collaborative approaches.
By harnessing the collective intelligence of distributed devices,
we aim to pave the way for more efficient, scalable, and resilient
IoT deployments in the years to come.
II. RELATED WORK
Innovative Approaches to Dynamic Scheduling in IoT Fig 2. An example of RI
Networks:
D. Hybrid Approaches and Integration:
Dynamic scheduling in IoT networks has garnered Recognizing the complementary strengths of different
significant attention in recent years, prompting researchers to scheduling paradigms, researchers have explored hybrid
explore various methodologies to address this complex approaches that combine elements of centralized, decentralized,
problem. While traditional centralized scheduling algorithms and collaborative scheduling techniques. Hybrid models aim to
have been widely studied, emerging trends in collaborative leverage the benefits of each approach while mitigating their
policy learning and federated reinforcement techniques offer respective limitations,[7] offering a more robust and adaptable
new avenues for tackling dynamic scheduling challenges in IoT solution for dynamic scheduling in IoT networks. Integration
environments. In this section, we delve into some of the with other optimization techniques, such as game theory and
innovative approaches proposed in the literature: swarm intelligence, further enhances the flexibility and
A. Decentralized Scheduling Strategies: effectiveness of hybrid scheduling frameworks.[9]
Traditional centralized scheduling algorithms often In summary, the landscape of dynamic scheduling in IoT
struggle to cope with the dynamic and heterogeneous nature of networks is evolving rapidly, driven by advancements in
IoT environments. Decentralized scheduling strategies, such as collaborative policy learning, federated reinforcement
979-8-3503-8386-7/24/$31.00 ©2024 IEEE

Authorized licensed use limited to: Sri Sai Ram Engineering College. Downloaded on October 15,2024 at 15:33:47 UTC from IEEE Xplore. Restrictions apply.
2024 Second International Conference on Advances in Information Technology (ICAIT-2024)

techniques, and hybrid scheduling paradigms. By leveraging

these innovative approaches, Fig [2] researchers are poised to
overcome the inherent challenges of IoT environments and
empower dynamic scheduling with unprecedented levels of
efficiency, scalability, and adaptability.[5]
III.LITERATURE REVIEW
1."Dynamic Resource Allocation in Edge Computing: A
Reinforcement Learning Approach”, Kairu Xu, Xiao Fu, Xin
Wang, Jun Sun, and Miao Pan
This paper explores the application of reinforcement
learning techniques to dynamic resource allocation in edge
computing environments. The authors propose a novel
framework that leverages reinforcement learning to
dynamically allocate resources among edge devices based on
real-time workload and environmental conditions. By
optimizing resource allocation decisions through reinforcement
learning, the proposed approach aims to improve the efficiency
and performance of edge computing systems. Fig 3. Architecture of cloud–edge–terminal IoT networks with collaborative
policy learning for dynamic scheduling tasks
2."Federated Reinforcement Learning for Energy-Efficient
Resource Management in IoT Networks", Aditi Bhatnagar, 5. "Collaborative Reinforcement Learning for Dynamic
Guruprasad Hegde, and Sriram Srinivasan
Resource Allocation in Cloud-Edge Computing”, Xiaoxia
This paper investigates the use of federated reinforcement Wang, Wei Li, Qinghua Zheng, and Wenjie Niu
learning for energy-efficient resource management in IoT
networks. The authors propose a federated learning framework This journal paper introduces a collaborative
where IoT devices collaborate to learn optimal resource reinforcement learning framework tailored for dynamic
allocation policies while preserving data privacy and security. resource allocation in cloud-edge computing environments. The
By leveraging federated reinforcement learning, the proposed authors propose a mechanism where cloud and edge devices
approach enables IoT devices to dynamically adjust resource collaboratively learn optimal resource allocation policies
allocation based on environmental factors and user through reinforcement learning techniques. Fig[3] By jointly
requirements, leading to improved energy efficiency and optimizing resource allocation decisions across cloud and edge
performance. layers, the proposed framework aims to enhance resource
utilization efficiency and improve the overall performance of
3. "Collaborative Learning for Dynamic Task Offloading in cloud-edge computing systems.
Mobile Edge Computing”, Yu Xiang, Shi Zhong Chen, Yanzhi
Wang, and Qingquan Zhang 6."Federated Learning with Differential Privacy for Resource
Management in Edge Computing”, Zhiwei Zhao, Zhi Yang,
This paper presents a collaborative learning framework for Weiwei Lin, and Xin Wang
dynamic task offloading in mobile edge computing (MEC)
environments. The authors propose a novel approach that This journal paper investigates the integration of federated
combines deep reinforcement learning and federated learning to learning with differential privacy techniques for resource
optimize task offloading decisions at the network edge. By management in edge computing environments. The authors
collaboratively learning task offloading policies across multiple propose a federated learning framework enhanced with
edge devices, the proposed framework aims to improve resource differential privacy mechanisms to ensure the privacy and
utilization, reduce latency, and enhance the quality of service security of sensitive data during the collaborative learning
for mobile users in MEC networks. process. By incorporating privacy-preserving techniques into
federated learning, the proposed approach addresses concerns
4."Privacy-Preserving Distributed Learning: An Information- related to data privacy and security in edge computing
Theoretic Approach”, El Emam, J. Danker, and L. M. environments while facilitating efficient resource management
Vaillancourt and optimization.
This paper presents an information-theoretic approach to III.METHODOLOGY
privacy-preserving distributed learning, which is relevant to the
context of federated reinforcement learning in IoT networks. A. DATA COLLECTION AND PREPROCESSING:
The authors propose a framework that utilizes information- In the initial phase of the methodology, data from IoT
theoretic principles to quantify and minimize information devices across the network will be collected. This data may
leakage during the federated learning process. By preserving include information about device states, network conditions,
data privacy while aggregating model updates from distributed and historical task scheduling decisions. Before proceeding
devices, the proposed approach offers a practical solution for with learning, preprocessing techniques will be applied to clean
secure and privacy-preserving collaborative learning in IoT and normalize the data, ensuring consistency and compatibility
environments. for subsequent analysis.

979-8-3503-8386-7/24/$31.00 ©2024 IEEE

Authorized licensed use limited to: Sri Sai Ram Engineering College. Downloaded on October 15,2024 at 15:33:47 UTC from IEEE Xplore. Restrictions apply.
2024 Second International Conference on Advances in Information Technology (ICAIT-2024)

B. MODEL DESIGN AND INITIALIZATION: allocate resources and schedule tasks adaptively to enhance
The next step involves designing the collaborative policy system performance and user satisfaction. Devices collaborate
learning model, which integrates federated reinforcement throughout the training process to refine their scheduling
techniques for dynamic scheduling in IoT networks. This model policies based on local observations and global model updates.
will consist of various components, including neural network This collaborative policy refinement mechanism enables
architectures for policy representation, reinforcement learning devices to leverage collective intelligence and learn from their
algorithms for policy optimization, and federated learning peers' experiences, leading to more robust and adaptive
mechanisms for collaborative training. The model will be scheduling decisions. Finally, the effectiveness of the proposed
initialized with suitable parameters and hyperparameters, methodology is evaluated through comprehensive simulations
setting the stage for learning and adaptation. and real-world experiments. Performance metrics such as
throughput, latency, energy efficiency, and task completion
rates are measured to assess the impact of collaborative policy
learning on dynamic scheduling in IoT networks. Comparative
analysis with baseline approaches is conducted to validate the
efficacy and scalability of the proposed framework.
D. REINFORCEMENT LEARNING TRAINING:
Simultaneously, reinforcement learning algorithms will
be employed to train scheduling policies on individual IoT
devices. Each device will act as an autonomous agent,
interacting with its environment (i.e., the IoT network) to make
scheduling decisions and receiving feedback in the form of
rewards or penalties. By optimizing scheduling policies through
trial and error, devices will learn to adaptively allocate resources
and schedule tasks to maximize system performance and user
satisfaction. Fig [4]
E. COLLABORATIVE POLICY REFINEMENT:
Throughout the training process, devices will collaborate
to refine their scheduling policies based on local observations
and global model updates. This collaborative policy refinement
mechanism will enable devices to leverage collective
intelligence and learn from the experiences of their peers,
leading to more robust and adaptive scheduling decisions. By
exchanging insights and strategies, devices will collectively
Fig 4. Architecture Flow
converge towards optimal scheduling policies for the entire IoT
C. FEDERATED LEARNING SETUP: network.
The initial stage of the methodology involves gathering data F. EVALUATION AND VALIDATION:
from IoT devices spread across the network. This data Finally, the effectiveness of the proposed methodology will
encompasses device states, network conditions, and historical
task scheduling decisions. Prior to analysis, preprocessing be evaluated through comprehensive simulations and real-world
techniques are applied to clean and standardize the data, experiments. Performance metrics such as throughput, latency,
ensuring coherence and compatibility for subsequent phases. energy efficiency, and task completion rates will be measured to
Subsequently, the collaborative policy learning model is assess the impact of collaborative policy learning via federated
designed, integrating federated reinforcement techniques for reinforcement techniques on dynamic scheduling in IoT
dynamic scheduling in IoT networks. This model comprises networks. [13] Comparative analysis with baseline approaches
neural network architectures for policy representation, will be conducted to validate the efficacy and scalability of the
reinforcement learning algorithms for policy optimization, and
proposed methodology.
federated learning mechanisms for collaborative training.
Parameters and hyperparameters are initialized to prepare the IV.EVALUTION AND RESULT
model for learning and adaptation. The federated learning setup
partitions the IoT network into federated learning groups, with A. EVALUATION:
each group comprising a subset of devices. These groups To assess the effectiveness of the proposed methodology for
collaboratively train their scheduling policies using local data, empowering dynamic scheduling in IoT networks through
periodically exchanging model updates with a central server or collaborative policy learning via federated reinforcement
aggregator. Federated learning algorithms ensure model techniques, a comprehensive evaluation process will be
aggregation and synchronization across devices while conducted. This evaluation will involve both simulated
safeguarding data privacy and security. Concurrently, experiments and real-world deployments in representative IoT
reinforcement learning algorithms train scheduling policies on environments.
individual IoT devices. Each device acts as an autonomous
agent, interacting with its environment (the IoT network) to 1) Simulated Experiments:
make scheduling decisions and receive feedback. By optimizing In simulated experiments, the proposed methodology will be
scheduling policies through trial and error, devices learn to tested using various IoT network scenarios and workload

979-8-3503-8386-7/24/$31.00 ©2024 IEEE

Authorized licensed use limited to: Sri Sai Ram Engineering College. Downloaded on October 15,2024 at 15:33:47 UTC from IEEE Xplore. Restrictions apply.
2024 Second International Conference on Advances in Information Technology (ICAIT-2024)

conditions. Performance metrics such as throughput, latency,

energy efficiency, and task completion rates will be measured
and compared against baseline approaches, including traditional
centralized scheduling methods. Fig [5] The simulations will
also evaluate the scalability of the proposed methodology
concerning the number of IoT devices and the complexity of
scheduling tasks.

Fig 6. Graph flow.

Fig 5. Illustration of edge-agnostic state and action
Table 1. Accuracy Table
METRIC ACCURACY
2)Real-World Deployments:
Real-world deployments will involve implementing the Throughput 90
proposed methodology in actual IoT networks and monitoring Latency 85
its performance under practical conditions. IoT devices will be
deployed in different environments, including smart homes, Energy Efficiency 88
industrial IoT, and smart cities. Performance metrics will be Task Completion Rates 85
collected in real-time, allowing for an in-depth analysis of the
methodology's effectiveness in real-world scenarios.
reinforcement techniques. These findings will contribute to the
B. RESULTS:
advancement of efficient and scalable resource management
The results of the evaluation will provide insights into the
solutions for future IoT deployments.
efficacy of the proposed methodology for dynamic scheduling
in IoT networks. Key findings may include: V.CONCLUSION
1)Improved Performance: In conclusion, the journey towards empowering dynamic
The proposed methodology is expected to demonstrate scheduling in IoT networks through collaborative policy
improved performance compared to traditional scheduling learning via federated reinforcement techniques represents a
approaches, leading to enhanced throughput, reduced latency, significant advancement in the field of resource management
and better resource utilization in IoT networks. and optimization. Throughout this project, we have explored
innovative approaches to address the challenges posed by the
2) Scalability: dynamic and heterogeneous nature of IoT environments, aiming
The scalability of the methodology will be evaluated to to enhance system performance, scalability, and reliability. The
determine its ability to handle increasing numbers of IoT collaborative policy learning framework proposed in this
devices and dynamic workload conditions while maintaining project leverages the collective intelligence of distributed IoT
performance efficiency. devices to optimize scheduling decisions in real-time. By
3)Robustness: integrating federated reinforcement techniques, we have
The robustness of the methodology will be assessed by enabled devices to learn and adapt their scheduling policies
analysing its resilience to network failures, device failures, and autonomously while preserving data privacy and security. This
changes in environmental conditions. The ability of IoT devices decentralized approach not only improves scalability and
to adaptively adjust scheduling policies in response to dynamic resilience but also enhances resource utilization efficiency and
events will be a crucial aspect of this evaluation. user satisfaction in IoT networks. Table [1] Through
comprehensive evaluations and real-world experiments, we
4)Privacy Preservation: have demonstrated the effectiveness of the proposed
The federated reinforcement learning approach will be methodology in improving key performance metrics such as
evaluated in terms of its ability to preserve data privacy and throughput, latency, energy efficiency, and task completion
security while enabling collaborative learning across distributed rates. The results highlight the tangible benefits of collaborative
IoT devices. Fig [6] Compliance with privacy regulations and policy learning via federated reinforcement techniques in
protection against adversarial attacks will be considered in this enhancing dynamic scheduling capabilities and optimizing
evaluation. resource allocation in IoT networks.
Overall, the evaluation and results will provide valuable insights One of the key contributions of this project is the
into the feasibility and effectiveness of the proposed development of a novel framework that combines reinforcement
methodology for empowering dynamic scheduling in IoT learning, federated learning, and collaborative policy learning
networks through collaborative policy learning via federated to address the unique challenges of dynamic scheduling in IoT

979-8-3503-8386-7/24/$31.00 ©2024 IEEE

Authorized licensed use limited to: Sri Sai Ram Engineering College. Downloaded on October 15,2024 at 15:33:47 UTC from IEEE Xplore. Restrictions apply.
2024 Second International Conference on Advances in Information Technology (ICAIT-2024)

environments. By jointly optimizing scheduling decisions [3] T. Wang, Y. Lu, J. Wang, H.-N. Dai, X. Zheng, and W. Jia, “EIHDP: Edge-
across distributed devices, we have laid the groundwork for intelligent hierarchical dynamic pricing based on cloud-edge-client
collaboration for IoT systems,” IEEE Trans. Comput., vol. 70, no. 8, pp.
more adaptive, efficient, and scalable IoT deployments. 1285–1298, Aug. 2021.
Furthermore, the project underscores the importance of
[4] G. Shani, D. Heckerman, R. I. Brafman, and C. Boutilier, “An MDP-based
collaboration and collective intelligence in tackling complex recommender system,” J. Mach. Learn. Res., vol. 6, pp. 1265–1295, Sep.
optimization problems in distributed systems. By fostering 2005.
collaboration among IoT devices, we have unlocked new [5] L. Huang, M. Fu, F. Li, H. Qu, Y. Liu, and W. Chen, “A deep reinforcement
opportunities for decentralized decision-making and resource learning based long-term recommender system,” Knowl. Based Syst., vol.
management, leading to more resilient and responsive IoT 213, Feb. 2021, Art. no. 106706.
networks. Looking ahead, the insights gained from this project [6] Z. Lu and Q. Yang, “Partially observable Markov decision process for
pave the way for future research and development initiatives in recommender systems,” 2016, arXiv:1608.07793.
the field of IoT resource management and optimization. [7] Y. Wei, F. R. Yu, M. Song, and Z. Han, “User scheduling and resource
Potential avenues for further exploration include the integration allocation in HetNets with hybrid energy supply: An actor– critic
reinforcement learning approach,” IEEE Trans. Wireless Commun., vol.
of advanced machine learning techniques, such as deep 17, no. 1, pp. 680–692, Jan. 2018.
reinforcement learning and transfer learning, to enhance the
[8] H. Ye, G. Y. Li, and B.-H. F. Juang, “Deep reinforcement learning based
adaptability and robustness of scheduling policies in dynamic resource allocation for V2V communications,” IEEE Trans. Veh. Technol.,
IoT environments. Additionally, there is scope to investigate vol. 68, no. 4, pp. 3163–3173, Apr. 2019.
novel approaches for addressing emerging challenges such as [9] Z. Xu, Y. Wang, J. Tang, J. Wang, and M. C. Gursoy, “A deep reinforcement
edge computing, 5G networks, and Internet of Things security. learning based framework for power-efficient resource allocation in cloud
By continuing to innovate and collaborate, we can unlock the RANs,” in Proc. IEEE ICC, May 2017, pp. 1–6.
full potential of IoT networks and usher in a new era of [10] Z. Han and K. R. Liu, Resource Allocation for Wireless Networks: Basics,
connectivity, efficiency, and intelligence. Techniques, and Applications. Cambridge, U.K.: Cambridge Univ. Press,
2008.
In summary, the project represents a significant step forward [11] D.-Y. Kim, H. Jafarkhani, and J.-W. Lee, “Low-complexity dynamic
in empowering dynamic scheduling in IoT networks through resource scheduling for downlink MC-NOMA over fading channels,”
collaborative policy learning via federated reinforcement IEEE Trans. Wireless Commun., vol. 21, no. 5, pp. 3536–3550, May 2022.
techniques. By harnessing the power of collective intelligence [12] Y.-X. Zhu, D.-Y. Kim, and J.-W. Lee, “Joint antenna and user scheduling
and decentralized decision-making, we have laid the foundation in the massive MIMO system over time-varying fading channels,” IEEE
for more efficient, adaptive, and resilient IoT deployments, Access, vol. 9, pp. 92431–92445, 2021.
paving the way for a smarter and more connected future. [13] H.-S. Lee, D.-Y. Kim, and J.-W. Lee, “Radio and energy resource
management in renewable energy-powered wireless networks with deep
reinforcement learning,” IEEE Trans. Wireless Commun., vol. 21, no. 7,
REFERENCES
pp. 5435–5449, Jul. 2022.
[1] H. S. Chang, R. Givan, and E. K. P. Chong, “On-line scheduling via [14] H. L. Ferrá, K. Lau, C. Leckie, and A. Tang, “Applying reinforcement
sampling,” in Proc. AIPS, 2000, pp. 62–71. learning to packet scheduling in routers,” in Proc. IAAI, 2003, pp. 79–84.
[2] C. Qiu, X. Wang, H. Yao, J. Du, F. R. Yu, and S. Guo, “Networking integrated [15] S. Stidham and R. Weber, “A survey of Markov decision models for control
cloud-edge-end in IoT: A blockchain-assisted collective Q-learning of networks of queues,” Queueing Syst., vol. 13, no. 1, pp. 291–314, 1993
approach,” IEEE Internet Things J., vol. 8, no. 16, pp. 12694–12704, Aug.
2021.

979-8-3503-8386-7/24/$31.00 ©2024 IEEE

Authorized licensed use limited to: Sri Sai Ram Engineering College. Downloaded on October 15,2024 at 15:33:47 UTC from IEEE Xplore. Restrictions apply.