Antonino Masaracchia is a Lecturer at Queen Mary University of London. Previously, he was a research fellow at the Centre for Wireless Innovation at Queen's University Belfast, UK. His research activities are focused on optimal resource allocation for enabling URLLCs services in future 5G and 6G networks, and the investigation of innovative real-time optimisation systems for future wireless networks with digital twins. His research interests include digital twins, URLLCs, UAV communications, real-time optimisation, and AI and ML techniques for wireless communication networks. He has authored or co-authored 50+books, book chapters, journals, and conference papers. He has been awarded with the seal of excellence for the project proposal UAV-SURE and UAV-DRESS submitted under the Horizon 2020 s Marie Sklodowska-Curie actions call H2020-MSCA-IF-2020 (September 2020), and Horizon Europe Marie Skodowska-Curie Actions call HORIZON-MSCA-2021-PF-01-01 MSCA Postdoctoral Fellowships (October 2021), respectively.Khoi Khac Nguyen completed his PhD degree with the School of Electronics, Electrical Engineering and Computer Science, Queen's University Belfast, UK. His research interests include machine learning and deep reinforcement learning for real-time optimisation in wireless networks, reconfigurable intelligent surfaces, unmanned air vehicle (UAV) communication, and massive Internet of Things (IoTs).Trung Q. Duong is a Canada Excellence Research Chair (CERC) and full professor at Memorial University, Canada. He is also an adjunct chair professor in telecommunications at Queen's University Belfast, UK and a research chair of the Royal Academy of Engineering. His research interests include machine learning, real-time optimisation, data analytics, and 5G-6G networks. He has authored or co-authored 500+ books, book chapters, journals, and conference papers. He is a recipient of the prestigious Royal Academy of Engineering Research Fellowship (2016-20) and won the Newton Prize in 2017.Vishal Sharma is a senior lecturer in the School of Electronics, Electrical Engineering and Computer Science (EEECS) at Queen's University Belfast (QUB), Northern Ireland, UK. At QUB, he focuses on cyber defence and security and leads research on drone security, digital twins (DT), and blockchain systems. Since 2022, he has been affiliated with the Global Innovation Institute as a fellow and works with the Centre for Secure Information Technologies (CSIT) and the Centre for Intelligent Sustainable Computing (CISC). He is the director of the Innovation-by-design lab at QUB. He leads the British Computer Society (BCS) Accreditation for QUB and is the chair of the Computer Science Programme Review Working Group. He is the co-investigator for the Northern Ireland Advanced Research and Engineering Centre (ARC). He has authored/co-authored more than 140 journal/conference articles and book chapters, co-edited four books and won seven best paper awards. He has served on the editorial board of IEEE Communications Magazine and as section editor-in-chief of Drones journal. Currently, he serves as associate editor of CAAI Transactions on Intelligence Technology, IET Networks, and ICT Express. He is also the co-chair of the IEEE UK and Ireland Diversity, Equity, and Inclusion Committee. He is a senior member of IEEE and a professional member of ACM. He received his PhD degree in Computer Science and Engineering from Thapar University, India.

• Part I: Introduction to machine learning and neural networksChapter 1: Artificial intelligence, machine learning, and deep learningChapter 2: Deep neural networks Part II: Deep reinforcement learningChapter 3: Markov decision processChapter 4: Value function approximation for continuous state-action spaceChapter 5: Policy search methods for reinforcement learningChapter 6: Actor-critic learning Part III: Deep reinforcement learning in UAV-assisted 6G communicationChapter 7: UAV-assisted 6G communicationsChapter 8: Distributed deep deterministic policy gradient for power allocation control in UAV-to-UAV-based communicationsChapter 9: Non-cooperative energy-efficient power allocation game in UAV-to-UAV communication: a multi-agent deep reinforcement learning approachChapter 10: Real-time energy harvesting-aided scheduling in UAV-assisted D2D networksChapter 11: 3D trajectory design and data collection in UAV-assisted networks Part IV: Deep reinforcement learning in reconfigurable intelligent surface-empowered 6G communicationsChapter 12: RIS-assisted 6G communicationsChapter 13: Real-time optimisation in RIS-assisted D2D communicationsChapter 14: RIS-assisted UAV communications for IoT with wireless power transfer using deep reinforcement learningChapter 15: Multi-agent learning in networks supported by RIS and multi-UAVs

Antonino Masaracchia is a Lecturer at Queen Mary University of London. Previously, he was a research fellow at the Centre for Wireless Innovation at Queen's University Belfast, UK. His research activities are focused on optimal resource allocation for enabling URLLCs services in future 5G and 6G networks, and the investigation of innovative real-time optimisation systems for future wireless networks with digital twins. His research interests include digital twins, URLLCs, UAV communications, real-time optimisation, and AI and ML techniques for wireless communication networks. He has authored or co-authored 50+books, book chapters, journals, and conference papers. He has been awarded with the seal of excellence for the project proposal UAV-SURE and UAV-DRESS submitted under the Horizon 2020 s Marie Sklodowska-Curie actions call H2020-MSCA-IF-2020 (September 2020), and Horizon Europe Marie Skodowska-Curie Actions call HORIZON-MSCA-2021-PF-01-01 MSCA Postdoctoral Fellowships (October 2021), respectively.Khoi Khac Nguyen completed his PhD degree with the School of Electronics, Electrical Engineering and Computer Science, Queen's University Belfast, UK. His research interests include machine learning and deep reinforcement learning for real-time optimisation in wireless networks, reconfigurable intelligent surfaces, unmanned air vehicle (UAV) communication, and massive Internet of Things (IoTs).Trung Q. Duong is a Canada Excellence Research Chair (CERC) and full professor at Memorial University, Canada. He is also an adjunct chair professor in telecommunications at Queen's University Belfast, UK and a research chair of the Royal Academy of Engineering. His research interests include machine learning, real-time optimisation, data analytics, and 5G-6G networks. He has authored or co-authored 500+ books, book chapters, journals, and conference papers. He is a recipient of the prestigious Royal Academy of Engineering Research Fellowship (2016-20) and won the Newton Prize in 2017.Vishal Sharma is a senior lecturer in the School of Electronics, Electrical Engineering and Computer Science (EEECS) at Queen's University Belfast (QUB), Northern Ireland, UK. At QUB, he focuses on cyber defence and security and leads research on drone security, digital twins (DT), and blockchain systems. Since 2022, he has been affiliated with the Global Innovation Institute as a fellow and works with the Centre for Secure Information Technologies (CSIT) and the Centre for Intelligent Sustainable Computing (CISC). He is the director of the Innovation-by-design lab at QUB. He leads the British Computer Society (BCS) Accreditation for QUB and is the chair of the Computer Science Programme Review Working Group. He is the co-investigator for the Northern Ireland Advanced Research and Engineering Centre (ARC). He has authored/co-authored more than 140 journal/conference articles and book chapters, co-edited four books and won seven best paper awards. He has served on the editorial board of IEEE Communications Magazine and as section editor-in-chief of Drones journal. Currently, he serves as associate editor of CAAI Transactions on Intelligence Technology, IET Networks, and ICT Express. He is also the co-chair of the IEEE UK and Ireland Diversity, Equity, and Inclusion Committee. He is a senior member of IEEE and a professional member of ACM. He received his PhD degree in Computer Science and Engineering from Thapar University, India.

• Part I: Introduction to machine learning and neural networksChapter 1: Artificial intelligence, machine learning, and deep learningChapter 2: Deep neural networks Part II: Deep reinforcement learningChapter 3: Markov decision processChapter 4: Value function approximation for continuous state-action spaceChapter 5: Policy search methods for reinforcement learningChapter 6: Actor-critic learning Part III: Deep reinforcement learning in UAV-assisted 6G communicationChapter 7: UAV-assisted 6G communicationsChapter 8: Distributed deep deterministic policy gradient for power allocation control in UAV-to-UAV-based communicationsChapter 9: Non-cooperative energy-efficient power allocation game in UAV-to-UAV communication: a multi-agent deep reinforcement learning approachChapter 10: Real-time energy harvesting-aided scheduling in UAV-assisted D2D networksChapter 11: 3D trajectory design and data collection in UAV-assisted networks Part IV: Deep reinforcement learning in reconfigurable intelligent surface-empowered 6G communicationsChapter 12: RIS-assisted 6G communicationsChapter 13: Real-time optimisation in RIS-assisted D2D communicationsChapter 14: RIS-assisted UAV communications for IoT with wireless power transfer using deep reinforcement learningChapter 15: Multi-agent learning in networks supported by RIS and multi-UAVs

Antonino Masaracchia is a Lecturer at Queen Mary University of London. Previously, he was a research fellow at the Centre for Wireless Innovation at Queen's University Belfast, UK. His research activities are focused on optimal resource allocation for enabling URLLCs services in future 5G and 6G networks, and the investigation of innovative real-time optimisation systems for future wireless networks with digital twins. His research interests include digital twins, URLLCs, UAV communications, real-time optimisation, and AI and ML techniques for wireless communication networks. He has authored or co-authored 50+books, book chapters, journals, and conference papers. He has been awarded with the seal of excellence for the project proposal UAV-SURE and UAV-DRESS submitted under the Horizon 2020 s Marie Sklodowska-Curie actions call H2020-MSCA-IF-2020 (September 2020), and Horizon Europe Marie Skodowska-Curie Actions call HORIZON-MSCA-2021-PF-01-01 MSCA Postdoctoral Fellowships (October 2021), respectively.Khoi Khac Nguyen completed his PhD degree with the School of Electronics, Electrical Engineering and Computer Science, Queen's University Belfast, UK. His research interests include machine learning and deep reinforcement learning for real-time optimisation in wireless networks, reconfigurable intelligent surfaces, unmanned air vehicle (UAV) communication, and massive Internet of Things (IoTs).Trung Q. Duong is a Canada Excellence Research Chair (CERC) and full professor at Memorial University, Canada. He is also an adjunct chair professor in telecommunications at Queen's University Belfast, UK and a research chair of the Royal Academy of Engineering. His research interests include machine learning, real-time optimisation, data analytics, and 5G-6G networks. He has authored or co-authored 500+ books, book chapters, journals, and conference papers. He is a recipient of the prestigious Royal Academy of Engineering Research Fellowship (2016-20) and won the Newton Prize in 2017.Vishal Sharma is a senior lecturer in the School of Electronics, Electrical Engineering and Computer Science (EEECS) at Queen's University Belfast (QUB), Northern Ireland, UK. At QUB, he focuses on cyber defence and security and leads research on drone security, digital twins (DT), and blockchain systems. Since 2022, he has been affiliated with the Global Innovation Institute as a fellow and works with the Centre for Secure Information Technologies (CSIT) and the Centre for Intelligent Sustainable Computing (CISC). He is the director of the Innovation-by-design lab at QUB. He leads the British Computer Society (BCS) Accreditation for QUB and is the chair of the Computer Science Programme Review Working Group. He is the co-investigator for the Northern Ireland Advanced Research and Engineering Centre (ARC). He has authored/co-authored more than 140 journal/conference articles and book chapters, co-edited four books and won seven best paper awards. He has served on the editorial board of IEEE Communications Magazine and as section editor-in-chief of Drones journal. Currently, he serves as associate editor of CAAI Transactions on Intelligence Technology, IET Networks, and ICT Express. He is also the co-chair of the IEEE UK and Ireland Diversity, Equity, and Inclusion Committee. He is a senior member of IEEE and a professional member of ACM. He received his PhD degree in Computer Science and Engineering from Thapar University, India.

• Part I: Introduction to machine learning and neural networksChapter 1: Artificial intelligence, machine learning, and deep learningChapter 2: Deep neural networks Part II: Deep reinforcement learningChapter 3: Markov decision processChapter 4: Value function approximation for continuous state-action spaceChapter 5: Policy search methods for reinforcement learningChapter 6: Actor-critic learning Part III: Deep reinforcement learning in UAV-assisted 6G communicationChapter 7: UAV-assisted 6G communicationsChapter 8: Distributed deep deterministic policy gradient for power allocation control in UAV-to-UAV-based communicationsChapter 9: Non-cooperative energy-efficient power allocation game in UAV-to-UAV communication: a multi-agent deep reinforcement learning approachChapter 10: Real-time energy harvesting-aided scheduling in UAV-assisted D2D networksChapter 11: 3D trajectory design and data collection in UAV-assisted networks Part IV: Deep reinforcement learning in reconfigurable intelligent surface-empowered 6G communicationsChapter 12: RIS-assisted 6G communicationsChapter 13: Real-time optimisation in RIS-assisted D2D communicationsChapter 14: RIS-assisted UAV communications for IoT with wireless power transfer using deep reinforcement learningChapter 15: Multi-agent learning in networks supported by RIS and multi-UAVs