Preface xv -- 1 Deep Reinforcement Learning Applications in Real-World Scenarios: Challenges and Opportunities 1 Sunilkumar Ketineni and Sheela J. -- 1.1 Introduction 1 -- 1.1.1 Problems with Real-World Implementation 2 -- 1.2 Application to the Real World 3 -- 1.2.1 Security and Robustness 3 -- 1.2.2 Generalization 5 -- 1.2.2.1 Overcoming Challenges in DRL 9 -- 1.3 Possibilities for Making a Difference in the Real World 11 -- 1.3.1 Transfer Learning and Domain Adaptation 11 -- 1.4 Meta-Learning 12 -- 1.5 Deep Reinforcement Learning (DRL) 13 -- 1.5.1 Hybrid Approaches 14 -- 1.6 Online vs. Offline Reinforcement Learning 15 -- 1.7 Human-in-the-Loop Systems 15 -- 1.8 Benchmarking and Standardization 16 -- 1.9 Collaborative Multi-Agent Systems 18 -- 1.10 Transfer Learning and Domain Adaptation 19 -- 1.11 Hierarchical and Multimodal Learning 21 -- 1.12 Imitation Learning and Human Feedback 22 -- 1.13 Inverse Reinforcement Learning 23 -- 1.14 Sim-to-Real Transfer 24 -- 1.15 Conclusion 25 -- References 26 -- 2 Deep Reinforcement Learning: A Key to Unlocking the Potential of Robotics and Autonomous Systems 29 Saksham and Chhavi Rana -- 2.1 Introduction 30 -- 2.1.1 Significance of DRL Field 30 -- 2.1.2 Transformative Advantages of DRL Field 32 -- 2.2 Fields of Investigation 33 -- 2.2.1 General Methods for Investigation 34 -- 2.3 Background 36 -- 2.3.1 Fundamentals of Deep Reinforcement Learning (DRL) 38 -- 2.4 Deep Reinforcement Learning (DRL) in Robot Control 39 -- 2.4.1 Navigation and Localization 40 -- 2.4.2 Object Manipulation 42 -- 2.5 Applications and Case Studies 43 -- 2.6 Challenges and Future Directions 44 -- 2.7 Evaluation and Metrics 46 -- 2.8 Summary 47 -- References 48 -- 3 Deep Reinforcement Learning Algorithms: A Comprehensive Overview 51 Shweta V. Bondre, Bhakti Thakre, Uma Yadav and Vipin D. Bondre -- 3.1 Introduction 52 -- 3.1.1 How Reinforcement Learning Works? 53 -- 3.2 Reinforcement Learning Algorithms 53 -- 3.2.1 Value-Based Algorithms 53 -- 3.2.1.1 Q-Learning 53 -- 3.2.1.2 Deep Q-Networks (DQN) 57 -- 3.2.1.3 Double DQN 58 -- 3.2.1.4 Dueling DQN 58 -- 3.3 Policy-Based 59 -- 3.3.1 Policy Gradient Methods 59 -- 3.3.2 REINFORCE (Monte Carlo Policy Gradient) 60 -- 3.3.3 Actor–Critic Methods 61 -- 3.3.4 Natural Policy Gradient Methods 62 -- 3.4 Model-Based Reinforcement Learning 63 -- 3.4.1 Probabilistic Ensembles with Trajectory Sampling (PETS) 63 -- 3.4.2 Probabilistic Inference for Learning Control (PILCO) 64 -- 3.4.3 Model Predictive Control (MPC) 65 -- 3.4.4 Model-Agnostic Meta-Learning (MAML) 66 -- 3.4.5 Soft Actor–Critic with Model Ensemble 67 -- 3.4.6 Deep Deterministic Policy Gradients with Model (DDPG with Model) 68 -- 3.5 Characteristics of Reinforcement Learning 69 -- 3.6 DRL Algorithms and Their Advantages and Drawbacks 71 -- 3.7 Conclusion 72 -- References 72 -- 4 Deep Reinforcement Learning in Healthcare and Biomedical Applications 75 Balakrishnan D., Aarthy C., Nandhagopal Subramani, Venkatesan R. and Logesh T. R. -- 4.1 Introduction 76 -- 4.2 Related Works 76 -- 4.3 Deep Reinforcement Learning Framework 80 -- 4.4 Deep Reinforcement Learning Applications in Healthcare and Biomedicine 81 -- 4.5 Deep Reinforcement Learning Employs Efficient Algorithms 82 -- 4.5.1 Deep Q-Networks 82 -- 4.5.2 Policy Differentiation Techniques 82 -- 4.5.3 Hindsight Experience Replay (HER) 82 -- 4.5.4 Curiosity-Driven Exploration 82 -- 4.5.5 Long Short-Term Memory Networks and Recurring Neural Network Designs 82 -- 4.5.6 Multi-Agent DRL 83 -- 4.6 Semi-Autonomous Control Based on Deep Reinforcement Learning for Robotic Surgery 83 -- 4.6.1 Double Deep Q-Network (DDQN) 83 -- 4.6.2 Materials and Methods 84 -- 4.6.3 Results 86 -- 4.6.4 Discussion 87 -- 4.7 Conclusion 87 -- References 88 -- 5 Application of Deep Reinforcement Learning in Adversarial Malware Detection 91 Manju and Chhavi Rana -- 5.1 Introduction 91 -- 5.1.1 Background 95 -- 5.1.2 Significance of Malware Detection 96 -- 5.1.3 Challenges with Adversarial Attacks 96 -- 5.2 Foundations of Deep Reinforcement Learning 97 -- 5.2.1 Overview of Deep Reinforcement Learning 98 -- 5.2.2 Core Concepts and Components 99 -- 5.2.3 Relevance to Malware Detection 100 -- 5.3 Malware Detection Landscape 101 -- 5.3.1 Evolution of Malware Detection Techniques 102 -- 5.3.2 Adversarial Attacks in Cybersecurity 103 -- 5.3.3 Need for Advanced Detection Strategies 104 -- 5.4 Deep Reinforcement Learning Techniques 104 -- 5.4.1 Application of Deep Learning in Malware Detection 105 -- 5.4.2 Reinforcement Learning Algorithms 106 -- 5.5 Feature Selection Strategies 107 -- 5.5.1 Importance of Feature Selection in Malware Detection 108 -- 5.5.2 Techniques for Feature Selection 108 -- 5.5.3 Optimization for Deep Reinforcement Learning Models 109 -- 5.6 Datasets and Evaluation 110 -- 5.7 Generating Adversarial Samples 111 -- Conclusion and Future Directions 112 -- Future Directions 112 -- References 112 -- 6 Artificial Intelligence in Blockchain and Smart Contracts for Disruptive Innovation 115 Eashwar Sivakumar, Kiran Jot Singh and Paras Chawla -- 6.1 Introduction 115 -- 6.1.1 Smart Contract 116 -- 6.2 Literature Review 117 -- 6.2.1 Blockchain and Smart Contracts in Digital Identity 117 -- 6.2.2 Blockchain and Smart Contracts in Financial Security 118 -- 6.2.3 Blockchain and Smart Contracts in Supply Chain Management 119 -- 6.2.4 Blockchain and Smart Contracts in Insurance 120 -- 6.2.5 Blockchain and Smart Contracts in Healthcare 121 -- 6.2.6 Blockchain and Smart Contracts in Agriculture 121 -- 6.2.7 Blockchain and Smart Contracts in Real Estate 122 -- 6.2.8 Blockchain and Smart Contracts in Education and Research 123 -- 6.2.9 Blockchain and Smart Contracts in Other Sectors 124 -- 6.3 Critical Analysis of the Review 125 -- 6.4 Blockchain and Artificial Intelligence 128 -- 6.5 Discussion on the Reasoning for Implementation of Blockchain 129 -- 6.6 Conclusion 130 -- References 130 -- 7 Clinical Intelligence: Deep Reinforcement Learning for Healthcare and Biomedical Advancements 137 Keerthika K., Kannan M. and T. Saravanan -- 7.1 Introduction 138 -- 7.2 Deep Reinforcement Learning Methods 138 -- 7.2.1 Model-Free Methods 138 -- 7.2.2 Policy Gradient Methods 139 -- 7.2.3 Model-Based Methods 139 -- 7.3 Applications of DRL in Healthcare 140 -- 7.3.1 Tailored Treatment Recommendations 140 -- 7.3.2 Optimization of Clinical Trials 141 -- 7.3.3 Disease Diagnosis Support 142 -- 7.3.4 Accelerated Drug Discovery and Design 142 -- 7.3.5 Enhanced Robotic Surgery and Assistance 142 -- 7.3.6 Health Management System 143 -- 7.4 Challenges 143 -- 7.5 Healthcare Data Types 144 -- 7.5.1 Electronic Healthcare Records (EHRs) 144 -- 7.5.2 Laboratory Data 145 -- 7.5.3 Sensor Data 145 -- 7.5.4 Biomedical Imaging Information 145 -- 7.6 Guidelines for the Application of DRL 147 -- 7.7 A Case Study: DRL in Healthcare and Biomedical Applications 147 -- 7.7.1 Optimizing Radiation Therapy Dose Distribution in Cancer Treatment 147 -- 7.7.2 Dose Strategy Model in Sepsis Patient Treatment 148 -- References 149 -- 8 Cultivating Expertise in Deep and Reinforcement Learning Principles 151 Chilakalapudi Malathi and J.

Sheela -- 8.1 Introduction 151 -- 8.1.1 Reinforcement Learning’s Constituent Parts 152 -- 8.1.2 Process of Markov Decisions (MDP) 152 -- 8.1.3 Learning Reinforcement Methods 153 -- 8.2 Intensive Learning Foundations 164 -- 8.2.1 A Definition of Deep Learning 164 -- 8.2.2 Deep Learning Elements 164 -- 8.2.2.1 Different Kinds of Deep Learning Networks 165 -- 8.3 Integrating Deep Learning and Reinforcement Learning 172 -- 8.3.1 Deep Reinforcement Learning 172 -- 8.3.2 Deep Reinforcement Learning Complexity Problems 174 -- Conclusion 175 -- References 175 -- 9 Deep Reinforcement Learning in Healthcare and Biomedical Research 179 Shruti Agrawal and Pralay Mitra -- 9.1 Introduction 180 -- 9.1.1 Reinforcement Learning 180 -- 9.1.2 Deep Reinforcement Learning 181 -- 9.2 Learning Methods in Bioinformatics with Applications in Healthcare and Biomedical Research 182 -- 9.2.1 Protein Folding 182 -- 9.2.2 Protein Docking 183 -- 9.2.3 Protein–Ligand Binding 185 -- 9.2.4 Binding Peptide Generation 187 -- 9.2.5 Protein Design and Engineering 188 -- 9.2.6 Drug Discovery and Development 190 -- 9.3 Applications in Biological Data 192 -- 9.3.1 Omics Data 192 -- 9.3.2 Medical Imaging 192 -- 9.3.3 Brain/Body–Machine Interfaces 193 -- 9.4 Adaptive Treatment Approach in Healthcare 193 -- 9.5 Diagnostic Tools in Healthcare and Biomedical Research 195 -- 9.6 Scope of Deep Reinforcement Learning in Healthcare and Biomedical Applications 196 -- 9.6.1 State and Action Space 196 -- 9.6.2 Reward 197 -- 9.6.3 Policy 198 -- 9.6.4 Model Training 199 -- 9.6.5 Exploration 199 -- 9.6.6 Credit Assignment 200 -- 9.7 Conclusions 200 -- References 201 -- 10 Deep Reinforcement Learning in Robotics and Autonomous Systems 207 Uma Yadav, Shweta V. Bondre and Bhakti Thakre -- 10.1 Introduction 208 -- 10.2 The Promise of Deep Reinforcement Learning (DRL) in Real-World Robot ...