List of Figures xv -- About the Authors xix -- Foreword xxi -- Preface xxiii -- Acknowledgments xxv -- Acronyms xxvii -- Introduction xxix -- 1 Motivation 1 -- 1.1 Introduction 1 -- 1.1.1 Cyberattack Campaigns via MITRE ATT&CK 4 -- 1.2 Attack Graphs 4 -- 1.3 Cyber Terrain 5 -- 1.4 Penetration Testing 6 -- 1.5 AI Reinforcement Learning Overview 6 -- 1.6 Organization of the Book 8 -- 2 Overview of Penetration Testing 11 -- 2.1 Penetration Testing 11 -- 2.2 Importance of Data 43 -- 2.3 Conclusion 56 -- 3 Reinforcement Learning: Theory and Application 61 -- 3.1 An Introduction to Reinforcement Learning (RL) 61 -- 3.2 RL and Markov Decision Processes 63 -- 3.3 Learnable Functions for Agents 66 -- 3.4 Enter Deep Learning 69 -- 3.5 Q-Learning and Deep Q-Learning 72 -- 3.6 Advantage Actor-Critic (A2C) 78 -- 3.7 Proximal Policy Optimization 83 -- 3.8 Conclusion 85 -- 4 Motivation for Model-driven Penetration Testing 89 -- 4.1 Introduction 89 -- 4.2 Limits of Modern Attack Graphs 91 -- 4.3 RL for Penetration Testing 93 -- 4.4 Modeling MDPs 95 -- 4.5 Conclusion 98 -- 5 Operationalizing RL for Cyber Operations 105 -- 5.1 A High-Level Architecture 105 -- 5.2 Layered Reference Model 107 -- 5.3 Key Challenges for Operationalizing RL 113 -- 5.4 Conclusions 117 -- 6 Toward Practical RL for Pen-Testing 121 -- 6.1 Current Challenges to Practicality 121 -- 6.2 Practical Scalability in RL 130 -- 6.3 Model Realism 136 -- 6.4 Examples of Applications 144 -- 6.5 Realism and Scale 154 -- 7 Putting it Into Practice: RL for Scalable Penetration Testing 161 -- 7.1 Crown Jewels Analysis 161 -- 7.2 Discovering Exfiltration Paths 165 -- 7.3 Discovering Command and Control Channels 171 -- 7.4 Exposing Surveillance Detection Routes 176 -- 7.5 Enhanced Exfiltration Path Analysis 183 -- 8 Using and Extending These Models 193 -- 8.1 Supplementing Penetration Testing 193 -- 8.2 Risk Scoring 199 -- 8.3 Further Modeling 201 -- 8.4 Generalization 214 -- 9 Model-driven Penetration Testing in Practice 225 -- 9.1 Recap 225 -- 9.2 The Case for Model-driven Cyber Detections 231 -- References 246 -- A Appendix 251 -- Index 253.