Testing and Reinforcement Learning – A Structured Literature Review

Reinforcement Learning has gained much attention in the last decade, leading to substantial progress in Artificial Intelligence and its applications, especially in games and other areas where agents interact to learn and adapt to complex environments successfully. In this paper, we focus on reinforcement learning in the context of software and system testing. Specifically, we want to review the use of reinforcement learning in testing particularly test automation and testing approaches used for assuring the quality of reinforcement learning implementations. For this purpose, we carried out a structured literature review that considered Scopus, the digital libraries of IEEE and ACM. Using exclusion and inclusion criteria, we finally obtained 79 scientific articles. We categorized these articles according to various criteria, like the testing methods used or the application of testing, e.g., for obtaining security vulnerabilities, for automated game testing devoid of human competitive intelligence aid, for exploring GUI actions and states, etc. Hence, most papers focus on applying reinforcement learning for testing, but there is little
work on testing implementations of reinforcement learning.