Real-World Study of the Security of Educational Test Systems

Computer science education has a unique setting where students write code commonly automatically tested in so-called “test systems”. While best practices for sandboxing are known in the academic community, the security of real-world test systems remains unclear.

In this work, we evaluate the security of 11 real-world test systems from computer science university classes, including computer security classes. We studied these systems between October 2023 and February 2024 and provide a systematic overview of the typical approaches these systems follow. We identify 3 categories of systems: GitLab Runners with a Docker registry, GitLab Runners with custom pipelines, and entirely custom test systems that do not rely on a CI system as a basis. We identify 13 types of security issues, the most widely spread ones affecting 5-6 of the test systems in our analysis. We practically show that all test systems in our analysis can be compromised and develop new techniques to exfiltrate secret and privileged information, including the use of side channels. We present 3 cases studies, demonstrating specific bypasses possible in these systems. Finally, in a user study, we assess the impact a potential breach together with the educators using these test systems. Our work shows that educational test systems are particularly critical, as a compromise can lead to the exposure of highly sensitive student and research data, and even embargoed vulnerabilities. Our results highlight that the real-world challenges to run and maintain secure test systems are not solved in practice. While we discuss best practices, our study reveals the need for new systematic security approaches to secure this very common type of software system.