Do Comments Matter? Investigating Students’ Source Code Comment Behaviour and Its Relation to Academic Success in a CS1 Course

Comments in source code have been perceived to enhance understandability, readability, and knowledge retention by professional programmers and educators alike. Programming novices in introductory programming courses are therefore taught to use comments to highlight specific code sections, clarify complex algorithms, and organize source code structure. Despite the perceived importance of comments, little research exists regarding the relation between commenting behavior and student performance in such courses, which is inherently linked to code correctness. To expand on this gap, this study analyzes over 40000 comments across 2800 submissions from 1085 students over two semesters enrolled in a first-semester computer science 1 (CS1) course at a Western European university. Our analysis reveals a notable thematic difference in the use of comments between high-performing and failing students: high-performing students utilize comments primarily for task-related explanations, whereas failing students tend to more frequently use them to describe the program’s syntax. However, there is no significant correlation between the actual number of comments and students’ course performance. Finally, all students exhibit a considerable shift in commenting behavior over time, transitioning from full sentences to conveying essential information through keywords instead. Altogether, this large-scale study, which draws upon data from a realistic educational context, sheds light on the role of code comments in student performance, offering insights beneficial to both educators and researchers.