TY - GEN
T1 - Ensuring Quality in AI-Generated Multiple-Choice Questions for Higher Education with the QUEST Framework
AU - Ebner, Martin
AU - Brünner, Benedikt
AU - Forjan, Noel
AU - Schön, Sandra
PY - 2025/6/29
Y1 - 2025/6/29
N2 - With the rise of generative AI models, such as large language models (LLMs), in educational settings, there is a growing demand to ensure the quality of AI-generated multiple-choice questions (MCQs) used in higher education. Traditional quiz development methods fall short in addressing the unique challenges posed by AI-generated content, such as consistency, cognitive demand, and question uniqueness. This paper presents the QUEST framework, a structured approach designed specifically to evaluate the quality of LLM-generated MCQs across five dimensions: Quality, Uniqueness, Effort, Structure, and Transparency. Following an iterative research process, AI-generated questions were assessed and refined using QUEST, revealing that the framework effectively improves question clarity, relevance, and educational value. The findings suggest that QUEST is a viable tool for educators to maintain high-quality standards in AI-generated assessments, ensuring these resources meet the pedagogical needs of diverse learners in higher education.
AB - With the rise of generative AI models, such as large language models (LLMs), in educational settings, there is a growing demand to ensure the quality of AI-generated multiple-choice questions (MCQs) used in higher education. Traditional quiz development methods fall short in addressing the unique challenges posed by AI-generated content, such as consistency, cognitive demand, and question uniqueness. This paper presents the QUEST framework, a structured approach designed specifically to evaluate the quality of LLM-generated MCQs across five dimensions: Quality, Uniqueness, Effort, Structure, and Transparency. Following an iterative research process, AI-generated questions were assessed and refined using QUEST, revealing that the framework effectively improves question clarity, relevance, and educational value. The findings suggest that QUEST is a viable tool for educators to maintain high-quality standards in AI-generated assessments, ensuring these resources meet the pedagogical needs of diverse learners in higher education.
UR - https://www.scopus.com/pages/publications/105010212871
U2 - 10.1007/978-3-031-95627-0_20
DO - 10.1007/978-3-031-95627-0_20
M3 - Conference paper
SN - 978-3-031-95626-3
T3 - Communications in Computer and Information Science
SP - 293
EP - 303
BT - New Media Pedagogy
A2 - Tomczyk, Lukasz
PB - Springer, Cham
T2 - 3rd International Conference, NMP 2024
Y2 - 28 November 2024 through 29 November 2024
ER -