Methodological foundations for artificial intelligence ‐driven survey question generation
Journal of Engineering Education · 2025-07 · 114 (3)
DOI: 10.1002/jee.70012 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Engineering Education, online erschienen am 22.05.2025. Verfasst von Ted K. Mburu und 3 weiteren Autor:innen. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 47 Literaturangaben. Gefördert von: Cornell University.
Abstract (Originalsprache)
Background
This study investigates the use of large language models to create adaptive, contextually relevant survey questions, aiming to enhance data quality in educational research without limiting scalability.
Purpose
We provide step‐by‐step methods to develop a dynamic survey instrument, driven by artificial intelligence (AI), and introduce the Synthetic Question–Response Analysis (SQRA) framework, a methodology designed to help evaluate AI‐generated questions before deployment with human participants.
Design
We examine the questions generated by our survey instrument, as well as compare AI‐to‐AI, generated through our SQRA framework, with AI‐to‐human interactions. Activity theory provides a theoretical lens to examine the dynamic interactions between AI and participants, highlighting the mutual influence within the survey tool.
Results
We found that AI‐generated questions were contextually relevant and adaptable, successfully incorporating course‐specific references. However, issues such as redundant phrasing, double‐barreled questions, and jargon affected the clarity of the questions. Although the SQRA framework exhibited limitations in replicating human response variability, its iterative refinement process proved effective in improving question quality, reinforcing the utility of this approach for enhancing AI‐driven surveys.
Conclusions
While AI‐driven question generation can enhance the scalability and personalization of open‐ended survey prompts, more research is needed to establish best practices for high‐quality educational research. The SQRA framework demonstrated practical utility for prompt refinement and initial validation of AI‐generated survey content, but it is not capable of replicating human responses. We highlight the importance of iterative prompt engineering, ethical considerations, and the need for methodological advancements in the development of trustworthy AI‐driven survey instruments for educational research.
Förderung
- Cornell University
Zitieren
APA
Mburu, T. K., Rong, K., McColley, C. J., & Werth, A. (2025). Methodological foundations for artificial intelligence ‐driven survey question generation. Journal of Engineering Education, 114(3). https://doi.org/10.1002/jee.70012
Vancouver
Mburu TK, Rong K, McColley CJ, Werth A. Methodological foundations for artificial intelligence ‐driven survey question generation. Journal of Engineering Education. 2025;114(3). doi:10.1002/jee.70012
BibTeX
@article{Mburu2025,
author = {Mburu, Ted K. and Rong, Kangxuan and McColley, Campbell J. and Werth, Alexandra},
title = {Methodological foundations for artificial intelligence ‐driven survey question generation},
journal = {Journal of Engineering Education},
year = {2025},
volume = {114},
number = {3},
doi = {10.1002/jee.70012}
}
EUVIMED