What Should We Engineer in Prompts? Training Humans in Requirement-Driven LLM Use
ACM Transactions on Computer-Human Interaction · 31.08.2025 · 32 (4) · 1-27
DOI: 10.1145/3731756 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Computer-Human Interaction, online erschienen am 18.08.2025. Verfasst von Qianou Ma und 5 weiteren Autor:innen. Die Arbeit wurde bisher 43-mal zitiert (Crossref) und nennt 84 Literaturangaben. Gefördert von: National Science Foundation.
Abstract (Originalsprache)
Prompting LLMs for complex tasks (e.g., building a trip advisor chatbot) needs humans to clearly articulate customized requirements (e.g., “start the response with a tl;dr”). However, existing prompt engineering instructions often lack focused training on requirement articulation and instead tend to emphasize increasingly automatable strategies (e.g., tricks like adding role-plays and “think step-by-step”). To address the gap, we introduce Requirement-Oriented Prompt Engineering ( ROPE ), a paradigm that focuses human attention on generating clear, complete requirements during prompting. We implement ROPE through an assessment and training suite that provides deliberate practice with LLM-generated feedback. In a randomized controlled experiment with 30 novices, ROPE significantly outperforms conventional prompt engineering training (20% vs. 1% gains), a gap that automatic prompt optimization cannot close. Furthermore, we demonstrate a direct correlation between the quality of input requirements and LLM outputs. Our work paves the way to empower more end-users to build complex LLM applications.
Förderung
- National Science Foundation
Zitieren
APA
Ma, Q., Peng, W., Yang, C., Shen, H., Koedinger, K., & Wu, T. (2025). What Should We Engineer in Prompts? Training Humans in Requirement-Driven LLM Use. ACM Transactions on Computer-Human Interaction, 32(4), 1-27. https://doi.org/10.1145/3731756
Vancouver
Ma Q, Peng W, Yang C, Shen H, Koedinger K, Wu T. What Should We Engineer in Prompts? Training Humans in Requirement-Driven LLM Use. ACM Transactions on Computer-Human Interaction. 2025;32(4):1-27. doi:10.1145/3731756
BibTeX
@article{Ma2025,
author = {Ma, Qianou and Peng, Weirui and Yang, Chenyang and Shen, Hua and Koedinger, Ken and Wu, Tongshuang},
title = {What Should We Engineer in Prompts? Training Humans in Requirement-Driven LLM Use},
journal = {ACM Transactions on Computer-Human Interaction},
year = {2025},
volume = {32},
number = {4},
pages = {1--27},
doi = {10.1145/3731756}
}
EUVIMED