A Reasoning and Value Alignment Test to Assess Advanced GPT Reasoning
ACM Transactions on Interactive Intelligent Systems · 30.09.2024 · 14 (3) · 1-37
DOI: 10.1145/3670691 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Interactive Intelligent Systems, online erschienen am 02.08.2024. Verfasst von Timothy R. Mcintosh und 4 weiteren Autor:innen. Die Arbeit wurde bisher 23-mal zitiert (Crossref) und nennt 65 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-SA 4.0).
Abstract (Originalsprache)
In response to diverse perspectives on artificial general intelligence (AGI), ranging from potential safety and ethical concerns to more extreme views about the threats it poses to humanity, this research presents a generic method to gauge the reasoning capabilities of artificial intelligence (AI) models as a foundational step in evaluating safety measures. Recognizing that AI reasoning measures cannot be wholly automated, due to factors such as cultural complexity, we conducted an extensive examination of five commercial generative pre-trained transformers (GPTs), focusing on their comprehension and interpretation of culturally intricate contexts. Utilizing our novel “Reasoning and Value Alignment Test,” we assessed the GPT models’ ability to reason in complex situations and grasp local cultural subtleties. Our findings have indicated that, although the models have exhibited high levels of human-like reasoning, significant limitations remained, especially concerning the interpretation of cultural contexts. This article also explored potential applications and use-cases of our Test, underlining its significance in AI training, ethics compliance, sensitivity auditing, and AI-driven cultural consultation. We concluded by emphasizing its broader implications in the AGI domain, highlighting the necessity for interdisciplinary approaches, wider accessibility to various GPT models, and a profound understanding of the interplay between GPT reasoning and cultural sensitivity.
Zitieren
APA
Mcintosh, T. R., Liu, T., Susnjak, T., Watters, P., & Halgamuge, M. N. (2024). A Reasoning and Value Alignment Test to Assess Advanced GPT Reasoning. ACM Transactions on Interactive Intelligent Systems, 14(3), 1-37. https://doi.org/10.1145/3670691
Vancouver
Mcintosh TR, Liu T, Susnjak T, Watters P, Halgamuge MN. A Reasoning and Value Alignment Test to Assess Advanced GPT Reasoning. ACM Transactions on Interactive Intelligent Systems. 2024;14(3):1-37. doi:10.1145/3670691
BibTeX
@article{Mcintosh2024,
author = {Mcintosh, Timothy R. and Liu, Tong and Susnjak, Teo and Watters, Paul and Halgamuge, Malka N.},
title = {A Reasoning and Value Alignment Test to Assess Advanced GPT Reasoning},
journal = {ACM Transactions on Interactive Intelligent Systems},
year = {2024},
volume = {14},
number = {3},
pages = {1--37},
doi = {10.1145/3670691}
}
EUVIMED