EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

STPrompt \(\boldsymbol{++}\) : Prompting Vision-Language Models for Weakly Supervised Video Anomaly Detection and Fine-Grained Localization

Peng Wu, Chengyu Pan, Guansong Pang, Xiangteng He, Zhiwei Yang, Peng Wang, Yanning Zhang

ACM Transactions on Multimedia Computing, Communications, and Applications · 31.08.2026 · 22 (8) · 1-22

DOI: 10.1145/3828722 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 10.08.2026. Verfasst von Peng Wu und 6 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 56 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, Guangdong Basic and Applied Basic Research Foundation, Singapore Ministry of Education (MOE) Academic Research Fund (AcRF) Tier 1.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Traditional weakly supervised video anomaly detection (WSVAD) tasks typically rely on coarse-grained frame-level labels for training. Although this approach reduces annotation costs, it results in weak semantic understanding and spatial localization capabilities due to the absence of fine-grained annotations, hindering precise pixel-level anomaly detection and localization. Thanks to the success of vision-language models (VLMs), e.g., CLIP, recent approaches leveraging large VLMs focus on exploiting their strong semantic understanding capabilities, but they typically feed only keyframes or short video segments into the models, without supplying sufficient prior contextual information (e.g., contextual frames around anomalies, zoomed-in anomaly regions, and detailed anomaly descriptions), which restricts the models’ capability for fine-grained anomaly understanding and precise localization. More recently, a few methods leveraging VLMs, attempt to achieve training-free spatial anomaly localization by fusing patch-level visual features with simple textual features. However, these methods employ simplistic textual descriptions, lacking deep semantic comprehension of anomalies, leading to coarse localization results with significant irrelevant background noise. To address these issues, we propose STPrompt \(++\) , a novel weakly supervised spatio-temporal video anomaly detection and localization method based on VLMs. In our work, we systematically leverage preliminary coarse localization regions derived from anomaly scores as spatial priors, together with contextual frames around keyframes, zoomed-in views of suspected anomalous regions, and refined textual descriptions of anomalies. This comprehensive prompting mechanism guides the VLMs toward deep semantic comprehension of video anomalies, enabling accurate pixel-level spatial localization. The proposed STPrompt \(++\) requires no additional training and significantly enhances the precision of anomaly understanding and localization through a carefully designed multi-round and multi-modal prompting mechanism. Extensive experiments on two widely used WSVAD benchmarks, UCF-Crime and UBnormal, show that our method achieves state-of-the-art spatial localization performance and competitive temporal anomaly detection results. Notably, on UCF-Crime dataset, our approach improves spatial localization accuracy (in TIoU) by 5.61% over the current best method (from 23.90% to 29.51%), underscoring its superior capabilities in precise anomaly localization and semantic understanding.

Förderung

  • National Natural Science Foundation of China
  • Guangdong Basic and Applied Basic Research Foundation
  • Singapore Ministry of Education (MOE) Academic Research Fund (AcRF) Tier 1
  • Lee Kong Chian Fellowship

Zitieren

APA

Wu, P., Pan, C., Pang, G., He, X., Yang, Z., Wang, P., & Zhang, Y. (2026). STPrompt \(\boldsymbol{++}\) : Prompting Vision-Language Models for Weakly Supervised Video Anomaly Detection and Fine-Grained Localization. ACM Transactions on Multimedia Computing, Communications, and Applications, 22(8), 1-22. https://doi.org/10.1145/3828722

Vancouver

Wu P, Pan C, Pang G, He X, Yang Z, Wang P, et al. STPrompt \(\boldsymbol{++}\) : Prompting Vision-Language Models for Weakly Supervised Video Anomaly Detection and Fine-Grained Localization. ACM Transactions on Multimedia Computing, Communications, and Applications. 2026;22(8):1-22. doi:10.1145/3828722

BibTeX
@article{Wu2026,
  author = {Wu, Peng and Pan, Chengyu and Pang, Guansong and He, Xiangteng and Yang, Zhiwei and Wang, Peng and Zhang, Yanning},
  title = {STPrompt \(\boldsymbol++\) : Prompting Vision-Language Models for Weakly Supervised Video Anomaly Detection and Fine-Grained Localization},
  journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
  year = {2026},
  volume = {22},
  number = {8},
  pages = {1--22},
  doi = {10.1145/3828722}
}
Kostenlos starten →