EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

CP-Diffusion: Conditional Prompt-Based Diffusion Models for Video Generation

Muhammad Saeed, Mustaqeem Khan, Muhammad Saad, Nasir Rahim, Wail Gueaieb, Abdulmotaleb El Saddik

ACM Transactions on Multimedia Computing, Communications, and Applications · 31.07.2026 · 22 (7) · 1-16

DOI: 10.1145/3793552 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 24.06.2026. Verfasst von Muhammad Saeed und 5 weiteren Autor:innen. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 46 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Motion customization plays a pivotal role in video generation by preserving the original appearance and context while adhering to specific motion patterns. In contrast, video generation techniques often lack coherence and realism due to difficulties in capturing and transferring motion patterns. Building upon the Video Motion Customization (VMC) framework, we proposed a few-shot learning approach using our unified Multi-Head Temporal Attention (MHTA) module for motion customization in text-to-video diffusion models. This significantly reduces computational requirements while maintaining and improving motion quality. Our model provides a streamlined mechanism for motion distillation while maintaining separate self-, cross-, and temporal attention. Moreover, the temporal attention layer is adapted through a simplified mechanism with efficient Q/K/V projections, while maintaining fixed spatial self- and cross-attention. The model distills a ground-truth motion vector from consecutive frames to align the predicted and ground-truth motion. Our proposed MHTA model outperforms the baseline in video generation using motion customization while being significantly more resource-efficient. Moreover, our approach can easily be applied to generate conditional prompt-based videos in the gaming industry.

Zitieren

APA

Saeed, M., Khan, M., Saad, M., Rahim, N., Gueaieb, W., & Saddik, A. E. (2026). CP-Diffusion: Conditional Prompt-Based Diffusion Models for Video Generation. ACM Transactions on Multimedia Computing, Communications, and Applications, 22(7), 1-16. https://doi.org/10.1145/3793552

Vancouver

Saeed M, Khan M, Saad M, Rahim N, Gueaieb W, Saddik AE. CP-Diffusion: Conditional Prompt-Based Diffusion Models for Video Generation. ACM Transactions on Multimedia Computing, Communications, and Applications. 2026;22(7):1-16. doi:10.1145/3793552

BibTeX
@article{Saeed2026,
  author = {Saeed, Muhammad and Khan, Mustaqeem and Saad, Muhammad and Rahim, Nasir and Gueaieb, Wail and Saddik, Abdulmotaleb El},
  title = {CP-Diffusion: Conditional Prompt-Based Diffusion Models for Video Generation},
  journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
  year = {2026},
  volume = {22},
  number = {7},
  pages = {1--16},
  doi = {10.1145/3793552}
}
Kostenlos starten →