Model Compression Algorithm via Reinforcement Learning and Knowledge Distillation
Mathematics · 09.11.2023 · 11 (22) · 4589
DOI: 10.3390/math11224589 ↗Kurz gefasst
Fachartikel in der Zeitschrift Mathematics, online erschienen am 09.11.2023. Verfasst von Botao Liu und 4 weiteren Autor:innen. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 39 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: New Generation Information Technology Innovation Project.
Abstract (Originalsprache)
Traditional model compression techniques are dependent on handcrafted features and require domain experts, with a tradeoff between model size, speed, and accuracy. This study proposes a new approach toward resolving model compression problems. Our approach combines reinforcement-learning-based automated pruning and knowledge distillation to improve the pruning of unimportant network layers and the efficiency of the compression process. We introduce a new state quantity that controls the size of the reward and an attention mechanism that reinforces useful features and attenuates useless features to enhance the effects of other features. The experimental results show that the proposed model is superior to other advanced pruning methods in terms of the computation time and accuracy on CIFAR-100 and ImageNet dataset, where the accuracy is approximately 3% higher than that of similar methods with shorter computation times.
Förderung
- New Generation Information Technology Innovation Project
Zitieren
APA
Liu, B., Hu, B., Zhao, M., Peng, S., & Chang, J. (2023). Model Compression Algorithm via Reinforcement Learning and Knowledge Distillation. Mathematics, 11(22), 4589. https://doi.org/10.3390/math11224589
Vancouver
Liu B, Hu B, Zhao M, Peng S, Chang J. Model Compression Algorithm via Reinforcement Learning and Knowledge Distillation. Mathematics. 2023;11(22):4589. doi:10.3390/math11224589
BibTeX
@article{Liu2023,
author = {Liu, Botao and Hu, Bing-Bing and Zhao, Ming and Peng, Sheng-Lung and Chang, Jou-Ming},
title = {Model Compression Algorithm via Reinforcement Learning and Knowledge Distillation},
journal = {Mathematics},
year = {2023},
volume = {11},
number = {22},
pages = {4589},
doi = {10.3390/math11224589}
}
EUVIMED