EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Model Compression Algorithm via Reinforcement Learning and Knowledge Distillation

Botao Liu, Bing-Bing Hu, Ming Zhao, Sheng-Lung Peng, Jou-Ming Chang

Mathematics · 09.11.2023 · 11 (22) · 4589

DOI: 10.3390/math11224589 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Mathematics, online erschienen am 09.11.2023. Verfasst von Botao Liu und 4 weiteren Autor:innen. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 39 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: New Generation Information Technology Innovation Project.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Traditional model compression techniques are dependent on handcrafted features and require domain experts, with a tradeoff between model size, speed, and accuracy. This study proposes a new approach toward resolving model compression problems. Our approach combines reinforcement-learning-based automated pruning and knowledge distillation to improve the pruning of unimportant network layers and the efficiency of the compression process. We introduce a new state quantity that controls the size of the reward and an attention mechanism that reinforces useful features and attenuates useless features to enhance the effects of other features. The experimental results show that the proposed model is superior to other advanced pruning methods in terms of the computation time and accuracy on CIFAR-100 and ImageNet dataset, where the accuracy is approximately 3% higher than that of similar methods with shorter computation times.

Förderung

  • New Generation Information Technology Innovation Project

Zitieren

APA

Liu, B., Hu, B., Zhao, M., Peng, S., & Chang, J. (2023). Model Compression Algorithm via Reinforcement Learning and Knowledge Distillation. Mathematics, 11(22), 4589. https://doi.org/10.3390/math11224589

Vancouver

Liu B, Hu B, Zhao M, Peng S, Chang J. Model Compression Algorithm via Reinforcement Learning and Knowledge Distillation. Mathematics. 2023;11(22):4589. doi:10.3390/math11224589

BibTeX
@article{Liu2023,
  author = {Liu, Botao and Hu, Bing-Bing and Zhao, Ming and Peng, Sheng-Lung and Chang, Jou-Ming},
  title = {Model Compression Algorithm via Reinforcement Learning and Knowledge Distillation},
  journal = {Mathematics},
  year = {2023},
  volume = {11},
  number = {22},
  pages = {4589},
  doi = {10.3390/math11224589}
}
Kostenlos starten →