EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

COMPUTE-EFFICIENT SCALING OF LARGE SEQUENTIAL RECOMMENDATION MODELS UNDER DYNAMIC USER BEHAVIOR

AoWei Shen

World Journal of Information Technology · 26.09.2023 · 1 (1) · 62-71

DOI: 10.61784/wjit4126 ↗

Kurz gefasst

Fachartikel in der Zeitschrift World Journal of Information Technology, online erschienen am 26.09.2023. Verfasst von AoWei Shen. Die Arbeit wurde bisher 1-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Transformer-based sequential recommenders are increasingly scaled up and retrained as interaction logs grow, yet user behavior drifts between retraining cycles, and repeated from-scratch training of large models consumes compute that rarely yields proportional accuracy. This paper studies how model scale, update strategy, and training compute interact under a streaming protocol in which a model trained on earlier blocks of a chronologically ordered log serves the next block. On MovieLens-1M and Amazon Grocery, widening a SASRec-style model from 6.6K to 397K non-embedding parameters raises training compute by 4.4-13.3x relative to width 32 without a consistent accuracy gain, and models that are not refreshed lose 36% and 82% of their NDCG@10, respectively. Building on these observations, we propose SPROUT, a compute-aware update framework that warm-starts each update from a recency-aware replay stream, expands capacity with an exactly function-preserving width-doubling operator for pre-LayerNorm Transformers, and accepts growth only when a paired validation test shows a gain larger than one standard error. Over four update cycles, SPROUT stays within 1.3% of the best periodic-retraining accuracy on MovieLens-1M with 44% less cumulative compute than the retrained small model and 93% less than the retrained large model, and it achieves the highest accuracy on Amazon Grocery, 5.0% above the best retraining baseline, with 47% less compute than retraining the large model. Its growth-free configuration offers the most economical operating point, reaching 99.1% and 102.1% of the retraining accuracy with 33% and 51% of its compute.

Zitieren

APA

Shen, A. (2023). COMPUTE-EFFICIENT SCALING OF LARGE SEQUENTIAL RECOMMENDATION MODELS UNDER DYNAMIC USER BEHAVIOR. World Journal of Information Technology, 1(1), 62-71. https://doi.org/10.61784/wjit4126

Vancouver

Shen A. COMPUTE-EFFICIENT SCALING OF LARGE SEQUENTIAL RECOMMENDATION MODELS UNDER DYNAMIC USER BEHAVIOR. World Journal of Information Technology. 2023;1(1):62-71. doi:10.61784/wjit4126

BibTeX
@article{Shen2023,
  author = {Shen, AoWei},
  title = {COMPUTE-EFFICIENT SCALING OF LARGE SEQUENTIAL RECOMMENDATION MODELS UNDER DYNAMIC USER BEHAVIOR},
  journal = {World Journal of Information Technology},
  year = {2023},
  volume = {1},
  number = {1},
  pages = {62--71},
  doi = {10.61784/wjit4126}
}
Kostenlos starten →