RS-DRL: Managing Uncertainty in Self-Adaptive Systems Based on a Novel Continuous Deep Reinforcement Learning Method
ACM Transactions on Autonomous and Adaptive Systems · 30.09.2026 · 21 (3) · 1-30
DOI: 10.1145/3779424 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Autonomous and Adaptive Systems, online erschienen am 10.06.2026. Verfasst von Alireza Kavianifar und 1 weiteren Autor:innen. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 53 Literaturangaben.
Abstract (Originalsprache)
Designing software systems within uncertain and evolving operational contexts is challenging, as pervasive uncertainty can compromise system objectives. Self-adaptive systems address this by dynamically adjusting configurations when objectives are at risk. However, exhaustive exploration of large configuration spaces is time-consuming and computationally expensive. Existing machine learning, search-based, and reinforcement learning approaches often degrade under changing conditions. To overcome these limitations, this article introduces an architecture based on the MAPE-K loop integrated with a Deep Reinforcement Learning (DRL) module enhanced by a novel reward-shaping mechanism. The proposed Reward-Shaped Deep Reinforcement Learning (RS-DRL) method reshapes rewards during experience replay, improving generalization, convergence, and adaptability across dynamic environments. Experiments on IoT case studies (DeltaIoTv1/v2) show that RS-DRL achieves an asymptotic multi-objective reward of \(0.9905\pm 0.0268\) compared to \(0.4559\pm 0.2233\) for the \(\epsilon\) -greedy DQN approach, and near-optimal per-objective asymptotic results (e.g., packet loss \(0.9850\pm 0.0071\) , latency \(0.9964\pm 0.0107\) ). Under the TT goal, RS-DRL reduces packet loss by up to \(22.23\%\) and latency by up to \(39.07\%\) compared to the best competing method ( DLASER+ ), while also consistently outperforming the exhaustive-search Reference baseline that analyzes the full adaptation space. All comparisons are based on 30 independent runs, with 25 of 28 baseline comparisons statistically significant (many with p < 0.001). These results demonstrate that RS-DRL offers a robust, efficient, and adaptive optimization strategy for self-adaptive systems operating under uncertainty and dynamic conditions.
Zitieren
APA
Kavianifar, A., & Jalili, S. (2026). RS-DRL: Managing Uncertainty in Self-Adaptive Systems Based on a Novel Continuous Deep Reinforcement Learning Method. ACM Transactions on Autonomous and Adaptive Systems, 21(3), 1-30. https://doi.org/10.1145/3779424
Vancouver
Kavianifar A, Jalili S. RS-DRL: Managing Uncertainty in Self-Adaptive Systems Based on a Novel Continuous Deep Reinforcement Learning Method. ACM Transactions on Autonomous and Adaptive Systems. 2026;21(3):1-30. doi:10.1145/3779424
BibTeX
@article{Kavianifar2026,
author = {Kavianifar, Alireza and Jalili, Saeed},
title = {RS-DRL: Managing Uncertainty in Self-Adaptive Systems Based on a Novel Continuous Deep Reinforcement Learning Method},
journal = {ACM Transactions on Autonomous and Adaptive Systems},
year = {2026},
volume = {21},
number = {3},
pages = {1--30},
doi = {10.1145/3779424}
}
EUVIMED