From Regularization to Risk-sensitivity- and Back Again
Title : From Regularization to Risk-sensitivity- and Back Again
Authors : Erfaun Noorani and John S. Baras
Conference : 6th IFAC International Conference on Intelligent Control and Automation Sciences (ICONS2022), Invited Track, "Reinforcement Learning and Machine Learning for Control pp. 33-38 , Romania
Date: July 13 - July 15, 2022
Authors : Erfaun Noorani and John S. Baras
Conference : 6th IFAC International Conference on Intelligent Control and Automation Sciences (ICONS2022), Invited Track, "Reinforcement Learning and Machine Learning for Control pp. 33-38 , Romania
Date: July 13 - July 15, 2022
We explore (h,f)-divergence regularized RL objectives (as a generalization of KL regularized and maximum entropy objectives) through the lens of risk sensitivity and offer two iterative schemes for approaching such regularized objectives. This allows for (I) understanding of RL algorithms through the lens of risk optimization which in turn provides a more cohesive view of well-known RL algorithms and (II) introduction of theoretically well-motivated regularization terms that lead to risk-sensitive RL algorithms. We offer two iterative frameworks for using (h,f)-divergence-based Convex risk measures to facilitate further algorithmic development in risk-sensitive RL.
Download Full Paper