Embracing Risk in Reinforcement Learning: The Connection between Risk-sensitive Exponential and Distributionally Robust Criteria
Authors : Erfaun Noorani and John S. Baras
Conference : 2022 American Control Conference (ACC2022) pp. 2703-2708 , Atlanta, GA
Date: June 08 - June 10, 2022
We explore the relation between the risk-sensitive exponential (exponential of total cost) and Distributionally Robust Reinforcement Learning objectives, and in doing so, we unify some of the popular Reinforcement Learning algorithms. Such equivalence (I) allows to understand a number of well-known Reinforcement Learning algorithms from a risk minimization perspective and (II) establishes the r robustness properties of risk-sensitive exponential objective in the Reinforcement Learning context, which in turn provides a theoretical justification for the robust performance of risksensitive
Reinforcement Learning algorithms in the literature. The robustness of exponential criteria motivates risk-sensitizing current risk-neutral Reinforcement Learning algorithms using such criteria.