A New Adaptive Aggregation Algorithm for Infinite Horizon Dynamic Programming

A New Adaptive Aggregation Algorithm for Infinite Horizon Dynamic Programming

Title : A New Adaptive Aggregation Algorithm for Infinite Horizon Dynamic Programming
Authors :
Zhang, Chang
Journal : Number: CSHCN TR 2001-5
December 31, 2001

Dynamic programming suffers the “curse of dimensionality” when it isemployed for complex control systems. State aggregation is used to solvethe problem and acceleratecomputation by looking for a sub-optimal policy. In this paper, a new method, which converges much faster thanconventional aggregated value iteration based on TD(0), is proposed for computing the valuefunctions of theaggregated system. Preliminary results show that the new method increases thespeed of convergence impressively. Aggregation introduces errorsinevitably. An adaptive aggregation scheme employing the newcomputation method isalso proposed to reduce the aggregation errors.

Download Full Paper