Skip to main content

A New Optimal Stepsize for Approximate Dynamic Programming

Author(s): Ryzhov, Ilya O; Frazier, Peter I; Powell, Warren B

To refer to this page use:
Abstract: Approximate dynamic programming (ADP) has proven itself in a wide range of applications spanning large-scale transportation problems, health care, revenue management, and energy systems. The design of effective ADP algorithms has many dimensions, but one crucial factor is the stepsize rule used to update a value function approximation. Many operations research applications are computationally intensive, and it is important to obtain good results quickly. Furthermore, the most popular stepsize formulas use tunable parameters and can produce very poor results if tuned improperly. We derive a new stepsize rule that optimizes the prediction error in order to improve the short-term performance of an ADP algorithm. With only one, relatively insensitive tunable parameter, the new rule adapts to the level of noise in the problem and produces faster convergence in numerical experiments.
Publication Date: Mar-2015
Citation: Ryzhov, Ilya O, Frazier, Peter I, Powell, Warren B. (2015). A New Optimal Stepsize for Approximate Dynamic Programming. IEEE Transactions on Automatic Control, 60 (3), 743 - 758. doi:10.1109/TAC.2014.2357134
DOI: doi:10.1109/TAC.2014.2357134
ISSN: 0018-9286
EISSN: 1558-2523
Pages: 743 - 758
Type of Material: Journal Article
Journal/Proceeding Title: IEEE Transactions on Automatic Control
Version: Author's manuscript

Items in OAR@Princeton are protected by copyright, with all rights reserved, unless otherwise indicated.