Skip to main content

Decentralized Reinforcement Learning: Global Decision-Making via Local Economic Transactions

Author(s): Chang, Michael; Kaushik, Sid; Weinberg, S Matthew; Griffiths, Tom; Levine, Sergey

Download
To refer to this page use: http://arks.princeton.edu/ark:/88435/pr1pk1g
Full metadata record
DC FieldValueLanguage
dc.contributor.authorChang, Michael-
dc.contributor.authorKaushik, Sid-
dc.contributor.authorWeinberg, S Matthew-
dc.contributor.authorGriffiths, Tom-
dc.contributor.authorLevine, Sergey-
dc.date.accessioned2021-10-08T19:51:14Z-
dc.date.available2021-10-08T19:51:14Z-
dc.date.issued2020en_US
dc.identifier.citationChang, Michael, Sid Kaushik, S. Matthew Weinberg, Tom Griffiths, and Sergey Levine. "Decentralized Reinforcement Learning: Global Decision-Making via Local Economic Transactions." In Proceedings of the 37th International Conference on Machine Learning 119 (2020): pp. 1437-1447.en_US
dc.identifier.issn2640-3498-
dc.identifier.urihttp://proceedings.mlr.press/v119/chang20b.html-
dc.identifier.urihttp://arks.princeton.edu/ark:/88435/pr1pk1g-
dc.description.abstractThis paper seeks to establish a framework for directing a society of simple, specialized, self-interested agents to solve what traditionally are posed as monolithic single-agent sequential decision problems. What makes it challenging to use a decentralized approach to collectively optimize a central objective is the difficulty in characterizing the equilibrium strategy profile of non-cooperative games. To overcome this challenge, we design a mechanism for defining the learning environment of each agent for which we know that the optimal solution for the global objective coincides with a Nash equilibrium strategy profile of the agents optimizing their own local objectives. The society functions as an economy of agents that learn the credit assignment process itself by buying and selling to each other the right to operate on the environment state. We derive a class of decentralized reinforcement learning algorithms that are broadly applicable not only to standard reinforcement learning but also for selecting options in semi-MDPs and dynamically composing computation graphs. Lastly, we demonstrate the potential advantages of a society’s inherent modular structure for more efficient transfer learning.en_US
dc.format.extent1437 - 1447en_US
dc.language.isoen_USen_US
dc.relation.ispartofProceedings of the 37th International Conference on Machine Learningen_US
dc.rightsFinal published version. This is an open access article.en_US
dc.titleDecentralized Reinforcement Learning: Global Decision-Making via Local Economic Transactionsen_US
dc.typeConference Articleen_US
pu.type.symplectichttp://www.symplectic.co.uk/publications/atom-terms/1.0/conference-proceedingen_US

Files in This Item:
File Description SizeFormat 
DecenReinforcementLearning.pdf1.39 MBAdobe PDFView/Download


Items in OAR@Princeton are protected by copyright, with all rights reserved, unless otherwise indicated.