Skip to main content

Explaining Landscape Connectivity of Low-cost Solutions for Multilayer Nets

Author(s): Kuditipudi, Rohith; Wang, Xiang; Lee, Holden; Zhang, Yi; Li, Zhiyuan; et al

Download
To refer to this page use: http://arks.princeton.edu/ark:/88435/pr12z6h
Full metadata record
DC FieldValueLanguage
dc.contributor.authorKuditipudi, Rohith-
dc.contributor.authorWang, Xiang-
dc.contributor.authorLee, Holden-
dc.contributor.authorZhang, Yi-
dc.contributor.authorLi, Zhiyuan-
dc.contributor.authorHu, Wei-
dc.contributor.authorGe, Rong-
dc.contributor.authorArora, Sanjeev-
dc.date.accessioned2021-10-08T19:50:57Z-
dc.date.available2021-10-08T19:50:57Z-
dc.date.issued2019en_US
dc.identifier.citationKuditipudi, Rohith, Xiang Wang, Holden Lee, Yi Zhang, Zhiyuan Li, Wei Hu, Rong Ge, and Sanjeev Arora. "Explaining Landscape Connectivity of Low-cost Solutions for Multilayer Nets." Advances in Neural Information Processing Systems 32 (2019).en_US
dc.identifier.issn1049-5258-
dc.identifier.urihttps://papers.neurips.cc/paper/2019/file/46a4378f835dc8040c8057beb6a2da52-Paper.pdf-
dc.identifier.urihttp://arks.princeton.edu/ark:/88435/pr12z6h-
dc.description.abstractMode connectivity is a surprising phenomenon in the loss landscape of deep nets. Optima---at least those discovered by gradient-based optimization---turn out to be connected by simple paths on which the loss function is almost constant. Often, these paths can be chosen to be piece-wise linear, with as few as two segments. We give mathematical explanations for this phenomenon, assuming generic properties (such as dropout stability and noise stability) of well-trained deep nets, which have previously been identified as part of understanding the generalization properties of deep nets. Our explanation holds for realistic multilayer nets, and experiments are presented to verify the theory.en_US
dc.language.isoen_USen_US
dc.relation.ispartofAdvances in Neural Information Processing Systemsen_US
dc.rightsFinal published version. Article is made available in OAR by the publisher's permission or policy.en_US
dc.titleExplaining Landscape Connectivity of Low-cost Solutions for Multilayer Netsen_US
dc.typeConference Articleen_US
pu.type.symplectichttp://www.symplectic.co.uk/publications/atom-terms/1.0/conference-proceedingen_US

Files in This Item:
File Description SizeFormat 
LowCostMultilayerNets.pdf671.4 kBAdobe PDFView/Download


Items in OAR@Princeton are protected by copyright, with all rights reserved, unless otherwise indicated.