Skip to main content

Local smoothness in variance reduced optimization

Author(s): Vainsencher, Daniel; Liu, Han; Zhang, Tong

Download
To refer to this page use: http://arks.princeton.edu/ark:/88435/pr14z3z
Full metadata record
DC FieldValueLanguage
dc.contributor.authorVainsencher, Daniel-
dc.contributor.authorLiu, Han-
dc.contributor.authorZhang, Tong-
dc.date.accessioned2020-04-07T18:54:09Z-
dc.date.available2020-04-07T18:54:09Z-
dc.date.issued2015-01-01en_US
dc.identifier.citationVainsencher, D, Liu, H, Zhang, T. (2015). Local smoothness in variance reduced optimization. Advances in Neural Information Processing Systems, 2015-January (2179 - 2187).en_US
dc.identifier.issn1049-5258-
dc.identifier.urihttps://papers.nips.cc/paper/5913-local-smoothness-in-variance-reduced-optimization.pdf-
dc.identifier.urihttp://arks.princeton.edu/ark:/88435/pr14z3z-
dc.description.abstractWe propose a family of non-uniform sampling strategies to provably speed up a class of stochastic optimization algorithms with linear convergence including Stochastic Variance Reduced Gradient (SVRG) and Stochastic Dual Coordinate Ascent (SDCA). For a large family of penalized empirical risk minimization problems, our methods exploit data dependent local smoothness of the loss functions near the optimum, while maintaining convergence guarantees. Our bounds are the first to quantify the advantage gained from local smoothness which are significant for some problems significantly better. Empirically, we provide thorough numerical results to back up our theory. Additionally we present algorithms exploiting local smoothness in more aggressive ways, which perform even better in practice.en_US
dc.format.extent2179 - 2187en_US
dc.language.isoen_USen_US
dc.relation.ispartofAdvances in Neural Information Processing Systemsen_US
dc.rightsFinal published version. Article is made available in OAR by the publisher's permission or policy.en_US
dc.titleLocal smoothness in variance reduced optimizationen_US
dc.typeJournal Articleen_US
pu.type.symplectichttp://www.symplectic.co.uk/publications/atom-terms/1.0/conference-proceedingen_US

Files in This Item:
File Description SizeFormat 
LocalSmoothnmessInOptimization.pdf1.46 MBAdobe PDFView/Download


Items in OAR@Princeton are protected by copyright, with all rights reserved, unless otherwise indicated.