Skip to main content

New algorithms for learning incoherent and overcomplete dictionaries

Author(s): Arora, Sanjeev; Ge, R; Moitra, A

Download
To refer to this page use: http://arks.princeton.edu/ark:/88435/pr14730
Full metadata record
DC FieldValueLanguage
dc.contributor.authorArora, Sanjeev-
dc.contributor.authorGe, R-
dc.contributor.authorMoitra, A-
dc.date.accessioned2019-08-29T17:05:07Z-
dc.date.available2019-08-29T17:05:07Z-
dc.date.issued2014en_US
dc.identifier.citationArora, S, Ge, R, Moitra, A. (2014). New algorithms for learning incoherent and overcomplete dictionaries. Journal of Machine Learning Research, 35 (779 - 806en_US
dc.identifier.urihttp://arks.princeton.edu/ark:/88435/pr14730-
dc.description.abstractIn sparse recovery we are given a matrix A ∈ Rnxm ("the dictionary") and a vector of the form AX where X is sparse, and the goal is to recover X. This is a central notion in signal processing, statistics and machine learning. But in applications such as sparse coding, edge detection, compression and super resolution, the dictionary A is unknown and has to be learned from random examples of the form Y = AX where X is drawn from an appropriate distribution - this is the dictionary learning problem. In most settings, A is over complete: it has more columns than rows. This paper presents a polynomial-time algorithm for learning overcomplete dictionaries; the only previously known algorithm with provable guarantees is the recent work of Spielman et al. (2012) who gave an algorithm for the undercomplete case, which is rarely the case in applications. Our algorithm applies to incoherent dictionaries which have been a central object of study since they were introduced in seminal work of Donoho and Huo (1999). In particular, a dictionary is μ-incoherent if each pair of columns has inner product at most μ/√n. The algorithm makes natural stochastic assumptions about the unknown sparse vector X, which can contain k ≤ cmin(√n/μlogn, m1/2-η) non-zero entries (for any η > 0). This is close to the best k allowable by the best sparse recovery algorithms even if one knows the dictionary A exactly. Moreover, both the running time and sample complexity depend on log 1/∈, where e is the target accuracy, and so our algorithms converge very quickly to the true dictionary. Our algorithm can also tolerate substantial amounts of noise provided it is incoherent with respect to the dictionary (e.g., Gaussian). In the noisy setting, our running time and sample complexity depend polynomially on 1/∈, and this is necessary.en_US
dc.format.extent779 - 806en_US
dc.language.isoen_USen_US
dc.relation.ispartofJournal of Machine Learning Researchen_US
dc.rightsAuthor's manuscripten_US
dc.titleNew algorithms for learning incoherent and overcomplete dictionariesen_US
dc.typeJournal Articleen_US
pu.type.symplectichttp://www.symplectic.co.uk/publications/atom-terms/1.0/conference-proceedingen_US

Files in This Item:
File Description SizeFormat 
New Algorithms for Learning Incoherent and Overcomplete Dictionaries.pdf272.39 kBAdobe PDFView/Download


Items in OAR@Princeton are protected by copyright, with all rights reserved, unless otherwise indicated.