Skip to main content

Communication-computation efficient gradient coding

Author(s): Ye, M; Abbe, Emmanuel

Download
To refer to this page use: http://arks.princeton.edu/ark:/88435/pr1jp2n
Full metadata record
DC FieldValueLanguage
dc.contributor.authorYe, M-
dc.contributor.authorAbbe, Emmanuel-
dc.date.accessioned2021-10-08T20:16:08Z-
dc.date.available2021-10-08T20:16:08Z-
dc.date.issued2018en_US
dc.identifier.citationYe, M, Abbe, E. (2018). Communication-computation efficient gradient coding. 12 (9716p - 9716pen_US
dc.identifier.urihttp://arks.princeton.edu/ark:/88435/pr1jp2n-
dc.description.abstractThis paper develops coding techniques to reduce the running time of distributed learning tasks. It characterizes the fundamental tradeoff to compute gradients in terms of three parameters: computation load, straggler tolerance and communication cost. It further gives an explicit coding scheme that achieves the optimal tradeoff based on recursive polynomial constructions, coding both across data subsets and vector components. As a result, the proposed scheme allows to minimize the running time for gradient computations. Implementations are made on Amazon EC2 clusters using Python with mpi4py package. Results show that the proposed scheme maintains the same generalization error while reducing the running time by 32% compared to uncoded schemes and 23% compared to prior coded schemes focusing only on stragglers (Tandon et al., ICML 2017).en_US
dc.format.extent9716p - 9716pen_US
dc.language.isoen_USen_US
dc.relation.ispartof35th International Conference on Machine Learning, ICML 2018en_US
dc.rightsAuthor's manuscripten_US
dc.titleCommunication-computation efficient gradient codingen_US
dc.typeConference Articleen_US
pu.type.symplectichttp://www.symplectic.co.uk/publications/atom-terms/1.0/conference-proceedingen_US

Files in This Item:
File Description SizeFormat 
Communication-computation efficient gradient coding.pdf414.41 kBAdobe PDFView/Download


Items in OAR@Princeton are protected by copyright, with all rights reserved, unless otherwise indicated.