Skip to main content

UVeQFed: Universal Vector Quantization for Federated Learning

Author(s): Shlezinger, Nir; Chen, Mingzhe; Eldar, Yonina C; Poor, H Vincent; Cui, Shuguang

To refer to this page use:
Abstract: Traditional deep learning models are trained at a centralized server using data samples collected from users. Such data samples often include private information, which the users may not be willing to share. Federated learning (FL) is an emerging approach to train such learning models without requiring the users to share their data. FL consists of an iterative procedure, where in each iteration the users train a copy of the learning model locally. The server then collects the individual updates and aggregates them into a global model. A major challenge that arises in this method is the need of each user to repeatedly transmit its learned model over the throughput limited uplink channel. In this work, we tackle this challenge using tools from quantization theory. In particular, we identify the unique characteristics associated with conveying trained models over rate-constrained channels, and propose a suitable quantization scheme for such settings, referred to as universal vector quantization for FL (UVeQFed). We show that combining universal vector quantization methods with FL yields a decentralized training system in which the compression of the trained models induces only a minimum distortion. We then theoretically analyze the distortion, showing that it vanishes as the number of users grows. We also characterize how models trained with conventional federated averaging combined with UVeQFed converge to the model which minimizes the loss function. Our numerical results demonstrate the gains of UVeQFed over previously proposed methods in terms of both distortion induced in quantization and accuracy of the resulting aggregated model.
Publication Date: 23-Dec-2020
Citation: Shlezinger, Nir, Chen, Mingzhe, Eldar, Yonina C, Poor, H Vincent, Cui, Shuguang. (2021). UVeQFed: Universal Vector Quantization for Federated Learning. IEEE Transactions on Signal Processing, 69 (500 - 514. doi:10.1109/tsp.2020.3046971
DOI: doi:10.1109/tsp.2020.3046971
ISSN: 1053-587X
EISSN: 1941-0476
Pages: 500 - 514
Type of Material: Journal Article
Journal/Proceeding Title: IEEE Transactions on Signal Processing
Version: Author's manuscript

Items in OAR@Princeton are protected by copyright, with all rights reserved, unless otherwise indicated.