Skip to main content

CDPAM: Contrastive Learning for Perceptual Audio Similarity

Author(s): Manocha, Pranay; Jin, Zeyu; Zhang, Richard; Finkelstein, Adam

To refer to this page use:
Abstract: Many speech processing methods based on deep learning require an automatic and differentiable audio metric for the loss function. The DPAM approach of Manocha et al. [1] learns a full-reference metric trained directly on human judgments, and thus correlates well with human perception. However, it requires a large number of human annotations and does not generalize well outside the range of perturbations on which it was trained. This paper introduces CDPAM –a metric that builds on and advances DPAM. The primary improvement is to combine contrastive learning and multi-dimensional representations to build robust models from limited data. In addition, we collect human judgments on triplet comparisons to improve generalization to a broader range of audio perturbations. CDPAM correlates well with human responses across nine varied datasets. We also show that adding this metric to existing speech synthesis and enhancement methods yields significant improvement, as measured by objective and subjective tests.
Publication Date: 2021
Citation: Manocha, Pranay, Zeyu Jin, Richard Zhang, and Adam Finkelstein. "CDPAM: Contrastive learning for perceptual audio similarity." In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 196-200. 2021. doi:10.1109/ICASSP39728.2021.9413711
DOI: 10.1109/ICASSP39728.2021.9413711
ISSN: 1520-6149
EISSN: 2379-190X
Pages: 196 - 200
Type of Material: Conference Article
Journal/Proceeding Title: ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Version: Author's manuscript

Items in OAR@Princeton are protected by copyright, with all rights reserved, unless otherwise indicated.