Skip to main content

A la carte embedding: Cheap but effective induction of semantic feature vectors

Author(s): Khodak, M; Saunshi, N; Liang, Y; Ma, T; Stewart, Brandon; et al

To refer to this page use:
Abstract: Motivations like domain adaptation, transfer learning, and feature learning have fueled interest in inducing embeddings for rare or unseen words, n-grams, synsets, and other textual features. This paper introduces à la carte embedding, a simple and general alternative to the usual word2vec-based approaches for building such representations that is based upon recent theoretical results for GloVe-like embeddings. Our method relies mainly on a linear transformation that is efficiently learnable using pretrained word vectors and linear regression. This transform is applicable “on the fly” in the future when a new text feature or rare word is encountered, even if only a single usage example is available. We introduce a new dataset showing how the à la carte method requires fewer examples of words in context to learn high-quality embeddings and we obtain state-of-the-art results on a nonce task and some unsupervised document classification tasks.
Publication Date: 2018
Citation: Khodak, M, Saunshi, N, Liang, Y, Ma, T, Stewart, B, Arora, S. (2018). A la carte embedding: Cheap but effective induction of semantic feature vectors. 1 (12 - 22
Pages: 12 - 22
Type of Material: Conference Article
Journal/Proceeding Title: ACL 2018 - 56th Annual Meeting of the Association for Computational Linguistics,
Version: Author's manuscript

Items in OAR@Princeton are protected by copyright, with all rights reserved, unless otherwise indicated.