Skip to main content

SUN3D: A Database of Big Spaces Reconstructed Using SfM and Object Labels

Author(s): Xiao, Jianxiong; Owens, Andrew; Torralba, Antonio

To refer to this page use:
Abstract: Existing scene understanding datasets contain only a limited set of views of a place, and they lack representations of complete 3D spaces. In this paper, we introduce SUN3D, a large-scale RGB-D video database with camera pose and object labels, capturing the full 3D extent of many places. The tasks that go into constructing such a dataset are difficult in isolation -- hand-labeling videos is painstaking, and structure from motion (SfM) is unreliable for large spaces. But if we combine them together, we make the dataset construction task much easier. First, we introduce an intuitive labeling tool that uses a partial reconstruction to propagate labels from one frame to another. Then we use the object labels to fix errors in the reconstruction. For this, we introduce a generalization of bundle adjustment that incorporates object-to-object correspondences. This algorithm works by constraining points for the same object from different frames to lie inside a fixed-size bounding box, parameterized by its rotation and translation. The SUN3D database, the source code for the generalized bundle adjustment, and the web-based 3D annotation tool are all available at
Publication Date: 2013
Citation: Xiao, Jianxiong, Andrew Owens, and Antonio Torralba. "SUN3D: A Database of Big Spaces Reconstructed Using SfM and Object Labels." In IEEE International Conference on Computer Vision (2013): pp. 1625-1632. doi:10.1109/ICCV.2013.458
DOI: 10.1109/ICCV.2013.458
ISSN: 1550-5499
EISSN: 2380-7504
Pages: 1625 - 1632
Type of Material: Conference Article
Journal/Proceeding Title: IEEE International Conference on Computer Vision
Version: Author's manuscript

Items in OAR@Princeton are protected by copyright, with all rights reserved, unless otherwise indicated.