Skip to main content

PlaneMatch: Patch Coplanarity Prediction for Robust RGB-D Reconstruction

Author(s): Shi, Yifei; Xu, Kai; Niebner, Matthias; Rusinkiewicz, Szymon; Funkhouser, Thomas

To refer to this page use:
Abstract: We introduce a novel RGB-D patch descriptor designed for detecting coplanar surfaces in SLAM reconstruction. The core of our method is a deep convolutional neural network that takes in RGB, depth, and normal information of a planar patch in an image and outputs a descriptor that can be used to find coplanar patches from other images. We train the network on 10 million triplets of coplanar and non-coplanar patches, and evaluate on a new coplanarity benchmark created from commodity RGB-D scans. Experiments show that our learned descriptor outperforms alternatives extended for this new task by a significant margin. In addition, we demonstrate the benefits of coplanarity matching in a robust RGBD reconstruction formulation. We find that coplanarity constraints detected with our method are sufficient to get reconstruction results comparable to state-of-the-art frameworks on most scenes, but outperform other methods on established benchmarks when combined with traditional keypoint matching.
Publication Date: 2018
Citation: Shi, Yifei, Kai Xu, Matthias Nießner, Szymon Rusinkiewicz, and Thomas Funkhouser. "PlaneMatch: Patch Coplanarity Prediction for Robust RGB-D Reconstruction." In European Conference on Computer Vision (2018): pp. 767-784. doi:10.1007/978-3-030-01237-3_46
DOI: 10.1007/978-3-030-01237-3_46
ISSN: 0302-9743
Pages: 767 - 784
Type of Material: Conference Article
Journal/Proceeding Title: European Conference on Computer Vision
Version: Author's manuscript

Items in OAR@Princeton are protected by copyright, with all rights reserved, unless otherwise indicated.