Skip to main content

FrameNet: Learning Local Canonical Frames of 3D Surfaces From a Single RGB Image

Author(s): Huang, Jingwei; Zhou, Yichao; Funkhouser, Thomas; Guibas, Leonidas

Download
To refer to this page use: http://arks.princeton.edu/ark:/88435/pr18541
Full metadata record
DC FieldValueLanguage
dc.contributor.authorHuang, Jingwei-
dc.contributor.authorZhou, Yichao-
dc.contributor.authorFunkhouser, Thomas-
dc.contributor.authorGuibas, Leonidas-
dc.date.accessioned2021-10-08T19:46:54Z-
dc.date.available2021-10-08T19:46:54Z-
dc.date.issued2019en_US
dc.identifier.citationHuang, Jingwei, Yichao Zhou, Thomas Funkhouser, and Leonidas J. Guibas. "FrameNet: Learning Local Canonical Frames of 3D Surfaces From a Single RGB Image." In IEEE/CVF International Conference on Computer Vision (ICCV) (2019): pp. 8637-8646. doi:10.1109/ICCV.2019.00873en_US
dc.identifier.issn1550-5499-
dc.identifier.urihttps://openaccess.thecvf.com/content_ICCV_2019/papers/Huang_FrameNet_Learning_Local_Canonical_Frames_of_3D_Surfaces_From_a_ICCV_2019_paper.pdf-
dc.identifier.urihttp://arks.princeton.edu/ark:/88435/pr18541-
dc.description.abstractIn this work, we introduce the novel problem of identifying dense canonical 3D coordinate frames from a single RGB image. We observe that each pixel in an image corresponds to a surface in the underlying 3D geometry, where a canonical frame can be identified as represented by three orthogonal axes, one along its normal direction and two in its tangent plane. We propose an algorithm to predict these axes from RGB. Our first insight is that canonical frames computed automatically with recently introduced direction field synthesis methods can provide training data for the task. Our second insight is that networks designed for surface normal prediction provide better results when trained jointly to predict canonical frames, and even better when trained to also predict 2D projections of canonical frames. We conjecture this is because projections of canonical tangent directions often align with local gradients in images, and because those directions are tightly linked to 3Dcanonical frames through projective geometry and orthogonality constraints. In our experiments, we find that our method predicts 3D canonical frames that can be used in applications ranging from surface normal estimation, feature matching, and augmented reality.en_US
dc.format.extent8637 - 8646en_US
dc.language.isoen_USen_US
dc.relation.ispartofIEEE/CVF International Conference on Computer Vision (ICCV)en_US
dc.rightsAuthor's manuscripten_US
dc.titleFrameNet: Learning Local Canonical Frames of 3D Surfaces From a Single RGB Imageen_US
dc.typeConference Articleen_US
dc.identifier.doi10.1109/ICCV.2019.00873-
dc.identifier.eissn2380-7504-
pu.type.symplectichttp://www.symplectic.co.uk/publications/atom-terms/1.0/conference-proceedingen_US

Files in This Item:
File Description SizeFormat 
FrameNetLocalCanonicalFrames3DSurfaces.pdf2.88 MBAdobe PDFView/Download


Items in OAR@Princeton are protected by copyright, with all rights reserved, unless otherwise indicated.