Home Abstract Dataset Paper Acknowledgements ☰

Abstract

Much recent progress has been made in reconstructing 3D object shape from an image, i.e. single view 3D reconstruction. However, due to the difficulty of collecting large datasets in the wild with 3D ground truth, it remains a significant challenge for methods to generalize across domain, viewpoint, and class. Current methods also tend to produce averaged "nearest-neighbor" memorized shapes instead of genuinely understanding the image, thus eliminating important details. To address this we propose REFINE, a postprocessing mesh refinement step easily integratable into the pipeline of any black-box method in the literature. At test time, REFINE optimizes a network per mesh instance, to encourage consistency between the mesh and the given object view. This, with a novel combination of losses addressing degenerate solutions, reduces domain gap and restores details to achieve state of the art performance. A new hierarchical multiview, multidomain image dataset with 3D meshes called 3D-ODDS is also proposed as a uniquely challenging benchmark. We believe that the novel REFINE paradigm and 3D-ODDS are important steps towards truly robust, accurate 3D reconstructions.

3D Object Domain Dataset Suite (3D-ODDS)


Download (Pass: cvpr2659) Download Mirror


Paper

Black-Box Test-Time Shape REFINEment for Single View 3D Reconstruction

Brandon Leung, Chih-Hui Ho, Nuno Vasconcelos
Paper Supplementary material Poster Code 
@InProceedings{Leung_2022_CVPR,
		author = {Leung, Brandon and Ho, Chih-Hui and Vasconcelos, Nuno},
		title = {Black-Box Test-Time Shape REFINEment for Single View 3D Reconstruction},
		booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
		month = {June},
		year = {2022}
		}

Acknowledgements

This work was partially funded by NSF award IIS-1924937, award IIS-2041009, and a gift from Amazon. It is supported by the National Science Foundation Graduate Research Fellowship. We also thank the use of the Nautilus platform.