SIGGRAPH Asia 2018
Transactions on Graphics (TOG) Special Issue Cover

PhotoShape: Photorealistic Materials for Large-Scale Shape Collections

Keunhong Park, Konstantinos Rematas, Ali Farhadi, Steve Seitz
PDF (High Res, 87.2 MB) PDF (Low Res, 8.6 MB) arXiv Code ACM

Abstract

Existing online 3D shape repositories contain thousands of 3D models but lack photorealistic appearance. We present an approach to automatically assign high-quality, realistic appearance models to large scale 3D shape collections. The key idea is to jointly leverage three types of online data – shape collections, material collections, and photo collections, using the photos as reference to guide assignment of materials to shapes. By generating a large number of synthetic renderings, we train a convolutional neural network to classify materials in real photos, and employ 3D-2D alignment techniques to transfer materials to different parts of each shape model. Our system produces photorealistic, relightable, 3D shapes (PhotoShapes).


Fully automatic texturing of 3D shapes with rich SV-BRDF reflectance models.

Code and Data

All of the code relevant to this project is available on github.

Material (SVBRDF) Dataset

Material Classifier

Photorealistic Shape Dataset

Provided as Blender scenes. Note that these archives contain the raw output of our pipeline which means they also contain failure cases. More detailed JSON inference results may be found in the Github repository.

Important Note: Blender >=2.79 is required to render these scenes due to the use of the Principled BRDF.

Bibtex

@article{photoshape2018,
 author = {Park, Keunhong and Rematas, Konstantinos and Farhadi, Ali and Seitz, Steven M.},
 title = {PhotoShape: Photorealistic Materials for Large-Scale Shape Collections},
 journal = {ACM Trans. Graph.},
 issue_date = {November 2018},
 volume = {37},
 number = {6},
 month = nov,
 year = {2018},
 articleno = {192},
}

Acknowledgements

This work was supported by the Samsung Scholarship, the Allen Institute for Artificial Intelligence, Intel, Google, and the National Science Foundation (IIS1538618).