A Tutorial on 3D Deep Learning

in conjunction with CVPR2017


  • Ballroom C (4th floor) of convention center

Course description:

3D understanding has been attracting increasing attention of computer vision and graphics researchers recently. It is particularly relevant due to its importance for many applications such as self-driving cars, autonomous robots, virtual reality, and augmented reality. Behind the wide spectrum of applications lies the fundamental techniques in analyzing 3D data.

This tutorial covers deep learning algorithms that analyze or synthesize 3D data. Different from 2D images that have a dominant representation as pixel arrays, 3D data possesses multiple popular representations, such as point cloud, mesh, volumetric field, multi-view images and parametric models, each fitting their own application scenarios. From the research perspective, each type of data format has its own properties that pose challenges to deep architecture design while also provide the opportunity for novel and efficient solutions.

In this course, we will introduce recent major advance of deep learning on each 3D representation type (up to July, 2017). We systematically introduce topics such as the characteristics of representation types, encoding of 3D representations as neural network input and output, and keys in network structure design. Throughout the talk we make comparisons among different representations and discuss the application scenarios of each. In this course, we will introduce both cutting-edge techniques and open problems in this emerging new field.


1:30PM-1:40PM Opening remark and overview of 3D deep learning PDF Hao Su
1:35PM-2:45PM Deep learning on regular 3D data formats (Multiview CNN and 3D CNN) PDF Evangelos Kalogerakis, Jimei Yang
2:45PM-3:00PM Break
3:00PM-4:15PM Deep learning on point cloud and other 3D forms PDF Hao Su, Charles Qi, Leonidas Guibas
4:15PM-4:30PM Break
4:30PM-5:45PM Intrinsic deep learning on manifolds (Spectral CNNs, Geodesic CNNs, Anisotropic CNNs, Mixture Model Networks, Embedding-based techniques) PDF Michael Bronstein


UC San Diego Leonidas Guibas
Evangelos Kalogerakis
Michael Bronstein
Università della Svizzera Italiana
Qixing Huang
UT Austin
Charles Qi
Jimei Yang
Adobe Research