A Tutorial on 3D Deep Learning
in conjunction with CVPR2017
3D understanding has been attracting increasing attention of computer vision and graphics researchers recently. It is particularly relevant due to its importance for many applications such as self-driving cars, autonomous robots, virtual reality, and augmented reality. Behind the wide spectrum of applications lies the fundamental techniques in analyzing 3D data.
This tutorial covers deep learning algorithms that analyze or synthesize 3D data. Different from 2D images that have a dominant representation as pixel arrays, 3D data possesses multiple popular representations, such as point cloud, mesh, volumetric field, multi-view images and parametric models, each fitting their own application scenarios. From the research perspective, each type of data format has its own properties that pose challenges to deep architecture design while also provide the opportunity for novel and efficient solutions.
In this course, we will introduce recent major advance of deep learning on each 3D representation type (up to July, 2017). We systematically introduce topics such as the characteristics of representation types, encoding of 3D representations as neural network input and output, and keys in network structure design. Throughout the talk we make comparisons among different representations and discuss the application scenarios of each. In this course, we will introduce both cutting-edge techniques and open problems in this emerging new field.