Seminario - Towards Photorealistic Volumetric Capture of Humans

Speaker: Sean Fanello, Google USA

Date: 17 Dec
Time: 15:00
Aula: 22
Dipartimento di Matematica e Informatica, Università degli Studi di Catania
Viale A. Doria 6

Title: Towards Photorealistic Volumetric Capture of Humans

Volumetric (4D) performance capture is fundamental for AR/VR content generation. Despite the efforts in the research community, we are still far to achieve truly photorealistic reconstructions of humans under unconstrained conditions. The problem is indeed challenging due to multiple factors such as: low quality depth sensors, occlusions, errors in the reconstruction, fast motion and segmentation failures. Designing a volumetric capture involves the perfect synergy among hardware, software and algorithms. In this talk I will detail ins and outs of building such a system from the ground up with particular emphasis on depth sensors and algorithms, non-rigid tracking and reconstruction of humans. I will show multiple examples of what this technology enables, I will discuss the remaining challenges and how machine learning can overcome most of them. Finally, I will introduce recent trends in neural rendering that aim at replacing traditional graphics pipelines with deep learning solutions for volumetric capture problems.

Sean Fanello is a Research Scientist at Google. His research focuses on 3D capture technologies that can bridge the gap between real and virtual world. Previously he was Senior Scientist at perceptiveIO where he developed machine learning techniques for 3D capture and perception. He spent 3 years at Microsoft Research working on Hololens, and Holoportation, the first real-time 3D telepresence system for augmented and virtual reality (shown at TED 2016). His work is published at top tier conferences including CVPR, ICCV, JMLR, SIGGRAPH, SIGGRAPH ASIA, UIST, ICRA, IROS and CHI.  He obtained his PhD in Robotics at the Italian Institute of Technology and University of Genoa in 2014.