CIS ALUMNI DISTINGUISHED LECTURE: GUOYU LU
SINGLE IMAGE DEPTH ESTIMATION AND 3D RECONSTRUCTION
UNDER MULTI-SENSING MODALITIES
Abstract: Recovering a 3D shape representation from one single image input has been attempted in recent years, as multiple images from different perspectives or 3D CAD models are not always available in real applications. We present a novel shape-from-silhouette method based on just a single image, which is an end-to-end learning framework relying on view synthesis and shape-from-silhouette methodology to reconstruct a 3D shape. Compared with normal images, high-resolution LiDARs are precise in depth estimation, but usually too expensive. Though single beam LIDAR enjoys the benefits of low cost, one beam depth sensing is not usually sufficient to perceive the surrounding environment in many scenarios. I present a deep neural network based framework to replicate similar or even higher performance as costly LiDARs with the designated self-supervised network and a low-cost single-beam LIDAR. To overcome the deficiency that RGB images are not suitable for dark and night environment with limited lighting resource, I present a framework to estimate the scene depth directly from a single thermal image that can still observe the scene in the low lighting condition. With the proposed approach, an accurate depth map can be predicted without any prior knowledge under various illumination conditions.
Bio: Dr. Guoyu Lu is an Assistant Professor at the Chester F. Carlson Center for Imaging Science of Rochester Institute of Technology (RIT). Prior to joining RIT, he was a research scientist on autonomous driving at Ford Research and computer vision engineer at ESPN Advanced Technology Group. Dr. Lu finished his PhD and MS in Computer Science at the University of Delaware with Prof. Chandra Kambhamettu. Before coming to UD, he was in European Master in Informatics (EuMI) Erasmus Mundus program. Dr. Lu obtained his Master degree in Computer Science at University of Trento and Master degree in Media Informatics at RWTH Aachen University. He was a visiting scholar at the Auckland University of Technology. He was a research intern at Siemens Corporate Research in Princeton and Bosch Research and Technology Center in Palo Alto. He finished his Bachelor degree in Software Engineering at Nanjing University of Posts & Telecommunications, with a minor in Business Administration and Management. Dr. Lu has broad research interests spreading across computer vision, machine learning, multimedia, biomedical imaging, and robotics. He has published over 40 papers on leading international journals and conferences and one US patent. He is a regular reviewer for multiple top journals (TPAMI, TIP, TMC, TCSVT, TMM, CVIU etc). He has been the vision and perception chair of 4th Ford Global Control Conference. He is the organizer and program chair of International Workshop on Visual Odometry and Computer Vision Applications Based on Location Clues in conjunction with CVPR on 2017, 2018, 2019, and 2020. He is a Guest Editor of the Springer Journal of Multimedia Tools and Applications. He has led his team to win 2020 SICK Inc. LiDAR challenge.
Friday, December 4, 2020 at 3:00pmVirtual Event