Published at: 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), Atlanta, GA, USA, 2020, pp. 575-576
Abstract:
In this paper, we present a method that estimates the real-world lighting condition from a single RGB image of an indoor scene, with information of support plane provided by commercial Augmented Reality (AR) frameworks (e.g., ARCore, ARKit, etc.). First, a Deep Neural Network (DNN) is used to segment the foreground. We only focus on the foreground objects to reduce computation complexity. Then we introduce Differentiable Screen-Space Rendering (DSSR), a novel approach for estimating the normal and lighting condition jointly. We recover the most plausible lighting condition using spherical harmonics. Our approach provides plausible results and considerably enhances the visual realism in AR applications.
Lighting Estimation via Differentiable Screen-Space Rendering
Read More: https://ieeexplore.ieee.org/document/9090444