Facebook has revealed new hyperrealistic virtual avatars they are creating for virtual reality environments developing previous work which the company has named ‘Codec Avatars’. Check out the video below to learn more about the virtual reality avatars and how they are created to provide a realistic image of a user’s face using just three standard cameras located inside the virtual reality headset. The cameras capture the user’s eyes and mouth, and using this data the system is able to represent the nuances of a specific individual’s complex face gestures more accurately than previous systems.
The hyper-realistic virtual avatars were showcased at this year’s SIGGRAPH 2019 technology show and a paper has been published by Facebook detailing the technology which is available via the link below.
“A key promise of Virtual Reality (VR) is the possibility of remote social interaction that is more immersive than any prior telecommunication media. However, existing social VR experiences are mediated by inauthentic digital representations of the user (i.e., stylized avatars). These stylized representa- tions have limited the adoption of social VR applications in precisely those cases where immersion is most necessary (e.g., professional interactions and intimate conversations).”
“In this work, we present a bidirectional system that can animate avatar heads of both users’ full likeness using consumer-friendly headset mounted cameras (HMC). There are two main challenges in doing this: unaccommodating camera views and the image-to-avatar domain gap. We address both challenges by leveraging constraints imposed by multiview geometry to establish precise image-to-avatar correspondence, which are then used to learn an end-to-end model for real-time tracking. We present designs for a training HMC, aimed at data-collection and model building, and a tracking HMC for use during interactions in VR.”