Facebook Working on Tech to Project Eyes Onto VR Headset

Watching a video on a virtual reality device is a dramatic but sell-isolating experience. Facebook is now trying to make it less solitary. Its research lab has devised a way to project the VR headset wearer’s eyes onto the 3D screen in the front of the device so that they are able to see people and things while also being in a virtual world. The “reverse passthrough” concept essentially makes it possible for people to make eye contact with VR headset wearers. Although Facebook said that the project is experimental, it looks quite terrifying to see puffed-up eyes displayed on the 3D screen of the device.

Facebook said the idea is to ensure “social co-presence” among people who may or may not be involved in the same virtual or real experience. It added that researchers have been working for almost two years to evolve the technology after Facebook Research Lab Chief Scientist Michael Abrash in 2019 termed the initial idea “goofy”.

Their work has resulted in the device now “featuring purpose-built optics, electronics, software, and a range of supporting technologies to capture and depict more realistic 3D faces”, Facebook’s Research Lab said in a blogpost.

When the VR headset wearer decides to see the real world, the display creates the illusion that makes the other person believe they are looking through the thick lenses and seeing a pair of eyes. If the wearer turns back to full VR, the display goes blank in a signal to people from the real world that the wearer is no longer engaging with them.

In a short video on Facebook, the social media giant showed the evolution of the experimental technology with three persons wearing the headset at three stages of development.

However, the research is still experimental. Tethered by many cables, it’s far from a standalone headset, and the eye and facial renderings are not yet completely lifelike.

The research lab plans to present the concept at the SIGGRAPH annual conference on computer graphics this year.