Consumer Oculus Rift may have facial tracking too

There's a lot of speculation ongoing at the moment about just what Oculus is working on. While many believed the Crescent Bay prototype to be a pretty good indicator of what the consumer version of the Rift headset will be like, we've since heard rumours that Oculus has delayed things to implement additional technologies like hand tracking and possibly even eye tracking. Now there's talk of the potential for facial tracking with a specialised camera rig too.

This information is unconfirmed, but comes from Dr. Martin Breidt,  a research technician at the Max Plank Institute for Biological Cybernetics. He spoke to the VoicesofVR podcast about facial tracking in real time and how some researchers, several of whom work for Oculus Research, have been developing some interesting ways to achieve it in VR.

“Anyone’s face can be instantly tracked and the users can be switched without an extra calibration step,” Breidt said. “During tracking, we explicitly segment face regions from any occluding parts by detecting outliers in the shape and appearance input using an exponentially smoothed and user-adaptive tracking model as prior.”

The idea behind the technology is to allow those taking part in social interactions in-game, to not only have their own face put into the experience, but to track their mouth movements, so lip syncing with a recorded voice could be possible in real time, making the VR version of video conferencing feel far more lifelike. It could also be good for presence in single player experiences where a player can see themselves, or offer up new types of in-game interaction. [yframe url='http://www.youtube.com/watch?v=QqXjVs6Tgm8′]

In the above video example, we get a good idea of what the technology is like and how it works. Most impressively though, it does so regardless of many actions taken by the wearer. They can put their hands over their face, flip their hair around, take a drink and impede the camera in many other ways. This would be key if the user was also wearing an Oculus Rift headset.

This is all being pushed forward by Hao Li, assistant professor at the University of Southern California Department of Computer Science. One of his students recently posted a conceptual image of a Rift headset that utilises a built in facial tracking system. It's not the most elegant and would potentially cause problems with leaning in certain directions, but its application would be very interesting, especially if combined with eye tracking.

Discuss on our Facebook page, HERE.

KitGuru Says: Would you guys wear a rig like this to interact with people online? 

Become a Patron!