Back29-year-old gamer leaves Sony behind to bring eye-tracking to virtual reality
Yuka Kojima's Fove is introducing the world's first commercially available VR goggles equipped with tiny infrared cameras to follow eye movements
Premium
Tokyo: Virtual reality is on the cusp of becoming mainstream, but one start-up in Japan is betting the technology won’t really succeed unless it cracks one critical piece of the puzzle: human eyeballs.
Fove Inc. is introducing the world’s first commercially available VR goggles equipped with tiny infrared cameras to follow eye movements. By tracking human irises, the gadget aims to reduce motion sickness, improve graphics performance and enhance social experiences by making virtual eye-contact possible, says Yuka Kojima, Fove’s founder and chief executive officer.
“We want to be the company that figures out VR’s unsolved problems," Kojima said. “The immediate goal for now is to get as many headsets into the hands of developers as possible."
While virtual reality can transport a user to a different world, eye-tracking offers a deeper experience because the eyes betray intent, reactions and a plethora of other emotional signals. For example, someone playing basketball in a video game could trick an opponent with a no-look pass or a game could surprise a player by introducing a threat outside their field of vision.
Kojima, 29, is convinced the technology will become commonplace in VR devices. That also explains why others are getting into the game, from Eyefluence, backed by LeapPad founder Jim Marggraff, to Eye Tribe, which makes a desktop eye-tracker for $200. SensoMotoric Instruments and Tobii are also exploring VR applications. Qualcomm Inc. earlier this month announced a reference design for an eye-tracking headset that uses its Snapdragon chip.
Kojima founded her startup in May 2014, soon after Facebook unveiled its $2 billion acquisition of VR headset maker Oculus. Fove ran a Kickstarter campaign, raising $500,000, and eventually got $13 million in financing from investors including Samsung Electronics Co. Ltd, game maker Colopl Inc., electronics manufacturer Foxconn Technology Co. and Taizo Son, the younger brother of SoftBank Group Corp. founder Masayoshi Son.
An early version of Fove’s eye-tracking VR goggles will be available to developers and enthusiasts for pre-order on 2 November, and the headset will cost about the same as Facebook’s $600 Oculus, Kojima said. She contends that her startup’s technology will help distinguish it from the first crop of rival headsets from Oculus, Sony Corp. and HTC Corp. Kojima, a former game producer at Sony, believes she’s one to two years ahead of the competition.
“I expect all the high-end headset companies to be assessing the benefits of eye tracking for next-generation VR headsets," said Piers Harding-Rolls, head of games research at IHS Markit. “Interaction and immersion is probably where eye tracking could have the most impact."
YouTube: “FOVE" The first eye tracking head mount display
One of the challenges of storytelling in VR is that users can divert their attention away from the action in front of them by looking away. Knowing where the person is looking could help developers come up with techniques to guide people’s attention, Harding-Rolls said. For Samsung’s Gear VR, a portable headset that uses a smartphone to display images, eye tracking can help extend battery life, according to Kojima. That’s accomplished by focusing precious processing power on the areas of the screen where the user is looking, a technique knows as foveated rendering.
An avid gamer, Kojima said she was always frustrated that interactions with digital characters involved too much text. She spearheaded a project at Sony to add eye tracking to the PlayStation Vita portable player, but that got axed. Then she shifted her focus to virtual reality and continued working nights and weekends. Kojima collaborated with Lochlainn Wilson, an engineer she met while studying abroad in Australia and the company’s current chief technology officer, to develop a prototype over Skype and email.
“It’s a VR experience where you are not just watching, but also being watched," Kojima said.
One demo by Fove offers a hint of what’s possible: You’re in a warehouse, being interrogated by a masked kidnapper. He presents you with photos of three people to look at, one of whom is an ally. Letting the gaze linger on your collaborator will cause the man to be shot. And don’t try to look away, it will only make the kidnapper more irate.
An engineer monitoring the demo on a separate screen reveals some of the magic: a ghostly view of the player’s moist eyeballs with a computer-generated overlay following the darting iris. The software has had to evolve to avoid getting fooled by a particularly heavy application of mascara or eyelashes that might be in the way.
The basic technology behind eye-tracking isn’t new, but where Fove seeks to make a difference is in the software. The start-up is seeking to amass a large quantity of eye-movement data, which Fove’s two deep-learning specialists will feed into neural networks to improve tracking algorithms. Launching a headset is the best way to collect and analyze that data, Kojima said.
“It all depends on how things play out in 2017," said Kojima, who is also considering licensing Fove’s technology to other headset makers. “I can see it going both ways. We could become something like Dolby. You may see Powered by Fove on other companies’ headsets." Bloomberg
Unlock a world of Benefits! From insightful newsletters to real-time stock tracking, breaking news and a personalized newsfeed – it's all here, just a click away! Login Now!
