Yahoo News Touching things in VR with your real hands is a total game changer
The first thing you instinctively want to do after strapping on a VR headset and diving into a virtual world is to reach out and touch what you see.
Unfortunately, this is not a reality on today's most popular VR headsets. Well, official support isn't. Instead, VR headsets either rely on traditional gamepads like the Xbox One controller (Oculus Rift), custom hand controllers (HTC Vive/Rift), side-mounted touchpads on the headset (Samsung Gear VR), or wireless remotes (Google Daydream View).
I've tried all of these controls, and while they work fine, none of them feel completely intuitive or natural; there's a learning curve for all of them. The Vive's hand controllers and the Rift's Touch controllers are good attempts at mimicking your hands, but neither provides full-finger tracking.
The guys at Leap Motion have figured out how to change that.
SEE ALSO: Can filthy VR headsets give you herpes? We have answers.
The startup has been working on its hand-tracking technology, called Orion, that works with its motion controller sensor. With the sensor strapped to the front of a VR headset and Orion running, users can experience VR with their hands (and every finger and joint) without the need for any physical controllers.
Orion has been available for the Rift and Vive since earlier this year, but now the company is taking it to the next level with the "Leap Motion Mobile Platform" which is the same setup but designed for mobile VR. Michael Buckwald, Leap Motion's CEO and co-cofounder, told me the company is targeting mobile VR headsets because it's the easiest entry point into VR for consumers since they don't need a powerful PC; all they need is a headset and a smartphone.
Leap Motion isn't getting into the VR headset business. The Leap Motion Mobile Platform is really a system the company hopes VR headset makers will license and integrate.
Image: raymond wong/mashable
As a proof of concept, Leap Motion built a reference design that straps over the front of the latest Gear VR and connects via its pass-through USB-C port. What I saw won't be released as a consumer product, but if mobile VR headset makers like Samsung incorporate the technology into their headsets, the issues of immersive controls will be a thing of the past.
Having never tried Orion, I was a little skeptical of its hand-tracking. When the Leap Motion sensor launched in 2013 and promised Minority Report-esque air gesture controls for computers, it didn't exactly set the world on fire. It was laden with buggy software and inaccurate gestures.
But all of that is in the past. The Leap Motion Mobile Platform is the real deal. Leap Motion increased the speed by 10x and the controller' dual camera field of view from 140 x 120 degrees to 180 x 180 degrees. The result from the new, more sophisticated module (see below), is faster hand-tracking with virtually no latency for more realistic 1:1 hand controls.
Image: raymond wong/mashable
My demo was only about 15-20 minutes, but it was enough to convince me that real hand-tracking is the future of VR controls.
The demo I tried put me in a rudimentary cyberspace chamber (nothing graphically fancy) that felt very much like the toy box demo I tried with the Oculus Rift at CES earlier this year.
With my hands held out in front of me, the Leap Motion sensor embedded within the Gear VR cover replicated them into the VR world. When I wiggled my fingers in real life my virtual fingers also wiggled. I was blown away by how accurate the tracking was; there was no motion blur and even when I held my hands out in a chopping like position with my ring and pinky fingers obscured by the fingers in front, the sensor still knew they were there.
Image: raymond wong/mashable
Within seconds I built blocks and balls and stacked them on one another in virtual space using simple gestures. All I had to do was select the shape from a menu that appeared when I opened my palm and then use both hands (thumbs + index fingers) to pull them apart to make them larger and smaller.
I launched a ball into the air with a makeshift catapult I made out of a long block. I played catch with another person (that's right, multiple people can connect and interact simultaneously with all virtual objects). I high-fived and fist-bumped another person's virtual hand. I was able to reverse gravity by raising the virtual ground up with two hands and manipulate objects as they floated in zero-gravity.
Image: raymond wong/mashable
I felt like a god, making objects out of thin air and hurling them into the air.
All of the objects have physics to them, too. Although, since there's no haptic feedback, you can't really get a sense of depth, so for example, when I threw an object, I wasn't sure how far it would go.
Still, the ability the touch objects in VR with my real fingers was enough to get me hooked.
I'm also told the Leap Motion sensor for VR is very low-power and only eats up about 5 percent of a phone's battery life.
Despite the short demo, the lag-free hand-tracking was so darn impressive (and didn't make me nauseous) that I could have spent hours playing with the VR blocks. I genuinely didn't want to take off the headset.
Compared to PC-based VR headsets, mobile VR headsets are somewhat handicapped when it comes to controls. Gear VR cleverly integrates a touchpad and DayDream View has a slot to store the touchpad remote, but neither are perfect. It's tiring to hold your hand up to the headset and you could lose the remote.
With Leap Motion Mobile Platform, there are no remotes to misplace and your arms don't get any more fatigued than real life.
Virtual reality has this stigma of being a very isolated experience; you are, after all, cut off from the outside world. But VR can also be very social, as Facebook CEO Mark Zuckeberg demonstrated with the Oculus Rift and Oculus Touch controllers at F8 this year.
Adding realistic hand-tracking to mobile VR is only going to accelerate its social use case, if only because the Leap Motion Mobile Platform doesn't even need any external cameras (like with the Rift and Vive) for hand-tracking to work. I can already picture a near future where people, millions of miles apart, put on a Leap Motion-equipped Gear VR and enter a virtual forum or playground and play ping pong in VR with nothing but their hands.
The goal right now is to get VR makers and developers on board with its hand-tracking sensor and software tools. There are certain experiences like shooting games where a physical controller makes more sense and feels more realistic with a trigger and buttons, but for most typical VR experiences, hands are the most natural input and Leap Motion is way ahead of the pack.
