We need your help now

Support from readers like you keeps The Journal open.

You are visiting us because we have something you value. Independent, unbiased news that tells the truth. Advertising revenue goes some way to support our mission, but this year it has not been enough.

If you've seen value in our reporting, please contribute what you can, so we can continue to produce accurate and meaningful journalism. For everyone who needs it.

Minority Report depicted a number of future technologies including gesture-based computer interactions, but a more advanced version of that may find its way to VR. Movieclips/YouTube

A Minority Report-style interface for your devices isn't sci-fi but a real possibility

The likes of Minority Report depict a world where you only need your hands to interact with software – and Leap Motion is working on such an experience.

WITH THE FIRST generation of virtual reality (VR) devices expected to appear next year, it’s an exciting time for the space. While the potential for the visual experience is there, the controls are still versions of the same hardware we had twenty years ago.

Oculus Rift has what may be the best solution with Touch, motion-controlled joysticks that act as hands when you hold them- think of how the Wii nunchuck controller looks and works and you have the right idea – but the majority use a controller to interact.

The interface is ultimately limited to the same format we’ve used for traditional or legacy devices like PCs; the technology might be different but the same principles for interaction still apply.

Leap Motion has been working on software that would track the movement of your hands and fingers directly. Instead of using a controller, you’re using your hands to interact with the world in front of you.

While Leap started off life as a way to interact with your PC and Mac, the company has its eyes set on VR.

Leap Motion / YouTube

So it’s unsurprising that its CEO Michael Buckwald feels that we’re due a change in how we interact with our devices.

“So much has happened in the last twenty years. We have unfathomable powerful computers, not just desktops, but in our pockets, in our cars, thermostats, but the ways we interact with them, I would change them all”, he says. “They’re still very simple, very binary inputs like pressing x or not pressing x, clicking the mouse button, not clicking the mouse button”.

It’s just a very narrow, simplistic way of talking to powerful devices. Of course, what we love about hands is thanks to thousands of years of evolution, every single person knows how to use them to do complicated things, and taking advantage of that is what we’re all about.

When you hear about motion capture, you would be forgiven for thinking it would involve specific gestures or movements to complete actions.

At the most basic level, they’re easy to remember, but they end up being limited to just one action at best.

While they’re easier to complete and have a place, they don’t solve the problem of improving how we interact.

Tracking helps solve the problem of unintentional actions. While it’s easy for a device to recognise a gesture or action when you didn’t mean it, complete tracking makes it difficult to mistake certain actions like drawing a picture or moulding clay.

Leap Motion gif Leap Motion / YouTube Leap Motion / YouTube / YouTube

The other major issue is tracking. For such a system to work and be adopted, there cannot be any lag or delays in interaction, something that Buckwald describes as a very hard technical challenge.

Yet there are a few things that work to our advantage when put in a VR situation. The fact that we have visual cues are so close to what we already experience when using our hands, in comparison to doing the same action on a 2D screen – means it’s easier to grasp how far you have to reach.

Sportsfile (Web Summit) Michael Buckwald speaking at the Web Summit earlier this month. Web Summit / Flickr Web Summit / Flickr / Flickr

Context is everything and even using the likes of Oculus Touch to pick up items, it’s surprisingly intuitive once you remove the lag – and compared to using a controller which requires you to adapt to its logic, hands don’t have that same barrier.

“That 3D context is all the difference”, says Buckwald. “In VR, even if I was wearing thick gloves, I could still go over there and grab a bottle. Visual cues are really helpful”.

It’s not just VR that would benefit from this. Augmented reality, or the use of lenses to project new information and screens – like what Google Glass was trying to achieve and other projects like Hololens and Magic Leap are doing – will also see improvements when such technology improves.

Exactly how that will take form or how it will make its way into everyday life isn’t clear, but if Google Glass taught us anything, it’s that such devices have many steps to take before they become the norm.

Oculus Oculus VR has its own controllers called Touch which tracks your hands, but is limited to what buttons you press. Eric Risberg / Press Association Images Eric Risberg / Press Association Images / Press Association Images

Buckwald says it’s working with most of the big VR companies to embed its technology into the headsets. It will likely be a mixture of headsets with the technology and using physical controllers being released over time.

Headsets with Leap Motion embedded will arrive in the middle of next year and since headsets are large anyway, including it won’t add to the weight.

“Everything will be 3D so your hands can really do the primary interface… and that’s great for us. Hands are really critical to the sense of presence with reality, right now a lot of VR is just a 3D movie”.

I think people’s expectation is when they put on a headset they’re going to be teleported to a different world and that really requires you to see your hands and fingers. It’s a really good time for us and an exciting time for VR.

Read: Facebook wants to end the nightmare of seeing everything your ex is up to >

Read: How a company made infamous by piracy is trying to reinvent itself >

Readers like you are keeping these stories free for everyone...
A mix of advertising and supporting contributions helps keep paywalls away from valuable information like this article. Over 5,000 readers like you have already stepped up and support us with a monthly payment or a once-off donation.

Your Voice
Readers Comments
    Submit a report
    Please help us understand how this comment violates our community guidelines.
    Thank you for the feedback
    Your feedback has been sent to our team for review.