Machine learning gives users ‘superhuman’ ability to open and control tools in virtual reality
Researchers have developed a virtual reality application where a range of 3D modelling tools can be opened and controlled using just the movement of a user’s hand.
The researchers, from the University of Cambridge, used machine learning to develop ‘HotGestures’ — analogous to the hot keys used in many desktop applications.
HotGestures give users the ability to build figures and shapes in virtual reality without ever having to interact with a menu, helping them stay focused on a task without breaking their train of thought.
The idea of being able to open and control tools in virtual reality has been a movie trope for decades, but the researchers say that this is the first time such a ‘superhuman’ ability has been made possible. The results are reported in the journal IEEE Transactions on Visualization and Computer Graphics.
Virtual reality (VR) and related applications have been touted as game-changers for years, but outside of gaming, their promise has not fully materialised. “Users gain some qualities when using VR, but very few people want to use it for an extended period of time,” said Professor Per Ola Kristensson from Cambridge’s Department of Engineering, who led the research. “Beyond the visual fatigue and ergonomic issues, VR isn’t really offering anything you can’t get in the real world.”
Most users of desktop software will be familiar with the concept of hot keys — command shortcuts such as ctrl-c to copy and ctrl-v to paste. While these shortcuts omit the need to open a menu to find the right tool or command, they rely on the user having the correct command memorised.
“We wanted to take the concept of hot keys and turn it into something more meaningful for virtual reality — something that wouldn’t rely on the user having a shortcut in their head already,” said Kristensson, who is also co-Director of the Centre for Human-Inspired Artificial Intelligence. More