Flying joysticks for better immersion in virtual reality

To interact with elements in the virtual world, common VR headsets usually come with controllers. Users hold these in their hands as they interact with all elements of the virtual user interface. However, the controllers usually do not feel like they look in virtual space. This reduces immersion, i.e., the feeling of how realistic the VR world is perceived.

To improve immersion, Prof. Schneegass' research group, supported by partners from LMU Munich's media informatics group, developed a toolkit for flying control elements.

They first selected five common user interface elements from the virtual world (button, knob, joystick, slider, 3D mouse) and fabricated them using 3D printers. The elements were then attached to quadrocopters, i.e., small drones. As soon as a virtual control element is visible in the virtual world, a flyable with the matching element steers to the person operating it—and the feeling when touching it matches the expectation.

One study already showed that the flyables enable realistic and playful interaction with virtual content. They also allowed for users to feel more immersed in the VR environment.

However, the flyables are currently in a prototype stage and cannot yet compete with modern controllers in terms of input precision. Consequently, the team is currently working on improving the force feedback. They also want to be able to produce flyable elements automatically and within a few minutes via 3D printing.