VR interfaces versus traditional interfaces

VR tools for 3D asset creation and scene management purposes are emerging in greater numbers than ever before, encouraged by the growing use of VR technology. The adoption rate of these VR tools is an indirect indicator of VR interfaces’ benefits, which are not clear when compared to traditional interfaces.

The potential of VR interfaces is nicely illustrated by Bruce Branit’s “World Builder” short film from 2009, which depicts a fictional VR tool for creating 3D worlds:

Such VR interfaces have a lot of promise for fun and intuitive creation and editing of digital assets. But can they match or surpass the productivity achieved with traditional interfaces that utilize a pointing device and a keyboard? This is a question that has been on my mind throughout my VR research career.

Unfortunately there is little research about the immersive VR interfaces’ advantages over traditional interfaces. This hasn’t stopped companies from adding VR interfaces to their existing software, or creating new VR applications for tasks that in the past have been mainly performed using more traditional input devices.

Oculus Medium is an example of the latter. Basically it is Zbrush in VR: a sculpting application for creating 3D models. Unlike Medium, conventional sculpting applications like Zbrush and Mudbox rely on traditional interfaces, and are heavily used in a professional capacity.

VR editors in game engines are another example of using VR interfaces for existing tasks. 3D asset creation and management functionality has been recently added to Unreal Engine and Unity. Below video demonstrates Unreal Engine’s VR editor:

Similar VR editor for scene management is available also for Unity. The Unreal and Unity VR editors are intended to be used via motion controllers, and both include 3D widgets and basic 3D interaction techniques such as navigation, as well as creating and manipulating objects.

For now, Unreal VR editor seems to be slightly more advanced, as it contains snapping, mesh editing tools, and the ability to paint textures on mesh objects. I have personally experimented with such features in the past, by creating an immersive 3D user interface for Blender.

So far I don’t see anything in the aforementioned VR editors, which couldn’t be done with a more traditional graphical user interface, which is also likely to be more productive. In the context of game development suites and 3D animation & modeling, I can imagine VR interfaces being faster only in tasks that do not require great accuracy: quick and dirty placement of objects, 3D sculpting, and animation blocking. More intelligent snapping tools could help this situation in the future.

In the below table I compare VR interfaces to traditional graphical user interfaces that rely on a pointing device (usually mouse) and a keyboard.

VR headsets offer 3D stereo and head tracking. These features provide better spatial perception when compared to 2D displays. There is plenty of research that supports this notion. VR interfaces also have the potential to be more intuitive or even more fun than traditional interfaces, since they are better suited for mimicking real world interaction. Furthermore, VR interfaces are more suitable for exergaming purposes.

On the other hand, 2D mice have better pointer accuracy and ergonomics  than VR controllers, because the user hand rests on a table and motion is restricted to a 2D plane. Current VR controllers are often held in mid-air, which elicits fatigue and strain.

More accurate pointing allows faster object selection using a smaller motion range, which helps traditional interfaces to be more efficient. Another factor contributing to the efficiency of traditional interfaces, is the use of keyboard. A keyboard has dozens of buttons that can all be mapped to different actions. Conversely, VR controllers have only a few buttons and much fewer button press combinations.

If you have any doubts about current VR interfaces being less productive in the terms of performed tasks, then I invite you to observe a professional 3D modeler or a professional gamer using a traditional input devices:

Researchers have created and will continue to create studies that compare task performance (e.g. 3d modeling tasks) between traditional interfaces and VR interfaces. Ultimately it is the adoption rate of VR interfaces by professionals that determines their usefulness compared to traditional interfaces. Therefore we ought to observe if and at what capacity professionals use VR interfaces in applications like Unity, Unreal, and Medium.

Will professional 3D artists start to prefer something like Oculus Medium over Zbrush or Mudbox? Only time will tell. I reckon that there are certain tasks where the use of VR interfaces could become popular (EDIT: for example creating the overall base mesh with a VR tool, while doing detailed sculpting and retopo with traditional tools), whereas majority of work in Unreal, Unity, and 3D animation & modeling software will remain to be performed with traditional input devices. Mouse and keyboard are here to stay for the foreseeable future.

There are no guarantees that VR interfaces would be rapidly adopted by professionals in a domain like 3D modeling; plenty of work still remains to be done in the design, software, and hardware aspects of VR interfaces.

Regarding terminology

In this article I have used the term “traditional interfaces” as an abbreviation for graphical user interfaces that utilize a 2D monitor, pointing device, and keyboard. I avoided using the term WIMP (windows, icons, menus, pointer) interfaces, because its definition is unclear about the inclusion of keyboard shortcuts.

Similarly, I used the term “VR interface” to mean 3D user interfaces that utilize an immersive display (head-mounted display or CAVE) and 3D input devices. Since my points also apply to augmented reality – a subset of mixed reality (MR) – I could have replaced “VR interfaces” with the term “MR interfaces”, if we consider VR to be a subset of MR. The latter notion is slightly misleading, because MR was originally defined as “…anywhere between the extrema of the virtuality continuum.” In other words, VR aims to provide completely artificial reality and not a mix of real and artificial.

Recently, a new term called “extended reality” (XR) has emerged, which is intended to cover the whole reality-virtuality continuum. When it comes to including VR under the XR umbrella, it’s debatable whether extending reality with artificial is any more appropriate than mixing the two. In summary, practitioners could benefit if user interface terminology was more refined.