Considerations for VR developers

I recently gave a talk titled “Staying Ahead of the Curve in Virtual Reality” at ARTtech Seminar of Assembly computer festival (video embedded at the end of this post). In the talk I discussed some of my own work, and gave an overview on current consumer VR and its near future. In this post I want to focus on the following three considerations for VR developers that I laid out in the finale of the talk:

1. Justification for VR

Ask yourself, why does your application need to utilize VR? Are you using VR just as a gimmick? Games and other applications do not get magically better just by converting them into VR. The use of VR is a no-brainer in certain types of games and entertainment where additional immersion and depth cues are important. But that is not the case for all games (2D games being the most obvious ones), let alone business software.

Take into account that currently the added immersion of head-mounted displays come at the cost of ergonomics; the user has to wear a heavy, sweaty headset that induces vergence-accommodation conflict and suffers from a low resolution. Over time all these issues will be alleviated and even removed altogether, but people will still be using 2D displays far into the foreseeable future. The added value from the use of VR must outweigh its costs for your application.

Furthermore, VR interfaces and traditional interfaces excel at different tasks (I’ll discuss that in a future post), and it is not clear yet in which application domains the use of VR brings noticeable benefits. Entertainment and training VR applications seem like safe bets. But the net benefits are less clear for other domains. For example, in the domain of 3D content creation we can pose the following question: which of the numerous real-world 3D modeling and animation tasks faced daily by production companies become more efficient in VR? Further experimentation is needed from researchers and practitioners of VR.

2. Social VR

I encourage VR developers to give thought on how they could benefit from having people interact together within their VR application. Mere head and controller tracking adds a good amount of expressive power to non-verbal communication in VR. A glimpse of this can be seen at the end of the Oculus Toybox demo video. Humans are social by nature, and all the major consumer VR companies are looking into bringing more human expressions into VR. Thus, in the future consumer VR tracking systems will track our whole bodies, eyes, and facial expressions.

At one end of the social VR spectrum is the concept of metaverse, along with massive online virtual worlds and social networks. AltspaceVR and JanusVR are examples of early attempts at creating immersive, online virtual worlds. At the moment they are still relatively small scale and could be described as VR playgrounds for multiple users. In contrast to open virtual worlds and VR town squares, there will also be demand for more intimate telepresence applications for two people or small groups to convene. Such applications could be a part of a larger metaverse or made available as a standalone, single-purpose app like Skype.

VR game developers should consider adding social dimensions to their games, even for single-player experiences. Simplest way to do this is to implement spectator modes and live VR streaming. A company called Vreal is developing a software platform for that purpose, with the aim of becoming the Twitch of VR.

Multiplayer games that involve waiting (e.g. for the start of the next match) could include simple VR waiting rooms for social interaction. As online communities tend to be riddled with abuse, you shoul look into ways of reducing such bad behavior.

And lets not forget the possibilities that online collaboration in VR presents for business software, as demonstrated by MiddleVR‘s Improov3 demo:

https://www.youtube.com/watch?v=91jvcYkv6Pk

3. Augmented Virtuality / Multitasking

Many current consumer VR headsets completely block the view to the real world, requiring the user’s full attention. If you’re a VR game developer, ask yourself whether your game is captivating enough to keep players away from other activities. Because when the player gets bored, the only escape is to remove the headset and quit the game.

All games become repetitive sooner or later, when the player gets used to the gameplay mechanic or is forced to do mindless grinding in order to proceed. Nevertheless, many players keep playing because they want to see what the game has to offer around the next corner. In non-VR games the players can cope with the boredom by simultaneously engaging some additional activity.

According to a study by Foehr (2006), American teenagers frequently consumed another media while playing video games; 41% of the total time spent on video games involved such multitasking. Playing video games was most commonly paired with watching TV and listening to music. Due to the reality-occluding nature of head-mounted displays, VR developers need to see extra effort to provide similar multitasking capacities that are implicitly present in non-VR games.

HTC and Valve were smart to allow some real-world multitasking by equipping the Vive headset with a camera. This augmented virtuality implementation allows the user to utilize their keyboard and grab their drink while wearing the headset.

Fortunately VR can be augmented with anything, and we are not limited only to real world elements. Envelop VR (EDIT: they folded, check out a company called V instead) and Virtual Desktop applications already demonstrate how to use 2D applications in VR. But rather than have a VR desktop application, I’m more enthusiastic about bringing arbitrary 2D content and software streams inside any VR application or game. This will allow the user to seamlessly multitask between the 2D streams and the actual VR application where they appear. I created the below concept image using a screenshot from Pool Nation VR to illustrate this idea:

In this example the player has augmented the game with two additional 2D streams positioned in the virtual world: a Reddit browser tab and an episode of The Office (streaming from a hard-drive or online video service such as Netflix). Both streams could be viewed and interacted with while playing virtual pool.

VR developers could allow user-specified 2D streams into their application: video screens, browser tabs, and 2D applications. This is somewhat analogous to custom radio stations of GTA games, which allowed players to listen to their own music while playing. Of course it doesn’t make sense for every developer to implement their own 2D stream functionality, and a Unity/Unreal plugin with those capabilities will emerge in the near future.

To summarize this section: a VR developer should decide if their application allows “in-app multitasking” or requires full attention from users. The latter approach could be too much to ask as far as games are concerned.

UPDATE1: User ‘BOLL’ from Steam forums told me that the following software for injecting “2D streams” into VR apps are already being used and further developed:
https://github.com/Hotrian/OpenVRDesktopDisplayPortal
https://github.com/scudzey/UVROverlay

Apparently these plugins are mostly used by Elite Dangerous players to watch TV streams during long and tedious space travels.

UPDATE2: Yet another “2D injection software” is by a company called V, who have just released a private beta. Upon googling, I found out that Road To VR covered them already in May.

Videos of My Talk and Interview

Thumbnail of the video was not chosen by me, and the talk is relatively vendor neutral.

I also participated in AssemblyTV Fireside chat interview, where the interviewer asked about future ramifications of VR. This led to discussion that I considered to be thought-provoking: