What is Virtual Reality Good For?

What is virtual reality good for? Scholars have been asking this question for at least six decades, ever since the technology began to proliferate in research laboratories in both academia and industry settings. Researchers have used VR as a tool to study the human mind, building elaborate simulations and measuring subject responses. In industry, niche tech companies have been building bespoke simulators based on the findings of scholars who have studied the effectiveness of the medium to train pilots, soldiers, and heavy industry workers in low-risk environments (Bailenson, 2024). Now that VR is more accessible than ever, efforts to explore its applications have been increasing, with the goal of making these simulations and their benefits available to a wider audience. The book I co-edited with Lissa Crofton-Sleigh, Past and Future Presence: Approaches for Implementing XR Technology in Humanities and Art Education (Amherst College Press, 2024), presents case studies of a variety of XR applications that allow users to virtually transport into historical settings, dive deeper into artwork, or learn languages in an immersive environment. Stanford’s Virtual Human Interaction Lab (VHIL), where I work, has a framework for understanding VR’s best use cases: anything that is dangerous, impossible, counterproductive, or expensive (aka DICE) to do in the real world is a good candidate for a VR application (Bailenson & Lesher, 2024). But how does one put this into practice? Many current VR applications have focused on creating simulacra of conference rooms and classrooms and all manner of real-world spaces built for socializing, using avatars that look like us. In my opinion, this is missing some of the potential for this technology, as we can already sit in chairs in a conference room or classroom in the real world and have an equal or better experience. To fully realize virtual reality’s potential takes a bit of understanding the historical context, expectations, and limitations of the technology. 

Virtual reality has an undeniable sci-fi quality to it, as movies like Ready Player One or The Matrix portray virtual worlds where people can do kung-fu, fly, dodge bullets, or drive the DeLorean time machine from Back to the Future all without having to get up from their chair. Never mind the dystopian nature of VR-focused media, people still wonder what it would be like to experience virtual worlds like this themselves. In my line of work people frequently ask me “how much longer until VR is like Ready Player One?” And to be honest, I don’t think we are anywhere close to achieving virtual worlds that resemble what Steven Spielberg was able to portray on film. But that doesn’t mean there aren’t other compelling use cases that the current state of the technology can allow.

Nearly all VR scholarship points back to Ivan Sutherland who made what is widely recognized as the first “head-mounted display,” or HMD, in the 1960s (Sutherland, 1968). Sutherland and his team were the first to put together the now crucial combination of display, rendering, and tracking into one system. Today we would consider this an augmented reality (AR) display, with see-through lenses rendering a wireframe polyhedron while tracking the user’s head movement. This was an incredible technological success for Sutherland, and he is rightly regarded as a pioneer in computer graphics. However, less well known is Mort Heilig, who in 1964 created the “Sensorama,” an immersive media project that showed the user a stereo video of a first-person experience of things like riding a motorcycle through the streets or going on a date. His system didn't track users’ movements, but did introduce novel ways of immersing them in a virtual world through their senses of touch and smell by generating wind while riding a bike or motorcycle and blasting smells into the viewer’s nose when their “date” put a bouquet of flowers toward their face. As the medium continues to wrestle with its own potential, Heilig’s creative use of multimedia still inspires VR enthusiasts to this day. 

In the mid-1990s, artists such as VR pioneer and Softimage cofounder Char Davies found that virtual reality’s power of immersion could be used to create compelling art installations (Davies, 1998). In her role as president of virtual research at Softimage, Davies worked with programmers and electrical engineers to create bespoke software and hardware. Davies was in the right place to see the power of VR while also having access to the resources needed to realize her vision. During that time any artist who wanted to make art using VR would have to spend hundreds of thousands of dollars on HMDs, specialized PCs, and software licenses that required arcane knowledge to develop for. Today it is comparatively easy to start making complex virtual environments using high-quality, free, and open-source tools like Blender and Godot Engine. Applications can be viewed on HMDs that cost less than a smartphone. New features for HMDs are being developed by some of the world’s largest tech companies, unlocking features such as eye and hand tracking that allow for more naturalistic interactions and “mixed reality” features that let users switch between augmented reality and virtual reality with the flip of a switch.

Some friction for the widespread adoption of HMDs still exists. The headsets on the market today are not comfortable for extended use and lack some accessibility features that limit their potential reach. To expand the audience for VR, tech companies have put smaller processors in standalone headsets that limit the quality of the graphics they are able to render, thus reducing the visual fidelity compared to other forms of media. It takes time to train users on how to use the headsets, and without effective onboarding people are quick to dismiss the technology due to frustration. To use the headsets, you must create or link social media accounts to access the hardware, creating another point of friction as simple tasks like creating and inputting passwords can feel awkward while wearing a headset. These accounts force you to input your personal data, which means handing over a great deal of trust to companies that users believe may not deserve it.

Back to the question at hand: what is VR actually good for? I can think of several use cases, many of which are being successfully implemented today. To the surprise of many, fitness has risen as a fantastic use case for HMDs. While it’s of course possible to exercise in the real world, how possible would it be to do a cardio routine on the Galapagos islands before you head to work? That’s something that the VR fitness app Supernatural allows you to do. In educational settings, scholars have observed the effectiveness of VR field trips in boosting student confidence in school, and collaborating during virtual lessons can increase learning (Queiroz, 2023). For people whose personal situations make it difficult or dangerous to express themselves in the real world, VR allows them to “wear” bodies that reflect how they feel on the inside, and find comfort and community with others who feel the same. 

These applications have been relatively small in scope, but when I dream big about the potential of VR, I imagine a future where we may travel to fully realized virtual spaces and imagine worlds wildly different from our own. I often think about the late film and media critic Roger Ebert, who reviewed the 1993 computer game Cosmology of Kyoto. Ebert famously disliked video games, but this one struck a chord with him. In Cosmology of Kyoto, players experienced an artfully rendered version of medieval Kyoto, Japan, through a series of interactive still images, voice acting, and seemingly infinite branching choices. For Ebert, despite the 2D graphics and screen-based control interface, what he experienced was “the sense, illusory but seductive, that one could wander this world indefinitely" (Ebert, 1994). Personally, I believe virtual reality could effectively make me feel as though I had been transported back in time to medieval Japan, if such an experience was created with care and research. Like Heilig’s Sensorama, could I put on a headset and experience the feeling of a moment in time with my whole being, every sense activated to tell me I am living a real experience? How incredible would it be to have first-hand exposure to how people lived over 500, 1000, or even 2000 years ago? What can we learn from life in ancient Rome, or the Qin Dynasty in China?

Today we are facing a crisis of the imagination. The problems of the present can be given context and solutions by understanding the past and future, if we can allow ourselves to imagine a perspective different from our own. Virtual reality is a tool that over the years has promised to show us the impossible. If we open our minds to the possibilities and roll up our sleeves, I believe we can build beautiful virtual worlds that will teach and inspire us and open our minds to alternative ways of being. And I don’t think you’ll get that from sitting in a virtual conference room.

Brian Beams is the manager of the Virtual Human Interaction Lab (VHIL) at Stanford University. He studies creative technology, builds educational virtual reality applications, and conducts research in the application of new technologies for art and education.

REFERENCES:

Bailenson, J. N. (2024). Virtual Reality. In M. C. Frank & A. Majid (Eds.), Open Encyclopedia of Cognitive Science. MIT Press. https://doi.org/10.21428/e2759450.83209e91

Bailenson, J., & Lesher, M. (2024). Virtual reality and its opportunities and risks. In OECD Digital Economy Outlook 2024: Embracing the Technology Frontier. (Vol. 1). https://doi.org/10.1787/a1689dc5-en

Sutherland, I. (1968). “A head-mounted three dimensional display,” in Fall joint computer conference (Fall, part I), San Francisco, California, December 9–11, 1968 (New York, NY: ACM Press), 757–764.

Davies, C. (1998). OSMOSE: Notes on being in Immersive virtual space. Digital Creativity, 9(2), 65–74. https://doi.org/10.1080/14626269808567111

Queiroz, A., McGivney, E., Liu, S., Anderson, C., Beams, B., DeVeaux, C., Frazier, K., Han, E., Miller, M., Petersen, X., Woolsey, E., Hancock, J., & Bailenson, J. (2023). Collaborative Tasks in Immersive Virtual Reality Increase Learning. Proceedings of the 16th International Conference on Computer-Supported Collaborative Learning-CSCL 2023.

Roger Ebert, Cosmology of Kyoto Review, Wired, September 1994.