Jeremy Bailenson - Experience on Demand: What Virtual Reality Is, How It Works, and What It Can Do

EXPERIENCE
ON DEMAND

WHAT VIRTUAL REALITY IS, HOW IT
WORKS, AND WHAT IT CAN DO

Jeremy Bailenson

W. W. NORTON & COMPANY

Independent Publishers Since 1923

NEW YORK | LONDON

FOR CLIFFORD NASS,

THE KINDEST GENIUS IVE EVER MET.

EXPERIENCE
ON DEMAND

M ark Zuckerberg is about to walk the plank.

Its March 2014, and were standing in the multisensory room of the Virtual Human Interactive Lab (VHIL) at Stanford University. Im making last-minute adjustments to his head mounted display (HMD), the bulky, expensive, helmet-like device that is about to take him into another world. Zuckerberg, at the moment plunged into darkness, is asking me questions about the technical specs of the VR hardware in my labwhats the resolution of the displays in front of his eyes? How quickly do the images on those displays update? Unsurprisingly, he is a curious and knowledgeable subject, and hes clearly done his homework. Hes come today because he wants to experience state-of-the-art virtual reality, and Im eager to talk to him because I have opinions on the ways in which virtual reality can be used on a social networking site like Facebook.

At Stanford, we are encouraged to face outward and to share our work, not just with academics but with decision makers of all types. I often do this kind of outreach, sharing my labs unique capabilities with business executives, foreign dignitaries, journalists, celebrities, and others who are curious about the experience of virtual reality. On this day, I am eager to show Zuckerbergsomeone who has demonstrated great philanthropic investment in areas of education, environment, and empathy researchthe ways in which our work on VR has direct applications for those same issues. But first I have to show him what my lab can do. I usually start with the plankits one of the most effective ways to evoke the powerful sensation of presence that good VR produces. And our VR is very goodone of the best in the world. The floor shakes, we have haptic devices that give feedback to the hands, 24 speakers that deliver spatialized sound, and a high-resolution headset with LEDs on the side that allow cameras mounted around the room to track head and body movement as the user walks around the room. All of this information is assimilated to render interactive, digitally created places that allow the user to experience almost anything we can think offlying through a city, swimming with sharks, inhabiting nonhuman bodies, or standing on the surface of Mars. Anything we decide to program can be rendered in a virtual environment.

The display comes on and Zuckerberg sees the multisensory room again, except that I and my lab assistants have disappeared. The room he is looking at is discernibly lower-resa bit like television used to look before High Definitionbut the carpeted floor, the doors, the wall in front of him are all there, creating an effective simulacrum of the space he was just standing in. Zuckerberg moves his head around to take it all in, everything smoothly scrolling into his vision as it would in real life. He steps forward and backward, and the illusion projected a few inches in front of his eyes corresponds with the movement of his body. Trippy, he says. I lead him to a spot on the floor (I will be constantly at his side spotting him during this demonstration, as its very easy to bump into real-world things when youre navigating a virtual space) and instruct my assistant in the control room to start the program. Lets do the pit.

Zuckerberg hears an industrial whine, the floor shudders, and the small virtual platform on which he stands shoots away from the ground. From his perspective, which I can see via a projection screen on the wall, hes now standing on a small shelf about 30 feet in the air, connected by a narrow plank to another platform about 15 feet away. Zuckerbergs legs buckle a bit and his hand involuntarily goes to his heart. OK, thats pretty scary. If we were measuring his stress signs wed see that his heart rate was speeding up and his hands were beginning to sweat. He knows he is standing on the floor of a campus lab, but his dominant sense is telling him that hes precariously balanced at a deadly distance above the ground. Hes getting a taste of presence, that peculiar sense of being there unique to virtual reality.

Over the nearly two decades that Ive been doing VR experiments and demonstrations, Ive witnessed this scenewhen a person is first enveloped by a virtual environmentthousands of times, and Ive seen a lot of reactions. Some gasp. Some laugh with delight. Depending on whats being rendered in the program, Ive also seen people cry out in fear, or throw their hands up to protect themselves as they hurtle toward a wall. An elderly federal judge once dove horizontally into a real table in order to catch an imaginary ledge after he fell off the virtual platform. At a demonstration at the Tribeca Film Festival, the rapper Q-Tip crawled across the plank on his hands and knees. Often my subjects just stand slack-jawed with wonder, gazing down, up, and around, amazed to see themselves suddenly surrounded by a digitally rendered world that nevertheless feels, in crucial ways, real.

Its a strange feeling, standing above the pit. No matter how prepared you think you are going to be the first time, it still takes you by surprise. Consider first that you, a subject immersed in the pit program, know you are participating in a VR demoits not like you accidentally wound up in a VR rig. So youre anticipating it. In fact, youve probably already witnessedand perhaps even been amused byanother person going through the pit demo before you. Youve looked at the projection of what theyre seeing as they stand over the ledge. Youve watched their legs bend as they lower their center of gravity, and seen their arms shoot out to their side for better balance. Watched as they tentatively shuffle across a thin beam that exists only in the programs code, and in the part of their brain that is interpreting the illusion projected into their eyes. Watched a person standing in the middle of the lab room, wearing an awkward helmet tethered to the ceiling by cables, ever so tentatively lean forward to gaze at a drop-off that doesnt exist.

In truth, its kind of funny. And yet still, there you are a few moments later wearing the HMD yourself, your feet planted firmly on the same, solid floor you traversed just moments before, now suddenly facing a deadly drop with only a narrow plank to walk across. And its not so funny anymore. If youre like one-third of the subjects in my lab, when I ask you to take a step off the platform into the virtual abyss, youll refuse, standing rooted to a spot on the floor.

Zuckerberg does walk the plank, although its not easy going. After he makes it to the other platform, I run a program in which his avatar grows a third arm and he has to learn how to move it by twisting his real limbs. Then he flies through the air like Superman. We put him in the body of a senior citizen and point him to a virtual mirror so he can see this strange doppelganger reflect the gestures he is making in the lab room. I load up another program, and he inhabits the body of a shark and swims around a coral reef. Its not bad, being a shark, he says. After a few more minutes hes had enoughthe experience of VR can be overwhelming and even the best equipment today can cause eyestrain and head discomfort after twenty minutes or so.

For the rest of his two-hour visit to the lab we discuss my research into the psychology of virtual reality, and how it has convinced me that there are many ways the unique power of VR can be applied to make us better people, more empathetic, more aware of the fragility of the environment, and more productive at work. We talk about how VR is going to improve the quality and reach of education, and open up the world for people who cant afford to travel, transporting users to the tops of mountains, or into Earths orbit, or into a calming oceanside setting at the end of a long day. How its going to allow us to share experiences like these with our friends or family, even if they live far away.