VRx

CHAPTER ONE

The Second Time I Died

I AM WEARING A CATSUIT WHILE RECLINING IN A CHAIR WITH MY LEGS kicked up on a coffee table in the research laboratory of Mel Slater, a professor of virtual reality at the University of Barcelona. Motion detectors and tiny vibration motors are affixed to my arms and legs. I am being tracked by an array of twelve cameras surrounding my chair that will soon inform a computer how my body is moving in space and time. Slater is standing behind me. A slight, bespectacled man straight out of central casting as the archetypal university professor, Slater speaks in a calm and light British accent. He is reviewing my final flight instructions for what is sure to be a most unusual journey. I’ve traveled six thousand miles from my home in Los Angeles for this moment.

“Just relax into the experience.”

I trust him. I’m ready.

Slater’s postdoc, Ramon Oliva, hands me a headset. I put it on and find myself in a comfortable living room with a lit fireplace, plush seats, and wood trimming. There is a mirror on the wall in front of me that reflects the image of my pixelated doppelgänger. And there’s the coffee table under my feet, just where I left it.

“Okay, move your legs around on the table,” says Oliva.

I oblige. In perfect synchrony, I see my own legs dance about on the digital tabletop. For a moment, I cannot tell whether those legs are mine. They look like my legs. They act like my legs. But are they my legs? I wiggle them back and forth. They move as expected. Yes, those are my legs. At least, I’m pretty sure they’re my legs. If not, then whose legs could they be?

Next, little blue balls drop from the ceiling. Pop! I feel a ball hit my foot. A vibration engine in the suit fires at the same moment the virtual ball strikes my virtual body. Pop! Pop! Two more balls hit my left and right hand. Pop! Pop! Pop! Balls keep falling from the ceiling and tapping my limbs.

I think for a second: What is happening here? A computer somewhere in this room is running thousands of lines of code that are creating an illusion of spherical balls. Yet, those virtual balls—those pixelated clusters—are hitting my body with a very real physical force. I am feeling those digital balls. Those balls are real. This body is real. It is my body. The physical and virtual worlds are becoming indistinguishable. They are starting to feel like one unified existence.

I am now locked into my digital self. I have assumed what Slater calls full body ownership. The synchronous visual and tactile stimulation has convinced my brain that it now resides in a virtual head, in a virtual body, in a virtual room. My virtual feet are on a virtual coffee table. I see my body in a virtual mirror. I exist in this virtual world. This virtual world is a real world.

I will not soon forget what happens next. It is indelible, even mystical. I will do my best to describe it.

I start to move. And I mean I start to move, but my body stays put. In a steady backward flow, my personhood—my thinking self—begins to pull away from my body. I imagine sinews stretching, pulling, and snapping as my body resists the separation. It cannot hold on. I vacate my body and watch it as my consciousness drifts up toward the ceiling. I sense a brief existential crisis, a sort of Cartesian fit where my mind and body seek to reconcile the separation. For the first time in my life, I am disconnected from myself. I cannot tell if this is a physical or metaphysical cleavage, but it doesn’t really matter. It feels as if I have been extracted from my body. I am floating like a balloon. Now I am up in the ceiling looking down upon my lifeless self. I am moving my hands, and those balls keep following me up to the ceiling and continue to strike my limbs. I’ve become an ethereal and disembodied entity hovering above my shell of a body below. That body down there is not moving. My arms are moving—I can feel them—but that body is not. I am in motion but that body is still. That body is unresponsive.

I am dead.

I am strangely entranced—even invigorated—by my virtual passing. I feel free from my body. And most confusing of all, I am no longer sure who the I is that’s observing this phenomenal scene.

Where do you end and the world begin? Where is the frontier between self and other? Are we defined by corporeal boundaries? Or do we extend beyond the physical? And how can answering these questions help explain my out-of-body experience? Just what happened in my brain, and why? After literally rising from the dead, I needed some answers.

Generations of philosophers, cognitive psychologists, and neuroscientists have grappled with these questions about the nature of self. As an undergraduate I was fortunate to study Philosophy of Mind at Tufts University under the tutelage of Daniel Dennett, a renowned cognitive scientist and philosopher who was among the first to align diverse fields into a unified theory of consciousness. Dennett’s work combines philosophy, psychology, and neuroscience to argue that what we experience as the mind is an emergent property of neurons oscillating in the brain. He says a lot more than that, but ultimately Dennett contends that physical phenomena and neurologic scintilla generate what we experience to be our conscious self. He argues that material stuff is the basis of all sensations, emotions, and cognitions. In other words, we end where our body ends, and the self emerges from the fantastic complexity of our wet inner world. Dennett’s materialist argument, which is now shared by most modern neuroscientists, offers a framework for understanding my own out-of-body experience.¹

Professor Dennett introduced me to the classic “brain-in-a-vat” thought experiment made famous in 1981 by Hilary Putnam,² an American philosopher who, like Dennett, spent much of his career studying the nature of mind. Putnam posed a curious question: How can you know that you’re not just a disembodied brain floating in a vat of nutrients, wired up and connected to a computer that pumps your noggin full of artificial illusions simulating reality? He imagined a mad scientist who created a supercomputer that commandeers the brain into experiencing the phenomena of life. The result would be sensing the world as if you were a walking, talking entity, but, in fact, you would just be a brain floating in a vat. Putnam’s thought experiment became the premise of The Matrix movie series, where people live in pods that nourish the body while the brain is occupied by a virtual reality supplied by sentient evil machines. In this dystopian world, the experience of life is just the result of electrical illusions.

What is often lost in discussing the brain-in-a-vat concept is that Putnam used his thought experiment to disprove that we are, in fact, brains in vats. His epistemic arguments are beyond the scope of this discussion, but suffice it to say, Putnam’s thought experiment allows us to both contemplate the notion of computers simulating reality and to ponder what that means about the nature of self and how our brains operate. What was a mere thought experiment in 1981 is now something close to reality with advances in VR technology.³

Just consider my virtual out-of-body experience. Rather than Putnam’s mad scientist in a lab, there was Professor Slater (who is certainly not mad) in his lab. Rather than some brain floating in a vat of chemicals connected to a computer, there was, well, my brain connected to a computer through sophisticated sensory feedback loops. Rather than lines of code convincing a floating brain that it is experiencing a spiritual event, there were lines of code convincing my brain that it was experiencing a spiritual event. The brain-in-a-vat is no longer a thought experiment; it is a real neurophysical experiment. And thinking about it can tell us something profound about the slippery relationship between what we see and what we feel.

So just what happened in my brain? How did a set of motion sensors, a bank of cameras, a VR headset, and a computer cause my brain to experience an ego-dissolving, selfhood-busting, mystical event that left me both dead and alive while spread apart in a virtual living room? How did I end up in The Matrix ?

The answer has something to do with that mirror in Professor Slater’s virtual living room. At first, I saw a reflection that made sense. I moved my body in space, and the reflection responded in kind, abiding the familiar laws of motion I’ve experienced my whole life. But then, like a magic trick, my reflection in that same mirror no longer made sense. I saw an inanimate body in that mirror. I watched myself pass through my motionless avatar and float behind it. I watched myself die. In the process, Slater’s brain-in-a-vat computer simulation coaxed my brain into accepting an alternative reality. I was no longer able to tell where my body ended and the world began. My physical sense of self in the world—my literal coordinates in space—were temporarily unbounded, allowing my mind freedom to roam in ways it never knew were possible. Even now, long after the demonstration finished, I feel differently about my relationship to my body. I can now say I’ve felt an out-of-body experience, and I can testify that it reduced my own personal fear of death, if even just a tiny bit. Understanding how it did this offers insights into the power of VR to alter mind and body, in both the short and long term. It helps us explain the unique ways that immersive therapeutics can improve health.

We begin this scientific journey by contemplating how a dime-store mirror can profoundly transform human consciousness.

The discovery that culminated in my mystical experience in Barcelona traces back to an astonishing yet simple study conducted by V. S. Ramachandran in the mid-1990s at the University of California, San Diego.⁴ A clinical neurologist and cognitive neuroscientist, Ramachandran was struggling to find an effective treatment for his patients suffering from phantom limb pain. Phantom limb is a maddening condition where people who suffer an amputated arm or leg continue to feel its ghostly presence long after the appendage is gone. For some patients, the phantom limb feels extremely painful—even spastic—like a hand balled up in a tight fist. Yet, there is no hand. There is no fist. In a bid to restore a sense of corporeal wholeness, the brain hallucinates the continued presence of a false limb.

Ramachandran and his colleagues had a conceptual breakthrough: if they could just fool the brain into thinking there was a real limb present after all, then they could offer patients a chance to regain control over their phantom sensations. The researchers did this by creating a simple and elegant device they called a virtual reality box, also dubbed a mirror box. In the case of a missing arm, it worked like this: patients placed their remaining, healthy arm through a hole cut in the side of a cardboard box. The box was split into two chambers with a mirror affixed vertically along the separation wall. When positioned correctly, the healthy hand reflected off the mirror and created the appearance of a complementary second hand exactly where it should be. The following image shows a healthy left hand reflecting off the mirror and simulating the presence of a healthy right hand. If you look carefully, you can see that the right arm does not extend into the box at all; the amputated stump stops short of the box, yet the reflected hand appears like an extension of the missing limb.

Ramachandran’s first mirror box patient was a fifty-five-year-old man with unrelenting sensations of spasms in his phantom arm. These sensations persisted for months after the amputation. The patient viewed his illusory hand in the mirror while attempting to resurrect his phantom hand. On the very first attempt, he reported that all movement had come back into his missing limb and that the pain subsided. Soon after, Ramachandran tested the mirror box with a twenty-eight-year-old man also suffering from severe phantom limb pain. His pain had persisted for nine years after a traumatic arm amputation. The first time he tried the box, the patient described the experience as “mind boggling” and reported that his arm was “plugged in again” and no longer felt “like it’s lying lifeless in a sling.”⁵ After three weeks of therapy, his phantom arm pain disappeared completely. It never returned. And with that single experiment, Ramachandran laid the scientific groundwork for therapeutic VR. He had discovered a rudimentary yet revolutionary technique to alter consciousness.

Much of the promise of VR is predicated on the fundamental insights gained by the mirror box experiments. Our sense of self—our perception of where we physically begin and end—is a substrate for feelings like pain and anguish, and it can be easily manipulated. Our very coordinates in space, which we take for granted as objective, essential, and absolute, are less certain than we might imagine. Virtual experiences have a remarkable ability to undermine certainty. This power may confuse or deceive, but if employed properly, it can strengthen cognitions and enhance well-being.

Admittedly, it seems like a big jump between using a mirror for a phantom limb and teleporting an entire body to the ceiling. To tell the next part of this story, we turn to the unlikely combination of a rubber hand, two paintbrushes, and a knife.

When Henrik Ehrsson was a child he wondered whether God had played a trick on him by placing his soul in his brother’s body, and vice versa. He felt there had been a cosmic mix-up that left his mind in the shell of the wrong person. This curious thought caused Ehrsson to ask some heady questions as a child, like, “How do I know that this is my body, and not my brother’s body? Why does it feel like I am located inside this body? And how do I experience the world through my own eyes?”⁶

So this little boy became interested in the brain and what it means to have a conscious human mind. He wondered how we can recognize that our limbs are part of our own body, and how we sense the physical location of our self within our body, and not elsewhere. Forty years later, Ehrsson is an esteemed professor of cognitive neuroscience at the Karolinska Institute in Sweden, where his lab is investigating the nature and boundaries of personhood. His provocative research challenges what it means to be a thinking mind within a physical body. Ehrsson’s work is important because it provides the missing link between Ramachandran’s mirror box and Slater’s virtual out-of-body experience.

When Ehrsson began planning experiments to address his childhood curiosities, he came across a fascinating 1998 study published in Nature by Matthew Botvinick from the University of Pittsburgh and Jonathan Cohen from Carnegie Mellon University.⁷ Inspired by the work of V. S. Ramachandran, Botvinick, and Cohen created a cognitive parlor trick that had the opposite effect of the mirror box illusion. Rather than disowning a false limb and extinguishing its misleading sensations, their experiment induced ownership of a false limb and removed ownership of a real limb. The bizarre result was a convincing illusion of transferring body ownership from one’s real arm to a fake arm.

Here’s how it worked: Botvinick and Cohen sat people down in a chair and asked them to rest their arms on a table. Then they positioned a vertical barrier between the subject’s eyes and one arm, putting the limb out of sight. Next, they placed a lifelike rubber hand on the table near the real hand, but on the visible side of the vertical barrier. The subject could not see the real hand, but instead viewed a life-sized fake hand in nearly the same location as the true hand. Once the setup was complete, the researchers stroked the rubber hand and real hand at the same time using a pair of fine-tipped paintbrushes, taking care to synchronize the brush strokes as closely as possible. This photo shows the experimental setup.

Botvinick and Cohen found that before long, their research subjects were convinced the rubber hand was their own. They had embodied a gag appendage as an extension of the real body. When subjects were asked to close their eyes and point to their real hand, they were more likely to point to the fake hand, not their own, demonstrating the limb transfer illusion was complete. Just like with Ramachandran’s work, the brain had been duped again, but this time into owning rather than disowning a false limb.

Ehrsson suspected this research could help satisfy his childhood questions. He set about studying just how the rubber hand illusion occurs and what it means about our sense of self. To achieve this, he placed research subjects into a functional magnetic resonance imaging (fMRI) scanner to take pictures of their brain during the rubber hand experiment.⁸ The fMRI revealed that brain patterns shifted abruptly once the subject accepted ownership of the rubber hand. There were upticks in areas that involve integration of touch, vision, and movement. Activity in an area called the premotor cortex most closely reflected the feeling of hand ownership. After a lifelong pursuit of his childhood curiosities, Ehrsson discovered a place in the brain where we feel the sense of physical embodiment.

And then came the knife part. In another dramatic experiment, Ehrsson used the same rubber hand procedure as before, but this time he pulled out a knife and threatened to injure the fake hand. Not surprisingly, people freaked out.⁹ The rubber hand illusion is so authentic that research subjects jump back, their heart rate speeds up, and they begin to sweat. On fMRI scanning, the pain-anticipation areas of the brain light up, demonstrating an immediate, hardwired response to the perception of impending bodily harm. Nobody was actually in danger in these experiments, but their brains weren’t taking any chances.

Ehrsson didn’t stop there. He asked an intriguing question: If we can fool the brain into owning or disowning a false hand, then why can’t we trick it into accepting an entire false body? Can we literally body swap using the modern equivalent of smoke and mirrors? And what is that modern equivalent? Ehrsson had a candidate technology in mind: a VR headset.

There is something special about the first-person point of view. As I type these words, I see them through my eyes. Photons streaming off the words trigger cells in my retina to fire signals that reach the visual cortex in the back of my brain. My visual cortex interprets the neural fusillade as words on a computer screen. My brain assigns meaning to the words, checks for typos, ensures they are expressing what I am trying to say, and so forth. I only know this experience through my eyes and my body. My first-person vantage point is intrinsic to who I am. It is hard for me to imagine experiencing these words through someone else’s eyes and body, just as it would be hard for you to imagine experiencing these words through my eyes and body.

What if we wanted to swap perspectives and experience the world through each other’s eyes? How could we do it? We might each wear a first-person camera and livestream our view. I could watch a monitor and see what you are seeing, and vice versa. But I wouldn’t really feel like I was in your head, or using your eyes, or occupying your body. Nor would you feel the same in reverse. We would both just see what the other person was seeing.

Henrik Ehrsson thought about these same questions and realized a VR headset might offer a solution. He devised another experiment where the research subject wore a head-mounted display and gazed straight ahead while sitting in a chair. There was a stereoscopic camera behind the chair that beamed livestream images to the headset. This arrangement allowed the seated individual to see an immersive view of his or her body from behind, as if experiencing the scene through the eyes of a third-person observer in close proximity. Next, Ehrsson tapped the chest of the research participant with a stick while synchronously poking under the camera with a matching stick. This picture shows the experimental setup.¹⁰

People in the chair felt displaced to the location of the camera behind their actual coordinates. They experienced a radical out-of-body illusion where they vacated their physical body. Rather than removing ownership of a phantom limb, as Ramachandran achieved with his mirror box, Ehrsson caused ownership over a phantom body using a souped-up version of the rubber hand illusion.

Ehrsson conducted additional experiments to explore the body-swap illusion. In one study, he used a VR headset to transport people into the body of a rubber mannequin.¹¹ Then he attacked the mannequin with a knife, causing people to react violently. In another experiment, he shrunk people down to the size of Barbie and Ken dolls, literally causing them to feel like Alice in Wonderland surrounded by oversized objects.¹² Other researchers followed suit, showing that people could feel a body swap with each other using a VR headset as the illusory conduit. An interdisciplinary group of scientists, artists, and designers created The Machine to Be Another, a program that allows anyone to experience the world from the viewpoint of another through a head-mounted display.¹³ The program features a library of what the developers call embodied storytelling experiences. The effect is well captured by Aaron Souppouris, a reporter for The Verge, who described his experience of switching gender:

After careful observation of these body-swap experiments, Ehrsson identified three rules that enable the illusion. First, he discovered that the tactile stimuli must be delivered simultaneously between the real and false bodies. If there is a delay between the visual and actual touches, then the illusion is violated. Second, the stimuli need to be in the same direction. For example, the brain will not accept a virtual hand if it’s stroked in the opposite direction of the real hand. Third, the virtual body needs to be very near the real body. If there is too much distance between the bodies, then the brain sees through the ruse. He called these key principles the temporal rule, spatial rule, and distance rule of body transfer.¹⁵ When these three conditions are met, the brain is tricked into relocation.

Ehrsson’s three rules offer a fundamental recipe for shedding the self. When all three criteria are satisfied we can patch into a new reality. Now think back to my out-of-body experience. Professor Slater fooled my brain using a combination of insights from Ramachandran’s mirror box, Botvinick and Cohen’s rubber hand, and Ehrsson’s body-swap rules. Using the temporal rule, he synchronized the virtual ball strikes with the sense of touch that was created by vibration engines in the catsuit. Using the spatial rule, he coordinated the directional movements of my actual and virtual legs on the coffee table. Using the distance rule, he started the illusion very close to my own body—even inside my own body—and only then distanced myself from, well, myself. The result was a complete out-of-body experience.

But more than that, Slater induced a transcendent, even mystical, experience. It was more than a parlor trick. Beyond the physical wonder of spatial displacement, I had the emotional experience of confronting mortality. Just how did one lead to the other? It is this leap, from sensorial to emotional, where we begin to appreciate not just the physical impact of VR on the body but also its therapeutic benefits on the mind.

Speaking of leaps, I have virtually died twice in this book so far: first, after virtually leaping off a building. Then, again, after teleporting to the ceiling and observing my lifeless body from above. I was petrified in the first instance. I felt oddly transcendent in the second. In both cases, VR created powerful cognitions and strong physical sensations. These were full body experiences.

I don’t think I was a brain-in-a-vat in either scenario. That just can’t be right. Both experiences required that I have a body, not just a brain. I reside in a physical body, and that body affects everything I think and know. It is shortsighted to think of the brain as a bundle of nerves riding atop a physiologic machine. Instead, it is an integrated part of the machine that is intertwined and harmonized within the body as a whole.

In 1641 the French philosopher René Descartes famously concluded, “I think, therefore I am.” It was his way of saying that he believed the thinking mind was made of immaterial stuff that was categorically separate from the physical material of the body. Descartes proposed that the mind and body interfaced at the pineal gland, a tiny structure buried deep in the brain. This view became known as dualism, meaning the mind and body are two distinct entities. Cartesian dualism dominated Western theories of mind for centuries. Turns out it was wrong. Modern neuroscience suggests the story is more complex and interesting. Our sense of self—our very consciousness—depends not only on having a brain but also on having a body. We need our bones, tendons, ligaments, viscera, and sense organs to be conscious.

You know this to be true if you’ve been to a rock concert and felt