Virtual Reality in Oakland

Jacob Levin had his first epileptic seizure when he was 11 months old. By age 15, he was having three or four a day, making it a challenge to enjoy activities that most teens take for granted.

Strong emotions were a trigger, so Jacob tried to keep his feelings in check. "During basketball games, I play my heart out, and I get into the game," he says. "I get really excited, then in the middle of the court, I’ll have a seizure."

Jacob tried various medications, which either didn’t help or caused side effects. Eager to find a solution, he and his parents traveled from their home in Reno, Nevada, to see Kurtis Auguste, M.D., at UCSF Benioff Children’s Hospital Oakland, a neurosurgeon who could remove the section of abnormal tissue that was causing Jacob’s seizures.

The phrase brain surgery makes most families anxious, but Auguste has found a unique way to help calm nerves. Before the big day, he took Jacob and his parents, Nathanie and Matt Levin, on a journey directly into Jacob’s brain.

Using a virtual reality (VR) system called Surgical Theater, Auguste can put on a headset and "fly" into a 3D model of a patient’s brain. While the technology is primarily used for surgical planning—to chart the best pathway to a tumor or aneurysm, for example—UCSF Benioff Children’s is the first hospital in the country to routinely share this remarkable VR experience with patients and their families.

For kids like Jacob, this tech​nology speaks their language. "I’m a gamer, so a VR headset is always cool," Jacob says. "I thought that going into my brain would be such a unique experience. I was really excited."

"It was such a cool idea," adds Nathanie. "I’m going to stand in the middle of my child’s brain and look around. When do you get the opportunity to do that? With all of the stress, this was almost fun."

So Jacob, Nathanie, and Matt popped on goggles and, a moment later, were transported into Jacob’s brain. Using hand controls to glide around, they saw blood vessels, vision fibers, and the source of Jacob’s troubles. Auguste, represented by an avatar wearing a white coat, acted as tour guide.

"Dr. Auguste showed us where the seizures started, and I could visualize where they were going in my son’s brain," Nathanie says. "That just brings a calm."

Auguste says the system is an invaluable tool for educating patients and their families. "We’ve tried to deal with it in the past—with our words, with fancy models—but it’s difficult to convey this information," he says. "Now we can break down those barriers with this fun, engaging tool. They can grasp the problem and what needs to be done."

"It gives you such knowledge of what you’re going to go through," confirms Jacob. "I felt a lot more confident."


Virtual Reality in Oakland

Neurosurgeon Kurtis Auguste uses virtual reality to educate and empower patients. Photo by Alain McLaughlin.


Inside the Brain

As a surgery-planning tool, VR is a game changer. During neurosurgery, doctors naturally need to avoid nicking important structures like key blood vessels, vision fibers, and the speech cortex. The standard imaging, a series of MRIs, only shows two-dimensional slices of the brain in muddy gray scale. VR provides a three-dimensional model with anatomical structures rendered in bright colors, and much sharper details. So a surgeon can more clearly see a tumor’s boundaries, for example, and better determine the "no-fly zones" for surgery. The added feature is that the surgeon can virtually fly through the anatomy, viewing it from different angles rather than just interacting with it on a flat screen.

"It’s very empowering, because you can see things that were otherwise hidden," Auguste says. "It makes me think back to all of the difficult surgeries I’ve ever done. If only I’d had this technology before, I wonder how much easier those experiences might have been."

When he enters a virtual brain, Auguste says it’s sometimes difficult to separate the neurosurgeon he is today from the little boy he once was, playing video games in his Brooklyn, New York, bedroom. Still, he admits that he has learned a trick or two from his young, tech-savvy patients.

"The same headsets and handsets are sitting in a lot of these kids’ living rooms," he says. "This is their world. To give kids the power to engage with their own diagnosis and have a little combat scenario with the thing that’s causing their problems—headaches, seizures, whatever it may be—is very empowering. I don’t like that kids are lying awake at night and crying because they’re so scared. We’ve gone from that to a laughing, happy child trying to reach out and grab her own brain tumor. That’s a powerful change."

Nathanie acknowledges that seeing the source of Jacob’s troubles up close made all the difference. "After that meeting, Jacob was totally relaxed," she says. "He didn’t seem overly stressed that the next week he was having brain surgery."

Jacob’s first surgery was minimally invasive—no incisions or craniotomies. Auguste drilled tiny holes into his skull and used sophisticated computer navigation to float electrodes to their destinations. Upon locating the seizure "hot spot," he then designed a second, tailor-made surgery to remove the abnormal tissue.

The second procedure went well, and Jacob has now been seizure-free for nine months. He’s even working toward his driver’s permit—something that didn’t seem possible

before surgery. "I’m really excited," he says. "I’m further back than my friends, but I know I’ll be there eventually."


Easing the Pain

Virtual reality is also helping young patients quell pain and stress. Patients at UCSF Benioff Children’s who are undergoing an uncomfortable procedure can, for example, slip on a headset and enter an immersive underwater experience called KindVR Aqua.

"Patients drift along and interact with dolphins and whales, and they get to recolor the ocean," says Simon Robertson, the founder of KindVR, the Alameda company that developed the technology.

It’s not just fun—although it is that. A growing body of research shows that patients experience less pain while they’re immersed in a virtual experience. "VR helps reduce patients’ perception of pain by directing their attention into this virtual world, and it leaves a little less bandwidth to process those pain signals," Robertson explains.

Distraction is a tool that health-care providers have long used, perhaps by playing a simple game with a child while administering a shot. "VR is the next step," Robertson says. "It’s a little bit better at hijacking the brain’s senses. It’s not just like watching a movie; it’s an interactive experience. And all of that added up with this transportive sense of presence—where you feel like you’re somewhere else—is a lot for your brain to chew on. We hear stories about patients who are having a simple 10-minute procedure that becomes a meltdown. So hearing, ‘Oh, suddenly my child didn’t have to be held down,’ is incredible."

When Robertson, a game developer, initially realized VR’s potential, he dove in headfirst. Grabbing a clunky VR headset, he visited UCSF Benioff Children’s to share the research. "They told me to slow down, which was the right move," Robertson admits. "They suggested that I become a volunteer in their Child Life Services department. So I worked bedside with kids that had cancer or sickle cell. I wasn’t using headsets. I was just trying to understand what’s a practical tool to help patients."

That time spent with patients gave him valuable insights as he brainstormed his product and launched KindVR. "A lot of [gaming] experiences want you to stand up and spin around or put a T. rex behind you," Robertson says. "That’s exciting, but it doesn’t make sense for patients who are in a hospital bed or a wheelchair." So instead, those patients cruise through the water in a little submarine, viewing the sea through a window in front of them.

"KindVR has been amazing," says Auguste, who uses the technology. "It’s transforming how kids navigate an inpatient experience. I can only imagine how frightening it can be for them. KindVR games and fly-throughs are visually enjoyable, but they often serve hugely important functional roles as well: motivating patients to maintain certain positions while practitioners work with them."

When KindVR develops a new virtual world, it runs a clinical trial to ensure it’s reducing the patient’s experience of pain or stress before rolling it out. A few years ago, Robertson worked with Anne Marsh, M.D., a specialist in pediatric hematology-oncology at UCSF Benioff Children’s, to design a study of sickle cell disease, which causes pain episodes. Their research found that a 15-minute KindVR Aqua session led to significant pain reduction. VR experiences can be used in concert with pain medication and also have the potential to help patients reduce their opioid consumption.

The power of virtual reality is nothing new to kids, and now, as VR makes its way into medicine, the applications seem limitless. "The medical fields, to varying degrees, use two-dimensional imaging to get themselves ready for procedures or to understand diagnoses," Auguste says. "I don’t know that there’s any walk of medical life that couldn’t benefit from this three-dimensional translation of a two-dimensional world."

He believes that the VR applications being used at UCSF Benioff Children’s set the stage for a lifetime of healthy living, as kids with positive health-care experiences are more likely to become adults who keep up with their doctor appointments. "It’s unfortunate, but really it’s our fear—whether acknowledged or hidden—that so often keeps us from being good stewards of our health," Auguste says. "This technology allows us to face those fears in a fun and exciting way." 


VR’s Young Pioneers

Virtual Reality at Berkeley, a multidisciplinary community of 500 students—studying engineering, molecular cell biology, cognitive science, and more—has become a hotbed of innovation in virtual and augmented reality.

"We’re looking at non-gimmicky uses of VR," says Bryan Tong, the group’s retiring president. "How can VR change people’s lives?" Among the students’ experiments are some intriguing projects focused on health.


Brain Games

One team aim to help people with mild traumatic brain injuries (mTBI) by creating experiences that restore cognitive functions. "We have a maze with a series of obstacles to essentially train different areas [of the brain]," says Ilian Herzi, a premed student and colead of the project. "The hope is that, after practicing sufficiently, you would see cognitive gains in memory."

"People could use it in the comfort of their home," says team colead Revekka Kostoeva, who studies computer science and applied mathematics. "A veterans center could perhaps invest in one or two headsets, and veterans could go through these treatments, kind of like a public library."


Mind Over Matter

Research shows that biofeedback—in which people learn to control bodily functions like blood pressure and heart rate—can help alleviate depression, anxiety, and ADHD. Those findings gave computer science and music student Joshua Yang an idea: "We could take that one step further by seeing what VR could offer. "

He assembled a project team to test whether users could control something simple: the watering of a virtual flower. The user wears a Muse Headband, which reads brain waves, and an Oculus Rift headset, to enter virtual reality. A flower appears with a watering can floating above it. "Once you focus, you tilt the can with your mind," Yang says. "The water flows, and your flower starts growing."

"This could help students with learning disabilities like ADHD focus in the classroom," says Shreya Sudarshana, who coleads the team.


Virtual Vision

Students from UC Berkeley’s School of Information have developed a VR prototype that might one day help the blind and visually impaired navigate unfamiliar places. Research included talking to accessibility experts and blind advocacy groups. "They wanted more ways to get semantic information about the environment," says team lead Dylan Fox. "There’s a lot of text that’s not available to blind people."