On a recent afternoon in a conference room in the Surgery Brain Research Pavilion at the University of Chicago Medicine, first year medical students took turns donning 3D glasses and sitting down at what looked like an arcade game. As they peered into the screen, they used a pen attached to a robotic arm to maneuver a virtual probe on the screen and poke at a series of colored balls. This wasn’t a new way for students to blow off steam between classes and studying, it was a demonstration of a virtual reality system that the University of Chicago Medicine is using to train the next generation of neurosurgeons.
The system provides both a visual simulation and what’s called haptic feedback, or the sense of resistance and position in space of the virtual instruments as users maneuver the robotic arm. Medical students use it to practice basic tasks, such as poking the colored balls to choose which one is the softest, an important skill for exploring the consistency of tissues and detecting tumors. Neurosurgery residents practice more advanced surgical procedures, such as a ventriculostomy, or drilling a hole in the skull to relieve pressure.
Ben Roitberg, MD, associate professor of surgery, said the system has become an integral part of the neurosurgery residency. “Residents do sets of certain procedures, or elements of procedures that require a high degree of coordination, spatial orientation and precision. Where there are many ways to be imprecise and dangerous, this kind of training is really helpful,” he said.
Roitberg began working on the system while he was at the University of Illinois at Chicago (UIC). Pat Banerjee, from UIC’s department of engineering, worked with neurosurgeons there to develop the system, and eventually formed a startup company called ImmersiveTouch that now produces the software and standalone training terminals. The software runs on a standard desktop PC with a beefed up graphics card, much like those used for high-end gaming. It’s housed in a cabinet with a robotic arm attachment and two 3D displays, one that a student looks at while they’re seated and one mounted above so others can watch.
After Roitberg came to the University of Chicago, he continued as an academic collaborator with ImmersiveTouch, helping them develop and test many of their neurosurgical applications. He said the combination of the on-screen, visual simulation with the haptic feedback from the robotic arm is crucial for learning delicate neurosurgical procedures. “I can do some surgeries with my eyes closed because it’s all about touch. You have to imagine the anatomy,” he said. “With this system you feel the resistance of the tissue and your position in space. You can even feel the vibration of the drill and how the bone gets removed.”
In the past, residents trained with plastic bones or cadavers before moving on to assist attending physicians with procedures on real patients. The virtual system now allows them to augment that training, but not replace it.
“The machine does not replace the teacher or coach. The machine augments them in a way that hasn’t been done before,” Roitberg said.
The surgical simulator can provide enhanced feedback on how well the would-be surgeons are doing on their virtual procedures. It records success rates for tasks, how far a student was from the intended target, and whether the steps taken were correct. It even tracks the vibration of the hand. Roitberg said this type of feedback adds to a teacher’s evaluation of a student’s performance because it can monitor things a person can’t.
“The machine is objective, and the machine is vigilant. An observer can notice your expression, your general posture, your attitude, a lot of things, and a mentor or teacher can direct you in many ways,” he said. “But the machines can monitor things the mentor cannot, store it, and then present it in a graphic form.”
Virtual training can be used for more than neurosurgical procedures too. Roitberg said it could apply to ophthalmology, head and neck surgery or basically anything that requires fine motor skills and involves difficult, delicate anatomy. It can also help experienced physicians plan for rare procedures that they may not get to perform that often.
“It is definitely a major future direction because ultimately, surgical simulation should be done fully in virtual reality,” he said. “That’s the way of the future, and that’s how training and practice of any procedure can be repeated multiple times, cost-effectively, anywhere in the world.”
Alaraj, A., Charbel, F., Birk, D., Tobin, M., Luciano, C., Banerjee, P., Rizzi, S., Sorenson, J., Foley, K., Slavin, K., & Roitberg, B. (2013). Role of Cranial and Spinal Virtual and Augmented Reality Simulation Using Immersive Touch Modules in Neurosurgical Training Neurosurgery, 72 DOI: 10.1227/NEU.0b013e3182753093