Once upon a time, a young doctor joined the surgical unit at a prestigious hospital as an anaesthesiologist. He was an Apple user, of course, wearing Vision Pro during surgical procedures, as it allowed him to monitor all the vital signs during operations.
This is not a fairy story. It is a prediction based on the known direction of travel. After all, the device can already be used to display patient medical records in one expansive view….
We saw with Google Glass that one of the few successful deployments for that device was in certain sectors of the medical profession. Apple’s Vision Pro brings all that device did, adding in the kind of power you’ll get from a wearable computer augmented with AI. That’s why the medical profession is paying a lot of attention to the potential for the device.
A tool for transformation
GlobalData is monitoring more than a dozen mixed-reality trends in the medical field. These include optomery applications, surgical tools, and remote-care solutions, the analysts said.
Sourabh Nyalkalkar, practice head of innovation products at GlobalData, explained some of the future in a statement:
“The basic premise of surgical AR/VR innovation revolves around integrating XR technologies into surgical procedures. However, the intricate nature of surgical care demands more than just superimposing computer-generated images onto a surgeon’s view. The goal extends to providing real-time visual guidance and enhanced awareness during surgery.”
This technology will usher in “transformative change” across multiple industries, and the good doctor at the top of this tail is joining one of them.
Take a look at the evidence.
In the operating theater
A company called eXeX has already used Vision Pro in the operating room — when Dr. Robert Masson used the device during spinal surgery. The doctor sees the use of visual holographic guidance during surgery as transformative.
“This advancement is not just about enhancing surgical precision, but about revolutionizing the entire surgical team’s approach, making each operating more calm, quiet, and effortlessly efficient,” he said. The work builds on existing use of augmented reality, which UC San Diego Health began to use during surgery in 2021.
There are also uses in the associated field of radiology. The Visage Ease VP app provides an immersive experience for viewing high resolution radiology images; because the viewing field is so immense, this can help deliver better patient care strategies.
“Technology that allows for sophisticated eye motion and gesture controls for reviewing 2D and 3D medical imaging could potentially help in efficient tumor board reviews and create collaborative spaces in healthcare,” said Paul Murphy, one of the radiologists involved in the project.
In the hospital
Sharp Healthcare in San Diego is exploring the use of Apple’s new wearables at its new Spatial Computing Center of Excellence program. Thirty of the devices are being used by healthcare workers across multiple roles and the researchers are working with health records systems leader Epic to explore how they might be used.
By far, the most obvious use is for patient monitoring, including the capacity to monitor multiple patients on wards; but use during surgery, particularly by anesthesiologists, is also an emerging possibility.
It means the surgical team — including that young doctor — can maintain focus on the patient while also seeing all available relevant patient data in the same view.
A tool for therapy
Researchers from Cedars-Sinai are working on Xaia, an AI-powered chatbot therapist for the device. The application offers users an immersive therapy session led by a trained digital avatar, programmed to simulate a human therapist.
Psychiatrist Omar Liran, who co-founded the app, says the application is “a culmination of years of rigorous research, clinical expertise and a vision to democratize mental wellness in a way that respects the uniqueness of every individual’s experience. Using the powerful capabilities of Vision Pro, we’re now able to bring this experience to life with its full potential.”
Virtual reality is already being used widely to help deliver exposure therapy to post-traumatic stress disorder. The idea is that patients are exposed to their PTSD triggers for short periods using XR displays.
Given the eye-tracking inherent to Vision Pro (albeit subject to the privacy controls Apple has in place, which might limit use), it’s possible such therapies could also be optimized.
Medical teaching and planning
Medical publisher Elsevier has introduced an interesting Complete HeartX health app for the device. This provides a new way to link tailored 3D anatomy and physiology with diagnoses and treatments to better prepare students for the real-world pressures of treating patients.
Boston Children’s Hospital has built CyranoHealth, which provides immersive training experience for nurses and other healthcare workers. And Apple has highlighted a couple of other apps, Insight Heart, which lets users understand the human heart, and CellWalk, which takes users on a tour of life’s molecular machines to explore a whole bacteria cell, down to individual atoms.
…And beyond
Unlike the good doctor, we’re just scratching the surface of the possible here. Researchers are getting deep into identifying ways in which Apple’s devices can optimize medical practice.
Medical researchers at Essen University Hospital in Germany are exploring more ways in which the device might be useful, for example in identifying early warning signs of vertigo, stroke, or dementia — can the device analyze eye movements to surface such warnings?
What we can achieve with a new tool as potentially disruptive as Apple’s latest invention will be determined during a marathon, not a sprint — and the race has only now begun.
Please follow me on Mastodon, or join me in the AppleHolic’s bar & grill and Apple Discussions groups on MeWe.