Is AR/MR/XR finally ready for surgery?
“Is Augmented Reality finally ready for surgery?”
This is a question I’d ask myself, time and again, ever since I saw a holography setup at SIGGRAPH 2000 in New Orleans. I still remember how I stood there in an awe (and repeating the stance a few more times during multiple follow up visits). (To put things in context, I was just a graduate student with only raw, impractical knowledge of Computer Graphics, and zero industrial experience).
A few 3D imaging, visualization, and image guidance experiences later, I met Dr. Salil Joshi (a practicing Interventional Radiologist) through a common friend. From the very first conversation, he would not stop bringing up the need for simple solutions to his everyday clinical problems. He would talk about how every other aspect of medicine was making leaps of progress (surgical robotics, precision medicine, gene therapy, nano-medicines …) but nothing really fundamentally had changed in interventional guidance space; and how he was still using the same 2D (sometimes 3D) information to make life critical decisions about his patients with no other resort but to keep trying the same tools again and again until an outcome was reached. With all the humility and knowledge I could gather from state of the art in alternatives, I had nothing but excuses for him — “Can’t do that. This won’t show soft tissue. That isn’t going to be feasible etc. etc.” We then lost contact, and I got busy helping a few notable advanced device missions in Surgical Robotics and Radio Surgery.
Then came a flurry of Augmented Reality and Mixed Reality product announcements that for the first time looked like will actually add value to our tech-consumed lives. As a hobby, I would try them all, one after the other — though none of them stood out (the lighter ones had very basic functionality, and the good ones too cumbersome to use for more than a few seconds).
Around the same time, a startup (Magic Leap) from part of the team that had kick started my career in medical imaging (Z-Kat, that later became MAKO Surgical) gained traction; and Microsoft’s HoloLens came out. While we could only see specs and videos of the latter, the promise was huge!
That was the sign I needed to drop everything else I was doing, call up Dr. Joshi and for the first time in almost a decade, say:
“I think we can make your life better.”
This time it was his turn to ask questions (“but what about this, and how will you do that?”). In the 2 months that followed, I setup WortheeMed, and started to sketch solutions using AR for a specific problem he was facing every day. Eventually, one of the dozen ideas stuck and he agreed to become the lead investor, and a clinical adviser. Our then-winning solution had answers for everything, but one remaining challenge before it would prove that AR can be repeatably and credibly useful in an interventional suite not just as a visualization aide, but as a decision-critical modality (as anyone conversant in both medicine and AR would know, it is not enough just to overlay contextual data on the subject site. The ‘mixing’ has to be pin-point-precise, and dynamically responsive to soft tissue changes for a surgeon or an interventionalist to trust ‘it’ with a life critical task).
Since then, we believe we have found solutions to all the major ‘what-if’s (a combination of live sensing, imaging, and precision high performance AR), have come a decent mile from the starting post, and are now evaluating a functional prototype of an iteration of the originally envisioned solution. We have been pleasantly encouraged by what a small team and some crazy ambition can achieve. But most importantly, we know this much now: that if we face a hiccup that will make us fall flat on our faces, and the mission does not see fruition, chances are it will not be because AR technology is not ready; it will be because of other unforeseen forces.
We believe it is time for Augmented (or Mixed) reality to add real value to our lives way beyond all the consumer and entertainment applications. And even though the market potential may not be comparable, and the perception of risk may be higher in medical device R&D, we are willing to bet everything we have ever known, earned, and built on XR, and it’s future in changing the way we deliver care — regardless of whether it is our team, or another, that brings it to fruition. For even an inch forward now, might someday enable an extra-terrestrial leap into the future of medicine through a network collective of industry leaders and entrepreneurs working together.
So, what do you think? Are you ready, and excited for the future of portable, bedside care? Please share your comments below :-)
P.S. Many many friends, mentors, leaders, and kind strangers’ time shared with us has brought us to this post, and to where we are with WortheeMed. While it is not possible to list every name here (including the first VCs, co-entrepreneurs, business leaders, analysts, recruiters, children, artists, and brave physicians), please know that you have all inspired us in part to push forward and continually fail until the only way left is up.
