Hololens augmented reality enters the operating theatre at Imperial College

surgery

Hololens augmented reality technology is being used during surgery at Imperial College. We spoke to Dr Philip Pratt about introducing the AR headsets into the operating theatre.

Share

Imperial College London is trialling Microsoft's mixed reality Hololens headsets for skin graft surgeons at St Mary's Hospital, London.

While at first glance, the Microsoft Hololens headset appears identical to a VR headset, this technology in fact creates a ‘mixed’, rather than virtual, reality.

This is where a layer of augmented reality is overlayed on top of the physical world, and can be used to enhance our perception of physical objects. The tech has currently found applications in manufacturing and construction, but is now being used, perhaps to most groundbreaking effect, in medicine.

“We have these incredible machines in the basement of the hospital - the CT and MRI scanners - that can essentially give us pictures of what's inside the patient...We can see all of that on the screen, but the way that information is currently used in the operating theatre is so antiquated,” says Dr Philip Pratt, research fellow at the Department of Surgery and Cancer at Imperial College London.

Now, using the Hololens headsets, Pratt wants to give surgeons access to these images while they are actually performing surgery, starting with skin grafts at St Mary's Hospital, London.

Skin grafts are suitable for the types of injury which result in trauma to the flesh - for example, those resulting from car accidents or severe burns.

To heal the area, a piece of healthy skin is taken from a different part of the body and attached to the damaged area. However, importantly, it must be attached in a location where it can be connected with a blood supply, to ensure the skin does not die.

Therefore, locating the position of major blood vessels under the skin surface is of the utmost importance in this type of surgery. In most hospitals around the world, this form of surgery involves a painstaking process where a sterile ruler is used to map out measurements taken from a scan, making multiple, small incisions on the skin’s surface. Another option is to use an instrument which amplifies blood flow, the Doppler Ultrasound Probe, to try to detect the location of blood vessels.  

“That's a very approximate method and there's no guarantee that where x marks the spot, you've actually got the right place. It's very important that when you make your incisions, you're exactly on top of the vessels of interest - it can take a long time to locate,” says Dr Pratt. “Time in operating theatres is very precious, you've got a lot of surgeons in the room and time is money,” he says, pointing out that it’s also preferable to reduce the amount of time patients need to be kept under general anaesthetic.

Information on the location of blood vessels is available from a CT scan - something that has been around since the ‘70s. However, it has been hard to mirror this two dimensional information onto the patient’s body. Until Hololens, that is.

“Hololens came along and I immediately thought, yes, this is a classic clinical problem that we can use the technology to help solve,” says Dr Pratt. “Because what surgeons want to do is visualise the interior of the patient and overlay that information intraoperatively.”

The technology is currently being used in the hospital for any cases that require lower limb reconstructive surgeries.

The Hololens ability to tether the overlayed images onto physical landmarks means that the accuracy of the image mapping is crucially preserved even when the body part moves. However, as this is the first time the technology has been used in this setting, lower limb surgeries were selected because the large, rigid bone structures in the legs means that there is less movement of the blood vessels when the patient changes position.

With a background in robotic surgery, Dr Pratt himself wrote the code for the software used in the hospital. He points out that to work, he had to make the technology as simple to use as possible.

“[The surgeons] did adapt to it very quickly. It has to be very intuitive in terms of how you manipulate the images, how you align images to the patient. If it takes someone more than you know 30 seconds or a minute to get the hang of it, then you've not done the job properly.” So far, the feedback from surgeons has been overwhelmingly positive. “They're delighted with it,” he says.

So, what could be next for the tech? Some of the immediate future applications also involve surgery where locating particular blood vessels is imperative.

An obvious area where this is true is breast reconstructive surgery, following breast cancer for example. Another potential area is colon or rectum cancers, where an MRI scan identifying the tumour could be projected onto the area during surgery to ensure that all of the malignant tissue is excised. However, as this involves more fluid, flexible structures, it will be more challenging to map imagery onto this area of the body.

However - despite being a pioneer of innovation in the medical sphere - Dr Pratt cautions against over-zealous attitudes towards technology. “We have to go first to the problem and not the solution,” he says. “It's not about taking a piece of technology and finding something to use it for. Instead, we have to start with a problem and then find the most appropriate technology to help it.”

Right now, the technology can only be used by hospitals associated with Imperial College, including St Mary's, Charing Cross and Hammersmith Hospitals. However, Dr Pratt is focused on increasing the number of case studies conducted using the technology, with the goal of eventually having the technology rolled out to every hospital in the UK.

This will involve a laborious, regulatory process before it can become a standardised tool across the NHS. But this aim remains firmly within the sights of the tech.

“My driving motivation behind this is to see it used as often and as geographically widespread as possible," says Dr Pratt. "I want it to be not just in one hospital or 10 hospitals, but 100 or 1,000 hospitals. That's definitely part of the grand scheme.”