A HoloLens Framework for Augmented Reality Applications in Breast Cancer Surgery

Abstract

This project aims to support oncologic breast-conserving surgery by creating a platform for better surgical planning through the development of a framework that is capable of displaying a virtual model of the tumour(s) requiring surgery, on a patient's breast. Breast-conserving surgery is the first clear option when it comes to tackling cases of breast cancer, but the surgery comes with risks. The surgeon wants to maintain clean margins while performing the procedure such that the disease does not resurface. This calls for the importance of surgical planning where the surgeon consults with radiologists and pre-surgical imaging such as Magnetic Resonance Imaging (MRI). The MRI prior to the surgical procedure, however, is taken with the patient in the prone position (face-down) but the surgery happens in a supine position (face-up). Thus mapping the location of the tumour(s) to the corresponding anatomical position from the MRI is a tedious task which requires a large amount of expertise and time given that the organ is soft and flexible.

For this project, the tumour is visualized in the corresponding anatomical position to assist in surgical planning. Augmented Reality is the best option for this problem and this, in turn, led to an investigation of the application capability of the Microsoft HoloLens to solve this problem. Given its multitude of sensors and resolution of display the device is a fine candidate for this process. However, the HoloLens is still under development with a large number of limitations in its use. This work tries to compensate for these limitations using the existing hardware and software in the device's arsenal.

Within this masters thesis, the principal questions answered are related to the acquiring of data from breast mimicking objects in acceptable resolutions, discriminating between the information based on photometry, offloading the data to a computer for post-processing in creating a correspondence between the MRI data and acquired data, and finally retrieving the processed information such that the MRI information can be used for visualizing the tumor in the anatomically precise position. Unfortunately, time limitations for this project led to an incomplete system which is not completely synchronized, however, our work has solidified the grounds for the software aspects toward the final goals set out such that extensive exploration need only be done in the imaging side of this problem.