Birmingham City University’s Digital Media Technology Lab is working on a mixed reality system that allows medical practitioners to view and interact with virtual models of organs and bones, and also change models of patient data. This augmented reality technology will eventually assist medics in their pre-operative assessments, as well as providing new learning models for medical students.
The basis of the system is advanced motion detecting sensors plus new developments with freehand interaction and the application of mixed reality software. It allows users to manipulate, navigate, and demonstrate patient data using hand motions and gestures, in order to assess medical procedures, lifestyle choices, and treatment effects using customized 3D virtual models and patients’ real medical records. Importantly, the augmented reality system can also tell medics what might happen if procedure B is selected over procedure A; or if treatment 2 was used in place of treatment.
In future, it’s expected that such systems will help to increase patients’ engagement about their treatments and their understanding of how lifestyle choices or medical procedures could impact their health.
Dr. Ian Williams, Associate Professor and Subject leader in Image and Video technology within the DMT Lab, said: “We are developing this system as a platform to allow medical professionals to interact with genuine patient data and manipulate it by hand to educate and inform patients. The real advantages this brings are being able to visually demonstrate parts of the anatomy, using virtual models which can be customized for each patient and show how they have been impacted by lifestyle choices or how they may be changed following treatments or surgery.”