Scientists have created a ‘digital mask’ that will allow facial images to be stored in medical records while preventing potentially sensitive personal biometric information from being extracted and shared.
In research published today in Nature Medicine, a team led by scientists from the University of Cambridge and Sun Yat-sen University in Guangzhou, China, used three-dimensional (3D) reconstruction and deep learning algorithms to erase identifiable features from facial images while retaining disease-relevant features needed for diagnosis.
Facial images can be useful for identifying signs of disease. For example, features such as deep forehead wrinkles and wrinkles around the eyes are significantly associated with coronary heart disease, while abnormal changes in eye movement can indicate poor visual function and visual cognitive developmental problems. However, facial images also inevitably record other biometric information about the patient, including their race, sex, age and mood.
With the increasing digitalisation of medical records comes the risk of data breaches. While most patient data can be anonymised, facial data is more difficult to anonymise while retaining essential information. Common methods, including blurring and cropping identifiable areas, may lose important disease-relevant information, yet even so cannot fully evade face recognition systems.
Due to privacy concerns, people often hesitate to share their medical data for public medical research or electronic health records, hindering the development of digital medical care.
Professor Haotian Lin from Sun Yat-sen University said: “During the COVID-19 pandemic, we had to turn to consultations over the phone or by video link rather than in person. Remote healthcare for eye diseases requires patients to share a large amount of digital facial information. Patients want to know that their potentially sensitive information is secure and that their privacy is protected.”
Professor Lin and colleagues developed a ‘digital mask’, which inputs an original video of a patient’s face and outputs a video based on the use of a deep learning algorithm and 3D reconstruction, while discarding as much of the patient’s personal biometric information as possible — and from which it was not possible to identify the individual.
Deep learning extracts features from different facial parts, while 3D reconstruction automatically digitises the shapes and movement of 3D faces, eyelids, and eyeballs based on the extracted facial features. Converting the digital mask videos back to the original videos is extremely difficult because most of the necessary information is no longer retained in the mask.
Next, the researchers tested how useful the masks were in clinical practice and found that diagnosis using the digital masks was consistent with that carried out using the original videos. This suggests that the reconstruction was precise enough for use in clinical practice.
Compared to the traditional method used to ‘de-identify’ patients — cropping the image — the risk of being identified was significantly lower in the digitally-masked patients. The researchers tested this by showing 12 ophthalmologists digitally-masked or cropped images and asking them to identify the original from five other images. They correctly identified the original from the digitally-masked image in just over a quarter (27%) of cases; for the cropped figure, they were able to do so in the overwhelming majority of cases (91%). This is likely to be an over-estimation, however: in real situations, one would likely have to identify the original image from a much larger set.
The team surveyed randomly selected patients attending clinics to test their attitudes towards digital masks. Over 80% of patients believed the digital mask would alleviate their privacy concerns and they expressed an increased willingness to share their personal information if such a measure was implemented.
Finally, the team confirmed that the digital masks can also evade artificial intelligence-powered facial recognition algorithms.
Professor Patrick Yu-Wai-Man from the University of Cambridge said: “Digital masking offers a pragmatic approach to safeguarding patient privacy while still allowing the information to be useful to clinicians. At the moment, the only options available are crude, but our digital mask is a much more sophisticated tool for anonymising facial images.
“This could make telemedicine — phone and video consultations — much more feasible, making healthcare delivery more efficient. If telemedicine is to be widely adopted, then we need to overcome the barriers and concerns related to privacy protection. Our digital mask is an important step in this direction.”
The research was largely supported by the Science and Technology Planning Projects of Guangdong Province, the National Key R&D Program of China, and the Construction of High-Level Hospitals. Professor Yu-Wai-Man is supported by the UK National Institute for Health and Care Research (NIHR).
