AI Can Identify Your Race By Looking At X-Rays, And Scientists Have No Clue Why
Photo 209238243 © Andrii Klapko | Dreamstime.com
Artificial intelligence has gone beyond what’s detectable by the human eye. Apart from spotting cells or cancers that doctors might miss out on, algorithms can now apparently guess a person’s race—to remarkable accuracy—by studying medical scans. Weirdly enough, scientists can’t explain why.
That’s according to a new research paper by international doctors and computer scientists, detailing that when shown X-rays, CT, scans, mammograms, and other forms of medical imagery portraying Black, white, and Asian individuals, AI systems could predict a person’s self-reported race almost perfectly. In fact, The Register reports that the best system could predict a patient’s race 99% of the time, while the worst-performing one was 80% accurate. The project has yet to be peer-reviewed at the time of publication.
The algorithms were first trained with datasets of images labeled with the patients’ self-reported races. Then, when tasked to identify new, unlabeled health scans, they did it anyway—a feat that surprised the researchers, since radiologists don’t consider a patient’s racial identity when looking at scans.
The AI could pinpoint races through pictures that look edbelow the skin, even when the scans were manipulated to the point that features could not be identified by the human eye. And that’s worrying, because there could be signs of racial bias in there that scientists cannot detect.
“That means that we would not be able to mitigate the bias,” Motherboard hears from Dr Judy Gichoya, a radiologist at Emory University and co-author of the study. She adds: “If these models are starting to learn these properties, then whatever we do in terms of systemic racism… will naturally populate to the algorithm.”
In the past, scientists were able to uncover the sources for racial bias in medical algorithms, such as when an AI under-reported how sick Black patients were due to how much lower hospitals were spending to treat Black patients.
This time around, when the team examined breast tissue scans for biological dissimilarities, as well as reviewed the images for quality differences, the members were unable to link the imagery to race.
A result like this doubles down the importance of scrutinizing health algorithms such that they prove to be fair for everyone. And with the US Food and Drug Administration (FDA) looking to work with a wider selection of AI medical devices, this is especially crucial.
Motherboard reports that the FDA currently only relies on AI healthcare devices that are trained on a fixed set of data and aren’t intended to take in more. In due course, though, it could look into technology with non-fixed algorithms.
[via VICE and The Register, cover photo 209238243 © Andrii Klapko | Dreamstime.com]
Also check out these recent news
