Studies Relying on Brain Scans Are Often Unreliable, Analysis Shows

Most brain studies that rely on MRI scans don’t include enough people to provide trustworthy results, researchers say.

These brain-wide association studies use MRI (magnetic resonance imaging) to see how brain structure and function connect with personality, behavior, thinking, neurological conditions and mental illness.

Such studies require thousands of participants to get accurate results, but typically enroll just a few dozen people, according to the authors of a report published March 16 in the journal Nature.

These “underpowered” studies can show strong but incorrect associations by chance while missing real but weaker ones, the researchers explained.

“For decades, we’ve been highlighting the potential for MRI to assist in the clinical care — including the diagnosis, risk, response to treatment, etc. — for mental health disorders and neurologic conditions. However, the full potential has not been realized,” said senior author Damien Fair. He is director of the Masonic Institute for the Developing Brain at the University of Minnesota in Minneapolis.

“We now know our missteps and are redefining the required parameters, the so-called ‘special sauce,’ to move forward effectively,” Fair added in a university news release.

For the study, the researchers analyzed data from nearly 50,000 participants in brain-wide association studies. In those studies, the median sample size was 25 — meaning half had fewer participants, half had more.

Fair’s team found that associations identified in such a small sample usually failed to replicate in a separate one.

Only as the sample size rose into the thousands were associations more likely to be replicated.

Senior study author Dr. Nico Dosenbach told The New York Times that enlisting study participants can be time-consuming and costly, ranging from $600 to $2,000 an hour. Studies that use MRI often include a note about the small size.

“It’s not a problem with any individual researcher or study. It’s not even unique to neuroimaging,” said Dosenbach, an associate professor of neurology at Washington University in St. Louis. “The field of genomics discovered a similar problem about a decade ago with genomic data and they took steps to address it.”

Dosenbach noted that the U.S. National Institutes of Health began funding larger data-collection efforts and requiring data to be shared publicly. That reduces bias, he said, and genome science has gotten much better as a result.

“Sometimes you just have to change the research paradigm,” Dosenbach said. “Genomics has shown us the way.”

If all data from multiple small studies were pooled and analyzed together, the result would likely be reliable, he noted.

Fair said the future rests in sharing data and resources across institutions in order to make large data sets available to any scientist who wants to use them.

“This very paper is an amazing example of that,” Fair said.

More information

There’s more on MRI at the American Academy of Family Physicians.

SOURCE: University of Minnesota, news release, March 16, 2022

Source: HealthDay