Developing a collaborative approach to data and analytics

A Swinburne research team involved in an international study with almost 200 researchers from across the world is pushing for a more collaborative future in scientific research.

In the unique paper published in Nature, titled "Variability in the analysis of a single neuroimaging dataset by many teams," 70 analysis teams from leading universities and research institutions analyzed the same neuroimaging dataset to test the same hypotheses.

Each team independently analyzed the same brain imaging dataset, collected from 108 participants performing a monetary decision-making task at Tel Aviv University.

The analysis teams were given up to three months to analyze the data, after which they reported final outcomes for the hypotheses as well as detailed information on the way they analyzed the data and intermediate statistical results.

"As the scientific process involves many variables and room for different approaches, there were differences in the outcomes from each team and how they answered the initial research questions," says Dr. Matthew Hughes, Australian National Imaging Fellow and one of the Swinburne research team members.

The study found that about half the tested hypotheses showed consistent results while the other half, varied substantially across research teams. By identifying the sources of discrepancies, this study suggests ways to improve future research.

Researchers require complex methods, big data and detailed analyzes when seeking to understand human behaviors and the physical world. The variability in outcomes demonstrated in this study is due to this complex process when obtaining scientific results.

"The large scale of this project shows the motivation of scientists in the field to improve science via transparency, as they seek to learn more about the brain and cognition," says co-author Dr. David White, Senior Research Fellow at Swinburne's Centre for Human Psychopharmacology.

Understanding the data

The Swinburne team, consisting of Professor Johnstone, Dr. David White and Dr. Matthew Hughes, see open data or 'open science' as extremely important to the future of research.

"I'm very much a proponent of the 'open science' movement that pushes for far more open and transparent presentation of science and making data available for people to reanalyze and check," says Professor Tom Johnstone, Director of Neuroimaging at Swinburne.

He says that while open data is a relatively new concept in many scientific disciplines, the approach should be used more widely.

"The teams that initially report findings, need to be open about what they've found. They need to be comfortable sharing the data so other people can critically evaluate it to help solve global problems."

"Of course there are roadblocks and considerations to be taken into account with sharing of data including privacy, security and accessibility but one big barrier, technology, has been largely overcome thanks to cloud-based storage," he says.

Bringing it to Swinburne

As a member of the Australian National Imaging Facility and the Australian Brain Alliance, Swinburne Neuroimaging is at the forefront of providing innovative methods and infrastructure to support Australian neuroscience researchers in open science and collaborative research.

The Swinburne team plans to take the findings from this international study and use them in the form of workshops, training a new generation of researchers in the use of transparent data analysis pipelines in a range of research areas.