Tilburg School of Social and Behavioral Sciences

Understanding the sociology and psychology of human actions

Department Methodology and Statistics

The Tilburg 'Truth Squad' in Science

The prestigious scientific journal Science will be paying tribute to the research group around Tilburg methodologists Jelte Wicherts and Marcel van Assen in its special issue on metaresearch this week. Under the heading ‘The Truth Squad’ , the entire Tilburg metaresearch group, part of TSB’s Department of Methodology & Statistics, will be put in the spotlight. So what do they do, why, and how?

"In its drive to expose weaknesses in science, an up-and-coming research group doesn't mind stepping on some toes."

Last year, three PhD students from the 'Squad' in particular stood out with their metafindings: Michèle Nuijten, Robbie van Aert, and Paulette Flore. Research on research. The ‘Truth Squad’ now consists of a dozen researchers.

Statcheck

An algorithm called Statcheck was developed, for instance, to discover possible statistical errors in psychology research. Statcheck was applied to tens of thousands of papers and the disturbing results were placed on PubPeer, an online platform where scientific publications are discussed. That revelation did not go down all that well in some places

Weaknesses

Professor Jelte Wicherts, who started the metaresearch group, wondered why smart researchers, for instance, psychologist Diederik Stapel, who admitted to having fabricated data for many of his research publications, “do stupid things with statistics”. Metaresearchers investigate how scholars operate and how they might have gotten off the rails. Wicherts identified approaches in publications that would not pass methodological muster. He decided to study this phenomenon in order to raise awareness about methodological weaknesses in academic research and to eliminate them.

Robuust

The Tilburg metaresearch team’s mission is thus to improve and advance science. By critically examining problems, they want to help scientists conduct research in more robust ways. Thanks to a € 2 million grant of the European Research Council (ERC), the group can built software to help researchers to analyse data with a lower risk of bias.

Further reading

  • Erik Stokstad: The Truth Squad, in: Science, September 21, 2018
  • The Tilburg University Metaresearch group
  • Jop de Vrieze: Meta-analyses were supposed to end scientific debates. Often, they only cause more controversy, in: Science, September 18, 2018
  • Robbie van Aert (cum laude dissertatie) cesigns new methods to analyse publication mountain, Tilburg University press release (in Dutch) 
  • How to improve psychological science? press release on Michèle Nuijten's PhD Defence 
  • Crisis in de psychologie: statistische inconsistenties, te kleine steekproeven en overschatte effecten, Press release on Michèle Nuijten's PhD thesis (in Dutch)
  • Psychologisch onderzoek kan betrouwbaarder, interview with Nuijten in Univers (in Dutch)
  • Grants and Awards of the Methodology and Statistics Department