Department Methodology and Statistics
The Tilburg 'Truth Squad' in Science
The prestigious scientific journal Science will be paying tribute to the research group around Tilburg methodologists Jelte Wicherts and Marcel van Assen in its special issue on metaresearch this week. Under the heading ‘The Truth Squad’ , the entire Tilburg metaresearch group, part of TSB’s Department of Methodology & Statistics, will be put in the spotlight. So what do they do, why, and how?
"In its drive to expose weaknesses in science, an up-and-coming research group doesn't mind stepping on some toes."
Last year, three PhD students from the 'Squad' in particular stood out with their metafindings: Michèle Nuijten, Robbie van Aert, and Paulette Flore. Research on research. The ‘Truth Squad’ now consists of a dozen researchers.
An algorithm called Statcheck was developed, for instance, to discover possible statistical errors in psychology research. Statcheck was applied to tens of thousands of papers and the disturbing results were placed on PubPeer, an online platform where scientific publications are discussed. That revelation did not go down all that well in some places
Professor Jelte Wicherts, who started the metaresearch group, wondered why smart researchers, for instance, psychologist Diederik Stapel, who admitted to having fabricated data for many of his research publications, “do stupid things with statistics”. Metaresearchers investigate how scholars operate and how they might have gotten off the rails. Wicherts identified approaches in publications that would not pass methodological muster. He decided to study this phenomenon in order to raise awareness about methodological weaknesses in academic research and to eliminate them.
The Tilburg metaresearch team’s mission is thus to improve and advance science. By critically examining problems, they want to help scientists conduct research in more robust ways. Thanks to a € 2 million grant of the European Research Council (ERC), the group can built software to help researchers to analyse data with a lower risk of bias.
- Erik Stokstad: The Truth Squad, in: Science, September 21, 2018
- The Tilburg University Metaresearch group
- Jop de Vrieze: Meta-analyses were supposed to end scientific debates. Often, they only cause more controversy, in: Science, September 18, 2018
- Robbie van Aert (cum laude dissertatie) cesigns new methods to analyse publication mountain, Tilburg University press release
- How to improve psychological science? press release on Michèle Nuijten's PhD Defence
- Crisis in de psychologie: statistische inconsistenties, te kleine steekproeven en overschatte effecten, Press release on Michèle Nuijten's PhD thesis
- Psychologisch onderzoek kan betrouwbaarder, interview with Nuijten in Univers (in Dutch)
- Grants and Awards of the Methodology and Statistics Department