Introduction to Metascience

Dr Gabriel Bennett, the pen name for Dr Matthew Bennett, holds a PhD in Disability Studies from Flinders University, Australia. He has lectured in Disability Studies at Griffith University, Queensland. He has also advised the Australian Government’s Autism CRC and has published articles for the Journal of Autism and Developmental Disorders. He is actively involved in supporting autistics to achieve their potential in society by disseminating his knowledge about the autism spectrum via lectures, conference presentations, and publications.Dr Emma Goodall is an adjunct research fellow at the University of Southern Queensland, Australia, holds a PhD in Education, and is focused on teaching students on the autism spectrum. She is an executive member of the Australian Society for Autism Research, independent researcher through Healthy Possibilities, and a published author and keynote speaker in the areas of autism, sexuality and relationships, education, and interoception.

• 1 An introduction to metascience 11.1 What is metascience? 1.2 The intended audience of this book 1.3 Pedagogical features in this book 1.4 Summary of the upcoming chapters 1.4.1 Chapter 2 – Mitigating biases during the production and dissemination of research 1.4.2 Chapter 3 – Journalology: the science of publishing 1.4.3 Chapter 4 – The impact of funding agencies on the production of research 1.4.4 Chapter 5 – Improving the culture in research workplaces 1.4.5 Chapter 6 – Understanding and addressing QRPs 1.4.6 Chapter 7 – Addressing the reproducibility crisis 1.4.7 Chapter 8 – Ethics and metascience References 2 Mitigating biases during the production and dissemination of research 2.1 Overview of biases in research 2.2 Biases whilst designing a study 2.2.1 Ethnocentric bias 2.2.2 Gender bias 2.3 Biases during data collection and analysis 2.3.1 Cognitive bias 2.3.2 Confirmation bias 2.3.3 Selection bias 2.3.4 Data collection bias 2.3.5 Measurement bias 2.3.6 Search engine bias 2.4 Biases during the reporting and dissemination of research 2.4.1 Time‑lag bias 2.4.2 Place of publication bias 2.4.3 Citation bias 2.4.4 Checklists to detect bias in manuscripts 2.5 Biases in artificial intelligence 2.5.1 The role of artificial intelligence in the creation of literature reviews 2.5.2 Limitations of AI 2.6 Conclusion Additional readings References 3 Journalology: the science of publishing 3.1 A description and historical origins of journalology 3.2 The peer review process 3.3 Issues and potential improvements of the peer review process 3.3.1 Making peer review a teamwork effort 3.3.2 Avoiding publication bias and increasing transparency 3.3.3 Improving the accuracy of the peer review process 3.3.4 Reducing the duration of the peer review process 3.3.5 Other suggestions to improve the peer review process 3.4 Preprinted articles 3.5 The creation of predatory publishers and Beall’s list 3.6 Consequences of predatory journals 3.6.1 Corrupting research 3.6.2 Undermining the training of scholars 3.6.3 Increased email correspondence to academics 3.7 Checklists and flow diagrams to identify predatory journals 3.8 Conclusion Additional readings References 4 Impact of funding agencies on the production of research 4.1 Improving the grant application process 4.1.1 Implementing a two‑stage application process 4.1.2 Overcoming the ‘incumbency advantage’ 4.1.3 Multiple application opportunities to reduce application burden and stress 4.1.4 Improving the quality of feedback that unsuccessful applicants receive 4.2 How funding agencies can improve the quality of research 4.2.1 Improving the evaluation of knowledge translation in research proposals 4.2.2 Addressing sex and gender bias in research 4.2.3 Preventing inappropriate influence by funding agencies 4.2.4 Making research reproducible 4.3 Conclusion Additional readings References 5 Improving equity, diversity, and inclusion in academia 5.1 Academic bullying 5.1.1 Overview of academic bullying 5.1.2 Factors that can cause and exacerbate academic bullying 5.1.3 Strategies to reduce academic bullying 5.2 Racism in research workplaces 5.3 Women in academia 5.3.1 Women’s participation in academia 5.3.2 Factors that inhibit the inclusion and promotion of women in academia 5.3.3 Improving the inclusion, retention, and promotion of women in academia 5.4 General recommendations for improving equity, diversity, and inclusion in academia 5.4.1 Changes to publishing policies 5.4.2 The role of universities and academic institutions 5.4.3 The role of funding agencies 5.5 Conclusion Additional readings References 6 Understanding and addressing questionable research practices 6.1 Defining questionable research practices 6.1.1 Cherry picking 6.1.2 P‑hacking 6.1.3 Hypothesising After Results are Known 6.2 Occurrence of questionable research practices 6.3 Strategies to reduce questionable research practices 6.3.1 Using evidence‑based language 6.3.2 Justifying specific tests for p‑values 6.3.3 Pre‑registering a study’s design 6.3.4 Reforming grant awarding agencies 6.3.5 Educating scholars about questionable research practices 6.3.6 Creating reporting procedures 6.3.7 Reforming the ‘publish or perish’ culture 6.3.8 Removing any financial incentives for academic publishing 6.3.9 Creating an independent research integrity agency 6.3.10 Making researchers pledge an oath to uphold research integrity 6.3.11 Developing a confidential reporting system 6.3.12 Aubert Bonn and colleagues’ suggestions about improving research integrity 6.4 Conclusion Additional readings References 7 Addressing the reproducibility crisis 7.1 Defining reproducibility 7.2 Consequences of irreproducible research 7.3 Strategies to increase reproducible research 7.3.1 Publishing datasets 7.3.2 Establishing journals that only publish replication studies 7.3.3 Teaching academic staff about reproducibility 7.3.4 Open Science Badges 7.3.5 Incorporating reproducibility requirements into the criteria for research funding 7.3.6 Reforming academic hiring practices to promote reproducible research 7.3.7 Pre‑registering studies 7.3.8 Improving the readability of a study’s methodology 7.3.9 Improving the clarity of conference presentations 7.3.10 Requiring researchers to self‑examine their previous research 7.4 Conclusion Additional readings References 8 Human Research Ethics Committees and Metascience 8.1 The creation of Human Research Ethics Committees 8.2 Operational issues with Human Research Ethics Committees 8.2.1 Providing ethics training to applicants 8.2.2 Educating members of Human Research Ethics Committees to examine ethics applications 8.2.3 Interactive ethics presentations 8.2.4 Retrospective ethics reviews 8.2.5 Participant feedback to ethics committees8.2.6 Creating consistent policies for Health Research Ethics Committees 8.2.7 Reducing HREC application rejection rates 8.3 Conclusion Additional readings References 9 Final remarks References