Export file:

Format

  • RIS(for EndNote,Reference Manager,ProCite)
  • BibTex
  • Text

Content

  • Citation Only
  • Citation and Abstract

Instead of “playing the game” it is time to change the rules: Registered Reports at AIMS Neuroscience and beyond

1 Cardiff University Brain Research Imaging Centre, School of Psychology, Cardiff University;
2 School of Psychology and Clinical Language Sciences, University of Reading;
3 School of Society, Enterprise and Environment, Bath Spa University

  Figure/Table
  Supplementary
  Article Metrics

Citation: Christopher D. Chambers, Eva Feredoes, Suresh D. Muthukumaraswamy, Peter J. Etchells. Instead of “playing the game” it is time to change the rules: Registered Reports at AIMS Neuroscience and beyond. AIMS Neuroscience, 2014, 1(1): 4-17. doi: 10.3934/Neuroscience.2014.1.4

References

  • 1. Ioannidis JPA. (2005) Why Most Published Research Findings Are False. PLoS Med 2: e124.    
  • 2. John LK, Loewenstein G, Prelec D. (2012) Measuring the prevalence of questionable research practices with incentives for truth telling. Psychol Sci 23: 524-532.    
  • 3. Simmons JP, Nelson LD, Simonsohn U. (2011) False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol Sci 22: 359-366.
  • 4. Kerr NL. (1998) HARKing: hypothesizing after the results are known. Pers Soc Psychol Rev 2: 196-217.    
  • 5. Makel MC, Plucker JA, Hegarty B. (2012) Replications in Psychology Research: How Often Do They Really Occur? Perspect Psychol Sci 7: 537-542.    
  • 6. Faneli D. (2010) “Positive” Results Increase Down the Hierarchy of the Sciences. PLos One 5: e10068.    
  • 7. Button KS, Ioannidis JP, Mokrysz C, Nosek BA, Flint J, et al. (2013) Power failure: why small sample size undermines the reliability of neuroscience. Nat Rev Neurosci 14: 365-376.    
  • 8. Wicherts JM, Bakker M, Molenaar D. (2011) Willingness to share research data is related to the strength of the evidence and the quality of reporting of statistical results. PLoS One 6: e26828.    
  • 9. Cohen J. (1962) The statistical power of abnormal-social psychological research: a review. J Abnorm Soc Psychol 65: 145-153.    
  • 10. Sterling TD. (1959) Publication Decisions and their Possible Effects on Inferences Drawn from Tests of Significance—or Vice Versa. J Am Stat Assoc 54: 30-34.
  • 11. de Groot AD. (2014) The meaning of "significance" for different types of research [translated and annotated by Eric-Jan Wagenmakers, Denny Borsboom, Josine Verhagen, Rogier Kievit, Marjan Bakker, Angelique Cramer, Dora Matzke, Don Mellenbergh, and Han L. J. van der Maas]. Acta Psychol (Amst) 148: 188-194.    
  • 12. Nosek BA, Spies JR, Motyl M. (2012) Scientific Utopia : II. Restructuring Incentives and Practices to Promote Truth Over Publishability. Perspect Psychol Sci 7: 615-631.
  • 13. Ioannidis JPA. (2012) Why Science Is Not Necessarily Self-Correcting. Perspect Psychol Sci 7: 645-654.    
  • 14. Chambers CD. (2013) Registered reports: a new publishing initiative at Cortex. Cortex 49: 609-610.    
  • 15. Wolfe J. (2013) Registered Reports and Replications in Attention, Perception, & Psychophysics. Atten Percept Psycho 75: 781-783.    
  • 16. Stahl C. (2014) Experimental psychology: toward reproducible research. Exp Psychol 61: 1-2.    
  • 17. Munafo MR, Strain E. (2014) Registered Reports: A new submission format at Drug and Alcohol Dependence. Drug Alcohol Depend 137: 1-2.    
  • 18. Nosek BA, Lakens D. (in press) Registered reports: A method to increase the credibility of published results. Soc Psychol.
  • 19. Chambers CD, Munafo MR. (2013) Trust in science would be improved by study pre-registration. The Guardian:http://www. theguardian. com/science/blog/2013/jun/2005/trust-in-science-study-pre-registration.
  • 20. Scott SK. (2013) Will pre-registration of studies be good for psychology? : https://sites. google. com/site/speechskscott/SpeakingOut/willpre-registrationofstudiesbegoodforpsychology.
  • 21. Scott SK. (2013) Pre-registration would put science in chains. Times Higher Education: http://www. timeshighereducation. co. uk/comment/opinion/science-in-chains/2005954. article.
  • 22. Rouder J, Speckman P, Sun D, Morey R, Iverson G. (2009) Bayesian t tests for accepting and rejecting the null hypothesis. Psychon Bull Rev 16: 225-237.    
  • 23. Wagenmakers EJ. (2007) A practical solution to the pervasive problems of p values. Psychon Bull Rev 14: 779-804.    
  • 24. Dienes Z. (2011) Bayesian Versus Orthodox Statistics: Which Side Are You On? Perspect Psychol Sci 6: 274-290.    
  • 25. Mathieu S, Chan AW, Ravaud P. (2013) Use of trial register information during the peer review process. PLoS One 8: e59910.    
  • 26. Gelman A, Loken E. (2014) The garden of forking paths: Why multiple comparisons can be a problem, even when there is no fishing expedition" or "p-hacking" and the research hypothesis was posited ahead of time. Unpublished manuscript: http://www. stat. columbia. edu/~gelman/research/unpublished/p_hacking. pdf.
  • 27. Strube MJ. (2006) SNOOP:a program for demonstrating the consequences of premature and repeated null hypothesis testing. Behav Res Methods 38: 24-27.    
  • 28. Fiedler K, Kutzner F, Krueger JI. (2012) The Long Way From α-Error Control to Validity Proper: Problems With a Short-Sighted False-Positive Debate. Perspect Psychol Sci 7: 661-669.    
  • 29. Whelan R, Conrod PJ, Poline JB, Lourdusamy A, Banaschewski T, et al. (2012) Adolescent impulsivity phenotypes characterized by distinct brain networks. Nat Neurosci 15: 920-925.    
  • 30. Brembs B, Button K, Munafo M. (2013) Deep impact: unintended consequences of journal rank. Front Hum Neurosci 7: 291.
  • 31. Nelson LD. (2014) Preregistration: Not just for the Empiro-zealots. http://datacoladaorg/2014/01/07/12-preregistration-not-just-for-the-empiro-zealots/.
  • 32. World Medical A (2013) World medical association declaration of helsinki: Ethical principles for medical research involving human subjects. JAMA 310: 2191-2194.    

 

This article has been cited by

  • 1. Susan Reh, Niels Van Quaquebeke, Steffen R. Giessner, The aura of charisma: A review on the embodiment perspective as signaling, The Leadership Quarterly, 2017, 10.1016/j.leaqua.2017.01.001
  • 2. Krzysztof J. Gorgolewski, Russell A. Poldrack, A Practical Guide for Improving Transparency and Reproducibility in Neuroimaging Research, PLOS Biology, 2016, 14, 7, e1002506, 10.1371/journal.pbio.1002506
  • 3. Nils Muhlert, Gerard R. Ridgway, Failed replications, contributing factors and careful interpretations: Commentary on Boekel et al., 2015, Cortex, 2016, 74, 338, 10.1016/j.cortex.2015.02.019
  • 4. Zoltan Dienes, How Bayes factors change scientific practice, Journal of Mathematical Psychology, 2016, 72, 78, 10.1016/j.jmp.2015.10.003
  • 5. Angela S. Attwood, Marcus R. Munafò, Navigating an open road, Journal of Clinical Epidemiology, 2016, 70, 264, 10.1016/j.jclinepi.2015.04.016
  • 6. Karen Niven, Luke Boorman, Assumptions beyond the science: encouraging cautious conclusions about functional magnetic resonance imaging research on organizational behavior, Journal of Organizational Behavior, 2016, 37, 8, 1150, 10.1002/job.2097
  • 7. M.J. Bayarri, Daniel J. Benjamin, James O. Berger, Thomas M. Sellke, Rejection odds and rejection ratios: A proposal for statistical practice in testing hypotheses, Journal of Mathematical Psychology, 2016, 72, 90, 10.1016/j.jmp.2015.12.007
  • 8. Daniël Lakens, Joe Hilgard, Janneke Staaks, On the reproducibility of meta-analyses: six practical recommendations, BMC Psychology, 2016, 4, 1, 10.1186/s40359-016-0126-3
  • 9. Winston Lin, Donald P. Green, Standard Operating Procedures: A Safety Net for Pre-Analysis Plans, PS: Political Science & Politics, 2016, 49, 03, 495, 10.1017/S1049096516000810
  • 10. Anna Elisabeth van 't Veer, Roger Giner-Sorolla, Pre-registration in social psychology—A discussion and suggested template, Journal of Experimental Social Psychology, 2016, 67, 2, 10.1016/j.jesp.2016.03.004
  • 11. Russell A. Poldrack, Chris I. Baker, Joke Durnez, Krzysztof J. Gorgolewski, Paul M. Matthews, Marcus R. Munafò, Thomas E. Nichols, Jean-Baptiste Poline, Edward Vul, Tal Yarkoni, Scanning the horizon: towards transparent and reproducible neuroimaging research, Nature Reviews Neuroscience, 2017, 18, 2, 115, 10.1038/nrn.2016.167
  • 12. , Promoting reproducibility with registered reports, Nature Human Behaviour, 2017, 1, 1, 0034, 10.1038/s41562-016-0034
  • 13. Bryan G. Cook, Reforms in Academic Publishing: Should Behavioral Disorders and Special Education Journals Embrace Them?, Behavioral Disorders, 2016, 41, 3, 161, 10.17988/0198-7429-41.3.161
  • 14. Jonathan P. Tennant, Jonathan M. Dugan, Daniel Graziotin, Damien C. Jacques, François Waldner, Daniel Mietchen, Yehia Elkhatib, Lauren B. Collister, Christina K. Pikas, Tom Crick, Paola Masuzzo, Anthony Caravaggi, Devin R. Berg, Kyle E. Niemeyer, Tony Ross-Hellauer, Sara Mannheimer, Lillian Rigling, Daniel S. Katz, Bastian Greshake Tzovaras, Josmel Pacheco-Mendoza, Nazeefa Fatima, Marta Poblet, Marios Isaakidis, Dasapta Erwin Irawan, Sébastien Renaut, Christopher R. Madan, Lisa Matthias, Jesper Nørgaard Kjær, Daniel Paul O'Donnell, Cameron Neylon, Sarah Kearns, Manojkumar Selvaraju, Julien Colomb, A multi-disciplinary perspective on emergent and future innovations in peer review, F1000Research, 2017, 6, 1151, 10.12688/f1000research.12037.1
  • 15. Dennis M. Gorman, The decline effect in evaluations of the Strengthening Families Program for Youth 10-14 (SFP 10-14) on adolescent substance use, Children and Youth Services Review, 2017, 10.1016/j.childyouth.2017.07.009
  • 16. Tom E. Hardwicke, Leila Jameel, Matthew Jones, Eryk J. Walczak, Lucía Magis-Weinberg, Only Human: Scientists, Systems, and Suspect Statistics, Opticon1826, 2014, 16, 10.5334/opt.ch
  • 17. Christoph Stahl, Experimental Psychology, Experimental Psychology, 2015, 62, 1, 1, 10.1027/1618-3169/a000289
  • 18. Gregory Francis, Talis Bachmann, Registered reports for Consciousness and Cognition, Consciousness and Cognition, 2017, 10.1016/j.concog.2017.10.007
  • 19. Dennis M. Gorman, Amber D. Elkins, Mark Lawley, A Systems Approach to Understanding and Improving Research Integrity, Science and Engineering Ethics, 2017, 10.1007/s11948-017-9986-z
  • 20. Hunter Gehlbach, Carly D. Robinson, Mitigating Illusory Results through Pre-Registration in Education, Journal of Research on Educational Effectiveness, 2017, 00, 10.1080/19345747.2017.1387950
  • 21. Jonathan P. Tennant, Jonathan M. Dugan, Daniel Graziotin, Damien C. Jacques, François Waldner, Daniel Mietchen, Yehia Elkhatib, Lauren B. Collister, Christina K. Pikas, Tom Crick, Paola Masuzzo, Anthony Caravaggi, Devin R. Berg, Kyle E. Niemeyer, Tony Ross-Hellauer, Sara Mannheimer, Lillian Rigling, Daniel S. Katz, Bastian Greshake Tzovaras, Josmel Pacheco-Mendoza, Nazeefa Fatima, Marta Poblet, Marios Isaakidis, Dasapta Erwin Irawan, Sébastien Renaut, Christopher R. Madan, Lisa Matthias, Jesper Nørgaard Kjær, Daniel Paul O'Donnell, Cameron Neylon, Sarah Kearns, Manojkumar Selvaraju, Julien Colomb, A multi-disciplinary perspective on emergent and future innovations in peer review, F1000Research, 2017, 6, 1151, 10.12688/f1000research.12037.2
  • 22. Michael Noetel, Joseph Ciarrochi, Brooke Van Zanden, Chris Lonsdale, Mindfulness and acceptance approaches to sporting performance enhancement: a systematic review, International Review of Sport and Exercise Psychology, 2017, 1, 10.1080/1750984X.2017.1387803
  • 23. Jake Bernards, Kimitake Sato, G. Haff, Caleb Bazyler, Current Research and Statistical Practices in Sport Science and a Need for Change, Sports, 2017, 5, 4, 87, 10.3390/sports5040087
  • 24. Jonathan P. Tennant, Jonathan M. Dugan, Daniel Graziotin, Damien C. Jacques, François Waldner, Daniel Mietchen, Yehia Elkhatib, Lauren B. Collister, Christina K. Pikas, Tom Crick, Paola Masuzzo, Anthony Caravaggi, Devin R. Berg, Kyle E. Niemeyer, Tony Ross-Hellauer, Sara Mannheimer, Lillian Rigling, Daniel S. Katz, Bastian Greshake Tzovaras, Josmel Pacheco-Mendoza, Nazeefa Fatima, Marta Poblet, Marios Isaakidis, Dasapta Erwin Irawan, Sébastien Renaut, Christopher R. Madan, Lisa Matthias, Jesper Nørgaard Kjær, Daniel Paul O'Donnell, Cameron Neylon, Sarah Kearns, Manojkumar Selvaraju, Julien Colomb, A multi-disciplinary perspective on emergent and future innovations in peer review, F1000Research, 2017, 6, 1151, 10.12688/f1000research.12037.3
  • 25. Dominique Makowski, The psycho Package: an Efficient and Publishing-Oriented Workflow for Psychological Science, The Journal of Open Source Software, 2018, 3, 22, 470, 10.21105/joss.00470
  • 26. Israel Halperin, Andrew D. Vigotsky, Carl Foster, David B. Pyne, Strengthening the Practice of Exercise and Sport-Science Research, International Journal of Sports Physiology and Performance, 2018, 1, 10.1123/ijspp.2017-0322
  • 27. Emma Marsden, Kara Morgan-Short, Pavel Trofimovich, Nick C. Ellis, Introducing Registered Reports at Language Learning: Promoting Transparency, Replication, and a Synthetic Ethic in the Language Sciences, Language Learning, 2018, 10.1111/lang.12284
  • 28. Brian A. Nosek, Charles R. Ebersole, Alexander C. DeHaven, David T. Mellor, The preregistration revolution, Proceedings of the National Academy of Sciences, 2018, 115, 11, 2600, 10.1073/pnas.1708274114
  • 29. Barbara A. Spellman, Elizabeth A. Gilbert, Katherine S. Corker, , Stevens' Handbook of Experimental Psychology and Cognitive Neuroscience, 2018, 1, 10.1002/9781119170174.epcn519
  • 30. Jan Lauwereyns, , Rethinking the Three R's in Animal Research, 2018, Chapter 5, 103, 10.1007/978-3-319-89300-6_5
  • 31. Harlan Campbell, Paul Gustafson, Daniele Marinazzo, Conditional equivalence testing: An alternative remedy for publication bias, PLOS ONE, 2018, 13, 4, e0195145, 10.1371/journal.pone.0195145
  • 32. ROBERT BLOOMFIELD, KRISTINA RENNEKAMP, BLAKE STEENHOVEN, No System Is Perfect: Understanding How Registration-Based Editorial Processes Affect Reproducibility and Investment in Research Quality, Journal of Accounting Research, 2018, 56, 2, 313, 10.1111/1475-679X.12208
  • 33. James A. Grand, Steven G. Rogelberg, George C. Banks, Ronald S. Landis, Scott Tonidandel, From Outcome to Process Focus: Fostering a More Robust Psychological Science Through Registered Reports and Results-Blind Reviewing, Perspectives on Psychological Science, 2018, 13, 4, 448, 10.1177/1745691618767883
  • 34. C. J. Brainerd, Valerie F. Reyna, Replication, Registration, and Scientific Creativity, Perspectives on Psychological Science, 2018, 13, 4, 428, 10.1177/1745691617739421
  • 35. Haley M. Woznyj, Kelcie Grenier, Roxanne Ross, George C. Banks, Steven G. Rogelberg, Results-blind review: a masked crusader for science, European Journal of Work and Organizational Psychology, 2018, 1, 10.1080/1359432X.2018.1496081
  • 36. Saul Albert, J. P. De Ruiter, Improving Human Interaction Research through Ecological Grounding, Collabra: Psychology, 2018, 4, 1, 24, 10.1525/collabra.132
  • 37. Eric Robinson, Kirsten E. Bevelander, Matt Field, Andrew Jones, Comments on methodological and reporting quality in laboratory studies of human eating behaviour, Appetite, 2018, 10.1016/j.appet.2018.07.001
  • 38. Felix D. Schönbrodt, Markus Maier, Moritz Heene, Markus Bühner, Forschungstransparenz als hohes wissenschaftliches Gut stärken, Psychologische Rundschau, 2018, 69, 1, 37, 10.1026/0033-3042/a000386
  • 39. Bryan G. Cook, John Wills Lloyd, David Mellor, Brian A. Nosek, William J. Therrien, Promoting Open Science to Increase the Trustworthiness of Evidence in Special Education, Exceptional Children, 2018, 001440291879313, 10.1177/0014402918793138
  • 40. Samuel Greiff, Mark S. Allen, EJPA Introduces Registered Reports as New Submission Format, European Journal of Psychological Assessment, 2018, 34, 4, 217, 10.1027/1015-5759/a000492
  • 41. Garret Christensen, Edward Miguel, Transparency, Reproducibility, and the Credibility of Economics Research, Journal of Economic Literature, 2018, 56, 3, 920, 10.1257/jel.20171350
  • 42. S. P. J. M. Horbach, W. Halffman, The changing forms and expectations of peer review, Research Integrity and Peer Review, 2018, 3, 1, 10.1186/s41073-018-0051-5
  • 43. Tom E. Hardwicke, John P. A. Ioannidis, Mapping the universe of registered reports, Nature Human Behaviour, 2018, 10.1038/s41562-018-0444-y
  • 44. S. P. J. M. Horbach, W. Halffman, The ability of different peer review procedures to flag problematic publications, Scientometrics, 2018, 10.1007/s11192-018-2969-2
  • 45. Eric M. Prager, Karen E. Chambers, Joshua L. Plotkin, David L. McArthur, Anita E. Bandrowski, Nidhi Bansal, Maryann E. Martone, Hadley C. Bergstrom, Anton Bespalov, Chris Graf, Improving transparency and scientific rigor in academic publishing, Brain and Behavior, 2018, e01141, 10.1002/brb3.1141
  • 46. Eric M. Prager, Karen E. Chambers, Joshua L. Plotkin, David L. McArthur, Anita E. Bandrowski, Nidhi Bansal, Maryann E. Martone, Hadley C. Bergstrom, Anton Bespalov, Chris Graf, Improving transparency and scientific rigor in academic publishing, Journal of Neuroscience Research, 2018, 10.1002/jnr.24340
  • 47. Eric M. Prager, Karen E. Chambers, Joshua L. Plotkin, David L. McArthur, Anita E. Bandrowski, Nidhi Bansal, Maryann E. Martone, Hadley C. Bergstrom, Anton Bespalov, Chris Graf, Improving transparency and scientific rigor in academic publishing, Cancer Reports, 2018, e1150, 10.1002/cnr2.1150

Reader Comments

your name: *   your email: *  

Copyright Info: 2014, Christopher D. Chambers, et al., licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution Licese (http://creativecommons.org/licenses/by/4.0)

Download full text in PDF

Export Citation

Copyright © AIMS Press All Rights Reserved