Meeting the challenge of the Psychonomic Society’s 2012 Guidelines on Statistical Issues: some success and some room for improvement

Morris, P. E. and Fritz, C. O. (2017) Meeting the challenge of the Psychonomic Society’s 2012 Guidelines on Statistical Issues: some success and some room for improvement. Psychonomic Bulletin and Review. 1069-9384.

The files below are currently restricted to repository staff only.

They may be awaiting processing or under a publisher's embargo.

Items under embargo will be available for download from the date noted.

Item Type:	Article
Abstract:	The Psychonomic Society (PS) adopted New Statistical Guidelines for Journals of the Psychonomic Society in November 2012. To evaluate changes in statistical reporting within and outside PS journals, we examined all empirical papers published in PS journals and in the Experimental Psychology Society journal, The Quarterly Journal of Experimental Psychology (QJEP), in 2013 and 2015, to describe these populations before and after effects of the Guidelines. Comparisons of the 2013 and 2015 PS papers reveal differences associated with the Guidelines, and QJEP provides a baseline of papers to reflect changes in reporting that are not directly influenced by the Guidelines. A priori power analyses increased from 5% to 11% in PS papers, but not in QJEP papers (2%). The reporting of effect sizes in PS papers increased from 61% to 70%, similar to the increase for QJEP from 58% to 71%. Only 18% of papers reported confidence intervals (CIs) for means; only two PS papers in 2015 reported CIs for effect sizes. Although variability statistics are important to understanding data, and to further analysis, they were only reported as numbers in just over half of the PS journal papers. Almost all PS and QJEP papers relied exclusively on null hypothesis significance testing to guide interpretation of the data. Changes associated with the Guidelines are in the desired direction with respect to reporting effect sizes and power analyses, but are not yet reflected in researchers’ practices in describing their data, addressing data assumptions, and thinking beyond the p value when interpreting their data.
Uncontrolled Keywords:	Statistical inference, statistics, confidence intervals, effect size, quantitative analysis
Creators:	Morris, Peter E and Fritz, Catherine O
Faculties, Divisions and Institutes:	Faculties > Faculty of Health & Society > Psychology
Date:	1 March 2017
Date Type:	Acceptance
Journal or Publication Title:	Psychonomic Bulletin and Review
Language:	English
DOI:	https://doi.org/10.3758/s13423-017-1267-y
ISSN:	1069-9384
Status:	Published / Disseminated
Refereed:	Yes
Related URLs:	https://pure.northampton.ac.uk/en/public...
URI:	http://nectar.northampton.ac.uk/id/eprint/9303

Actions (login required)

Edit Item

Altmetric