Confusion over reporting statistical significance
July 20th, 2012, by Guy Harris
The reporting of statisitcal significance in papers continues to confound authors. Should you mention p values in your paper or not? Some journals now actively discourage it, but the message is not always clear. We recently reviewed a letter from the editor of a journal to one of our clients. The paper had been submitted to the journal once, returned with reviewer’s comments, submitted again, and then returned again with additional comments from the Editor, as follows:
‘We're interested in the possibility of publishing your paper. However, before the editors can make a final decision, we would like to see a revision that takes account of the following concerns.
1) First, we apologize for any confusion regarding our policy on p-values. DESPITE the fact that one of the reviewers requested more p-values, our journal strongly discourages p-values and (in particular) discussions of statistical significance. Tests for trend are acceptable, but p-values should be removed from table 1 and the text. Any mention of statistical significance should be removed.
Here is an interesting case - a reviewer had requested additional p-values, even though though the journal’s policy was against it. Somehow, the discrepancy was not identified before the reviewer’s comments were sent to the author, who then did additional but unnecessary work. This is an interesting case which highlights the ongoing uncertainty about statistical reporting. How is this reporting viewed in your field?