In a day and age of multiple statistical methods being readily available through software packages, and a decline in the number of methods courses being required of PhD students in behavioral accounting, it is little surprise when a reviewer encounters a statistical analysis in a paper where they have substantive expertise.  [ Aside: When I was a boy I took two classes in probability and statistical theory, three applied methods courses and that was just the requirements.  I took another three methods courses beyond what was the minimum for credit and audited two others. ]

So what does the reviewer do?  Do they assume the author(s) has picked an obscure statistical methods for data analysis correctly? OR Do they educate themselves on the method and consider whether it is appropriate for the type of data?

Of course, this assumes that the reviewer is well enough trained in statistical methods to recognize what assumptions are needed to be fulfilled in order to use any given method.  While the previous generation of researchers certainly knew how to check for gross violations of underlying statistical assumptions in ANOVA and other linear models, some understood the assumptions underlying SEM and its variants, when it gets to hazard rate models, logit and probit models, and other such models only a handful new.  I wonder today how many newly trained behavioral PhD’s could say the same????

The beauty is that if a reviewer has any basic knowledge of model assumptions that need to be fulfilled in general, our good friend “google” can help you find what the assumptions are for models that you many not have heard of.  For example, this week I had a paper that employed Poisson log linear regression models in my doctoral class.  While I had heard of Poisson distribution and I had heard of log linear regressions, this is the first time I had heard this combination.  Yep, after 30 years in the business, a new one.

So what did I do, I went to my trusty “google” and looked for “Assumptions underlying Poisson loglinear regression models.”  The first thing that popped up was that the dependent variable is suppose to be count data that was theoretically unbounded at the upper end of the distribution. (“If the data are anything but non-negative integers that are (in principle, at least) unbounded, Poisson regression is the wrong model to use. “) Whoops, the data had only three values 0, 1, 2.  Well that suggests further research is necessary by the reviewer and at least a strongly worded review note should result.  [Second aside: I will find out whether my doctoral students read this blog when I see whether they bring up the point in class!]

Unfortunately this is a real example of a paper that is in the later rounds of review at a major journal (NOT BRIA of course).  It will be interesting to see if the journal review process catches it and whether the data stand up to proper statistical analysis (likely an ordered probit but I could be wrong)!