Interesting observation about International Reception at AAA – the relatively low numbers of European academics compared to previous East Coast conferences. Indeed, maybe it was me, but the international reception seemed a lot emptier than in previous years and the room was certainly not any bigger. It as the same room we were in six years ago! I wonder why? ( tongue very much in cheek).

Imagine – Bob Libby, Lisa Koonce, Kathryn Kadous, and two “other guys” (i.e. I cannot spell their names but Ryan and Scott are first names) on a panel discussing process variables in experimental research.  Furthermore, the AAA was videoing the session for ye old website!!!

It was a wonderful session overall. Lisa basically made the case for replications, whether they be done within or across papers to rule out boundary conditions, alternative explanations, etc.  In other words run more experiments if the finding is important to ensure you have the “right” explanation/theory.  the two guys ( Scott and Ryan) made the case for understanding what statistics can and cannot do and gave one of the best explanations I have heard for differences in purposes between OLS, two stage least squares and causal or structural models.

However, we had a little over generalization on the panel based on a paper I had not thought about in years Nisbett and Wilson 1977 that was situated in the middle of the verbal process tracing debate with the later Nobel Winning economist Herbert Simon.  The message of Nisbett and Wilson was that verbal protocols cannot give us insight into cognitive processes that operate at the unconscious level.  Very good point and totally correct – and the panelist was totally correct up to this point.  The overgeneralization was made by the remaining panelist – that we cannot verbalize anything about processes.  Indeed, at one point it seemed like she was saying asking people how they do or see things is just junk data.

Sorry, forty years of research since then have fleshed out Nisbett and Wilson a fair bit and to say that there are no times that people can report on the processes in their judgments is a gross oversimplication.  Certainly when it comes to automatic processes of expert accountants, it might be unreliable to ask them about the factors at affect their routine judgments; but then the simple solution is to study those in the midst of learning how to do the task. There they have to cognitively think about how to do it if you want unconscious process data that Is a place to start as for those folks it is still conscious (I did this in Salterio 1994 with audit managers learning to find precedents). Furthermore, Nisbett and Wilson is located in the unconscious processing or automatic processing realm – System 1/heuristics/peripheral route depending on your dual processing theory of choice – not the System 2/systematic/central route decisions.  So again, it all depends on your definition of “process”.  If you are focusing on automatic processes – she is right.  Anything else – not so right!!!

so great panel – lots of good thought’s but evidence based researchers need to consider carefully not to overgeneralize.