Last year I took the message that the unit of analysis for research impact had to shift when it came to accessing impact outside of the academic world.  While flawed citation rates can give us a good first order read of the value of an individual paper on the research literature as long as that is not the sole measure examined especially in the case of areas where there are smaller numbers of researchers working.

But Wednesday at FAR I am going to take the case for change in audit knowledge transfer one step further and argue that when it comes to research impact outside the “scientific” literature that “single paper” approach is fundamentally flawed.  All areas of evidence based policy making that I have read suggest one thing – single papers rarely prove anything but may give a path forward or backward (in the case of contradictory results).

For knowledge transfer the only consistently effective method that I have been able to locate in the literature is that of a properly done research synthesis (NOT a literature review) that responds to specific questions that are on the minds of real world participants.  While the results are still in process of being analyzed and written up – we have got to the point where we are certain we can present evidence of proof of concept – it works!  To make this work consistently we will have to supplement existing institutions, devise some new education approaches.  Indeed, as Christian Leuz and I agreed on at the EAA Conference in Milan, there needs to be a division of labor that reflects different competencies and it is not clear that the individual researcher is the best person to be carrying out research syntheses.