I was recently asked by a top MIS journal to review a paper for them.  It was an experiment using a theoretical base that I knew well.  The experiment featured a convenience sample of students carrying out an MIS related task that the authors claimed needed no in-depth knowledge of MIS at the technical level.  Further, the issue was one of what would an MIS related actor do if he or she discovered the problem.  In other words it was NOT a user oriented task but a production oriented task.

I had to ask myself the question, what can we learn about MIS from an experiment carried out by a sample of students on a task that was MIS production oriented (i.e. coding) yet claimed to need no MIS specific technical knowledge to carry out????

Sounds like a generic psychology experiment to me about human information processing with an MIS cover story to go along with it.  Indeed, having sit on a recent PHD in psychology dissertation committee, the cover story would not have looked out of place in their experiment.  But the psychology student claimed he was studying basic human information processing.

It was a very well done experiment but for the life of me I could not figure out what I learned about MIS from reading it.  Yet there were all sorts of claims about the fact that this experiment had implications for MIS (along with any other substantive subject matter that could fit into that task features).