Questions for Research Article Critique

October 7, 2011 § Leave a comment

  • What questions are the authors trying to address? Is this an important research question?
  • What assumptions are made in the research? What is the validity of these assumptions?
  • How is the research question approached? What is the design of the research? If empirical, is the methodology appropriate?
  • What conclusions are made in the paper? Are these conclusions appropriate given the empirical evidence? Are the conclusions important?
  • Could this work be extended or refined? If so, how?
  • What is the relationship between this and other articles in the same session?, or between this and other articles in other sessions?
  • Briefly summarize the question the author is trying to answer. Is this question interesting and important? Why or why not?
  • In evaluating the importance of the question, you should consider whether the author’s review of the literature suggests a logical need for this research.
  • Some issues you might want to think about are: Is this research the first empirical test of an important theoretical prediction?
  • Does this research extend existing theory? Does this research test competing theoretical predictions?
  • Does this research remedy important flaws in past empirical research? A good empirical piece does not need to do all of these, but it should be more than just a minor extension of an overly complex model.
  • Briefly describe the model the author uses to answer the research question.
  • What are the key concepts in the model and what are the relationships between those concepts? Try to be as specific as possible in describing the model (e.g., do not say “this paper tests a transactions costs theory of the employment relationship,” rather say that “this paper examines the idea that the firm specificity of employees’ skills affects the mechanisms a firm uses to govern the employment relationship”.)
  • Your summary of the model should be brief; it should indicate that you understand the author’s model without describing it in detail. If you are able, you may wish to comment on how well the model fits with existing literature in the area.
  • How well does the model represent what we already know about the author’s research question?
  •  Discuss the appropriateness of the author’s methodology. Does the methodology appear to be able to answer the author’s research question?
  • Some issues to consider are: Is the sample appropriate (e.g., if the author wants to study factors that affect the death of firms, does the sample include both firms that died and comparable firms that did not die?).
  • Are the measures reasonable representations of the constructs in the author’s model (e.g., in the study just described, how does the author measure organizational death – is this measure consistent with the author’s theoretical description of the construct?).
  • Include some evaluation of the appropriateness of the statistical tests.
  • Are the tests reasonable, given the author’s research question?
  • Does the author explore alternative explanations for the results and test them where possible?
  • How well does the author explain anomalous or unexpected results? Are the author’s explanations for these results tested, where possible? Be careful not to get bogged down in detail.
  • Do not criticize the statistical tests unless you can offer some reasonable explanation for why the test is inappropriate or for why an alternative test would be more appropriate. For example, do not say “the author should have used a regression analysis instead of comparing means across organizations” unless you can explain why a simple comparison of means is not appropriate and how a regression analysis would have improved the author’s results.
  • Evaluate the author’s conclusions.
  • Do the conclusions address the author’s research question?
  • Are the conclusions consistent with the results?
  • Are there any untested alternative explanations for the author’s results?
  • If these alternative explanations cannot be tested in this research, does the author suggest ways in which these explanations might be tested in future research?
  • Does the author discuss the limitations of the research and describe ways of remedying those limitations in future research?
  • Provide suggestions for improvement (this can be done in conjunction with each of the items listed above or as a separate section of the critique).
  • For each major criticism of the work, suggest ways in which the work could be improved.
  • For example, if you feel that the research question is not important, suggest a related, but more important, research question.
  • If you believe that the sample is flawed, suggest a more appropriate, but still reasonable sample.
  • You should refrain from making suggestions that are correct theoretically but unfeasible practically.
  • Your suggestions for improvement should focus on practical, reasonable steps that the author could take to improve the research.
  • Outline the ideas and indicate which of these ideas you think would be the most valuable to pursue.

Where Am I?

You are currently browsing the Article Review category at Outside the Gate: Gospel & Organization.