Evaluating research

This activity focuses on students ability to read and understand even complex articles. It’s goal isn’t necessarily for the student to understand everything in the text, but to be able to pinpoint important parts and to discuss valid points the article makes.

The activity follows classes on research. Students are given each question immediately after appropriate lesson has been covered (e.g. question 10 after ethics have been gone through). In the end, students hand out a summary of a scientific paper.

Goal

Teaching students how to read research which might be a bit over their heads with its complexity. Showing the plethora of methodologies used. Showing the structure that scientific articles usually have (IMRD). Learning to think critically about conducted research - was it done well? Does it actually answer the question it set to answer?

Structure

Let students select a paper from a list of approved studies. Read through one paper together in class and answer all questions - even those which they don’t know what they are - sometimes, some students do :)

Let them “browse” the paper and answer questions 1-4. Then, after each lesson on a particular topic (say methods, ethics, statistics), give them those questions to answer as well. After all questions have been answered (usually 3-4 weeks) ,give them extra week to finalize the report.

Schedule in points

  1. Go over one psych paper together (procrastionation by Dan Ariely)

  2. Read the article and Answer questions 1-4 - even though they don’t know perfectly how or they don’t understand the entire paper

  3. Get their answers and give brief pointers. Do not mark at this point

  4. Get back to each question as you do each appropriate section (sampling, ethics) and ask them to hand you answer to those questions as you cover the lessons.

  5. After all questions have been answered, give them one week to “finalize” the report.

  6. Get their new answers (and mark them).

Questions

  1. What was the purpose of the study

  2. What were the hypotheses?

  3. What was the conclusion?

  4. What did the authors say that can be a problem with their conclusion?

  5. What was the method used for the research? Describe it and explain whhy it was chosen.

  6. What was the research design used? (within subject, between subject, matched pairs)

  7. What other methods could authors use to study their theory? Describe at least one and compare it to the one they used.

  8. If the research had independent and dependent variables, describe what they were and how they were operationalised.

  9. Can the results be generalized to bigger sample? To which one and which one not?

  10. Does the research seem valid? (Or does it strike you as weird, too laboratory, not well designed, unusable etc.) Explain and provide examples.

  11. What was the sampling method used in the paper?

  12. Was the sampling method well chosen? Explain and provide alternatives.

  13. Was the research conducted ethically according to the principles laid out by the Belmont report? State why - what was done and what wasn’t.

  14. Does the research follow the APA code link?

  15. Was there some potential danger regardless of the ethical requirements being supposedly met? How it could have been prevented or diminished?

Recomended journals

Authors Title Journal Full journal Publication year
Ambady N,Rosenthal R Half a minute: Predicting teacher evaluations from thin slices of nonverbal behavior and physical attractiveness J. Pers. Soc. Psychol. Journal of personality and social psychology 1993
Roberts SC,Kralevich A,Ferdenzi C,Saxton TK,Jones BC,Debruine LM,Little AC,Havlicek J Body odor quality predicts behavioral attractiveness in humans Arch. Sex. Behav. Archives of sexual behavior 2011
Peterson LR,Peterson MJ Short-term retention of individual verbal items J. Exp. Psychol. Journal of experimental psychology 1959
Festinger L,Carlsmith JM Cognitive consequences of forced compliance J. Abnorm. Psychol. Journal of abnormal psychology 1959
Neisser U,Harsch N Phantom flashbulbs: False recollections of hearing the news about Challenger 1992
Ekman P,Friesen WV Constants across cultures in the face and emotion J. Pers. Soc. Psychol. Journal of personality and social psychology 1971
Rosenthal R,Jacobson L Pygmalion in the classroom Urban Rev. The Urban review 1968
Bargh JA,Chen M,Burrows L Automaticity of social behavior: direct effects of trait construct and stereotype-activation on action J. Pers. Soc. Psychol. Journal of personality and social psychology 1996
Cialdini RB,Borden RJ,Thorne A,Walker MR,Freeman S,Sloan LR Basking in reflected glory: Three (football) field studies J. Pers. Soc. Psychol. Journal of personality and social psychology 1976
Heston LL Psychiatric disorders in foster home reared children of schizophrenic mothers Br. J. Psychiatry The British journal of psychiatry: the journal of mental science 1966
Bandura A,Ross D,Ross SA Imitation of film-mediated aggressive models J. Abnorm. Soc. Psychol. Journal of abnormal and social psychology 1963
Freedman JL,Fraser SC Compliance without pressure: the foot-in-the-door technique J. Pers. Soc. Psychol. Journal of personality and social psychology 1966
Milgram S BEHAVIORAL STUDY OF OBEDIENCE J. Abnorm. Psychol. Journal of abnormal psychology 1963
Darley JM,Batson CD \ From Jerusalem to Jericho\”: A study of situational and dispositional variables in helping behavior” J. Pers. Soc. Psychol. Journal of personality and social psychology 1973
Rosenhan DL On being sane in insane places Science Science 1973
Loftus EF,Palmer JC Reconstruction of automobile destruction: An example of the interaction between language and memory Journal of Verbal Learning and Verbal Behavior 1974
Rosenzweig MR,Bennett EL Psychobiology of plasticity: Effects of training and experience on brain and behavior Behav. Brain Res. Behavioural brain research 1996
Steele CM,Aronson J Stereotype threat and the intellectual test performance of African Americans J. Pers. Soc. Psychol. Journal of personality and social psychology 1995
Maguire EA,Gadian DG,Johnsrude IS,Good CD,Ashburner J,Frackowiak RS,Frith CD Navigation-related structural change in the hippocampi of taxi drivers Proc. Natl. Acad. Sci. U. S. A. Proceedings of the National Academy of Sciences of the United States of America 2000
Raine A,Buchsbaum M,LaCasse L Brain abnormalities in murderers indicated by positron emission tomography Biol. Psychiatry Biological psychiatry 1997
comments powered by Disqus