Goal: To define naturalistic observations and understand the different ways that they may be done: overt vs covert and participant vs non-participant.  In addition, students should be able to evaluate the use of a naturalistic observation.

Short answer question: Students are asked to watch Deb Roy's "Birth of a Word."  Then they are asked the following question: Write a paragraph in which you describe the study to a friend.  Be sure to identify this as a lab or naturalistic observation; overt or covert; participant or non-participant.  Explain at least one strength and one limitation of the study.

You may either give feedback or a mark for this task.  I use criterion D on the ERQ rubric:

The response consistently demonstrates well-developed critical thinking.

Evaluation or discussion of relevant areas is consistently well developed.

Responses that are in the top markband earn a "6" or a "7" - in other words, I take the score they would have received for criterion D and add 1 mark to give them a score on this assignment.  Students that only state limitations but do not explain them may not earn higher than a 4.  Remember, to get into the high markband, the evaluation should be explanatory - so not just, the study has high ecological validity - but why that is true and what it means with regard to the findings.

Potential strengths include: the naturalistic environment in which the study takes place; the study is a longitudinal study that allows the research to observe change over time; the use of technology allowed for an objective collection of the data.

Potential limitations include:  only a single participant, so it is difficult to generalize from this study; there is potential researcher bias in the fact that the researcher is studying his own child; there is an ethical question about a researcher studying his own child in such a manner.

Checking for understanding:  This is a task where students answer questions about observations in order to practice terminology and demonstrate conceptual understanding. Answers are provided for students to self-assess their own understanding. No grade is entered into your grade book.

Goal: To evaluate three types of interviews: unstructured, semi-structured, and focus groups. In addition, questionnaires are discussed.

Assessing your understanding: Students are given an abstract and asked to answer questions 1a, 1b, and 1c from Paper 3, as well as a modified question 3.

To assess this, I mark questions 1a, b, and c as if it were a Paper 3 - awarding up to three marks apiece.  For the final question, I also award 3 marks.  1 mark for a correct bias and up to 2 marks for a correct explanation of why this may be a bias.

In awarding marks, I use the following markbands: 7: 11 - 12; 6: 10; 5: 8 - 9; 4: 7 ; 3: 5 - 6; 2: 3 - 4; 1: 0 - 2.

Sample answers and tips for marking are below.

1a. Identify the method used and outline two characteristics of the method.

The method used is a focus group.  One characteristic of a focus group is that it is a group interview, usually consisting of between 5 and 8 participants. A second characteristic is that an interviewer poses questions or topics for discussion, but the conversation is natural between the members of the group.

One mark is awarded for correctly identifying the method and then one marks for each of the correct characteristics.

1b. Describe the sampling method used in the study.

The sampling method is a bit complex.  It is a sample of opportunity in that it was taken from a single clinic in Paris.  The researchers then asked for recommendations from the clinic, which shows that the sample was not randomly selected.  Using this list, they then asked these participants to volunteer to be in the study.  Of those that volunteered, the researchers had exclusion criteria which meant that those with other health issues could not take part in the study.

If the method is identified and simply defined with no link to the text, a maximum of one mark should be awarded.  Two marks should be awarded if there is a basic answer with a link to the study.  Three marks should be awarded if the description shows some understanding of the complexity of the sampling technique.

1c. Suggest an alternative or additional research method giving one reason for your choice.

There are several potential answers here, but the most logical would be either a one-to-one interview (either unstructured or semi-structured) with each participant, to see if they give the same responses when they are not in the group.  This is good to control for conformity effects.  The other potential response is to use a questionnaire.  This can also be anonymized so that demand characteristics are lowered. Questionnaires also provide a standard set of questions that allow for easier analysis than a focus group.

The alternative method has to make sense in the replication of the study with the same aim.  Changing the aim of the study results in a score of zero marks. One mark is awarded for identifying an appropriate method; one mark for describing how it would be used; one mark for the explanation as to why it would be used.

2. What would be one potential bias of this sample?

The researchers noted in the original that all of the participants were religious.  This is most likely why they chose to volunteer for this study, as the researcher would have had to identify the aim of the study when asking for their participation.  The findings may have been different if the participants had not self-identified as being spiritual/religious. Other potential responses could be that there was a cultural bias or gender bias.

One mark is awarded for identifying a bias (e.g. sampling bias) but not linking it to the study.  Up to two more marks are awarded for the "unpacking" of the bias.

Goal: To evaluate the use of the case study method and to differentiate between the different methods of triangulation.

Thinking about case studies:  This task asks students to watch two videos on hyperthymesia - people with perfect recall of autobiographical memory.  Students are then asked to explain why these studies would be considered case studies.  The question is self-assessed with an answer provided.  No information is entered into your grade book.

Checking for understanding:  This a set of 10 multiple choice questions about the three methods required for paper 3: interviews, case studies, and observations. The questions are automatically scored and entered into your grade book.  Explanations of the correct responses are provided for the students.

Goal: Revise ethical considerations for all research.  Explain ethical considerations in socially sensitive research with regard to the publication and application of research findings.

Applying terminology: Students are given five scenarios of socially sensitive research and asked what ethical considerations there would be in the publication and application of the findings.  They are asked to use the terminology of Sieber & Stanley (1988) in their responses.  There are sample answers for them to self-assess their understanding. Nothing is entered into your grade book.

Checking for understanding: Students are given an abstract and asked to write a response for both ethics questions from Paper 3.  Each response is worth six marks.

In awarding marks, I use the following markbands: 7: 11 - 12; 6: 10; 5: 8 - 9; 4: 7 ; 3: 5 - 6; 2: 3 - 4; 1: 0 - 2.

Sample answers and tips for marking are below.

Describe the ethical considerations that were applied in the study and explain if further ethical considerations could be applied.

The researchers would need to get consent from the women in this study. They should be made aware of the aim of the study and how their data will be used.  They should also be reminded of their rights as a participant. For example, the identities of the women will be confidential - that is, when the study is published, no information that would identify the women would be included.  This is important because they may face stigma as a result of the findings. They should be reminded that they have the right to withdraw. They should be allowed not only to decline participation in the interview, but they may have their data removed even after completing the interview. There is no deception used in this study.  The aim is honestly communicated as the researcher wants to do a valid evaluation of the mental health of the participants. Researchers have to consider potential undue stress to the women, carrying out a telephone interview in which they are asked personal questions about their partner and their own mental health.  Perhaps by having the interview over the phone, there will be less stress for the participants.  Finally, the researcher should debrief the participants.  Based on the findings of this study, in addition to sharing the findings, the researcher should suggest ways in which the women may receive support.

When awarding marks, one mark is awarded for each description/explanation. If the ethical consideration is only identified without reference to the actual study, then no marks are awarded.  For example, participants have the right to withdraw.

Describe the ethical considerations in reporting the results and explain additional ethical considerations that could be taken into account when applying the findings of the study.

The identities of the women must be kept confidential.  This is important because they could face stigma in their community if their identities were revealed.  In addition, it would also reveal the mental health status of their partner, possibly leading to discrimination. It is important that the publication does not imply that this is true of all women married to combat veterans.  The study should not result in a general statement about the lives of military couples, potentially stereotyping such couples.  As the findings suggest that women married to veterans with PTSD may need psychological support themselves, another ethical consideration is that the researchers advocate for such mental health care for women in this situation. In presenting their findings, it is important that they are clearly stated and that any potential bias or error in their findings is clearly communicated to the reader. It is also important that the researchers use reflexivity, documenting their own potential biases - eg. my father was a combat veteran or I work with veterans with PTSD. This will help the reader to decide on any potential bias in the reporting of the data.  Finally, it is transparency is important.  Is the study funded by the military? If so, the goal may be to help support spouses of combat veterans.  Is the study funding by a drug company that markets anxiolytic drugs (anti-anxiety)?  If so, there may be some concerns about the findings. 

When awarding marks, award one mark for each relevant point with regard to ethical considerations up to a maximum of six marks.  In this example, one mark for each of the following: the need for confidentiality for the women; confidentiality for the partner, avoid stereotyping; advocacy for spouses of combat veterans with PTSD, revealing any personal biases; explaining any potential errors in the study; transparency of funding.

Goal: To differentiate between generalizability and transferability.  Discuss the extent to which the findings of a study may be generalized or transferred to another population.

Presentation: The presentation explains the three different types of generalizability applied in qualitative research: representational, inferential, and theoretical (analytical). The presentation ends with three scenarios, asking students to apply the terminology to answer questions.  There are answers provided so that students may self-assess.  Nothing is added to your grade book.

Short Answer Question: Students are presented with a fictitious study of cheating. The study is based on information in this article, but it is not meant to be an actual study.  It is designed to make it possible for students to answer questions about the three types of generalization.

You may choose to only give feedback or to give marks for this task.  If you give marks, you may want to give 3 marks for each question, for a total of 9 marks.  Marks should be awarded based on how well developed the response is. This would be equal to the marks awarded for question 3 on Paper 3.  You may want to award marks as follows: 7: 8 - 9 marks; 6: 6 - 7 marks - and then all other marks are equal to an IB grade - that is, 5 marks is a 5.

Suggested answers to the questions are below.

1.  What questions or concerns do you have about the sample in order to determine whether it is representative of the population from which it was drawn? (representational generalization)

The sample was a volunteer sample on a university campus.  The participants knew that the study was about why people cheat. This may mean that "cheaters" are over-represented in the sample.  In addition, we would want to know the gender breakdown, as well as the diversity of the sample compared to the population of the university.  We would also want to know the majors (subjects) that the students are studying.  Are they all psychology majors?  Mostly maths majors?  It would also be important to know the age range of the sample and whether they are living on campus or whether they live with family off campus; whether they are working or not working.

2.  What variables would we have to consider with regard to the sample or the environment of the study in order to determine if we could transfer the findings to another population? (inferential generalization)

Inferential generalization would mean to what extent can we apply the findings from this university population to another university population?  Variables that would be important include, the size of the university, the competitiveness of the university, the location of the university (the country with regard to cultural dimensions; rural vs urban); socioeconomic status of the student body; diversity of the student body; whether the university has strict penalties for cheating; whether the university integrates academic integrity into the curriculum. 

3. To what other contexts do you think that we could generalize the findings? (theoretical generalization)

Cheating is not just about university. The findings that social or community norms may encourage cheating could be applied to paying taxes or traveling without paying for public transportation. It might also be generalized to cheating in relationships - that is, when internal motivation for the relationship decreases (e.g. love) and external motivation increases (financial benefit, keeping family happy), then cheating may increase.

Goal: To differentiate between validity and credibility.  Discuss the ways in which the credibility of a study may be ensured.

Presentation: The presentation explains three key strategies for ensuring credibility: triangulation, member checking, and the credibility of the researcher. The presentation ends with a research scenario, asking students guiding questions to apply the terminology and concepts.  There are answers provided so that students may self-assess.  Nothing is added to your grade book.

Short Answer Question: Students are presented with a fictitious study of community service and prosocial behaviour. The study is based on information in this study, but it is not meant to be an actual study.  It is designed in line with IB exam expectations.

You may choose to only give feedback or to give marks for this task.  If you give marks, you may want to give 3 marks for each of the three main points about credibility (member checking, the credibility of the researcher and triangulation), for a total of 9 marks.  Marks should be awarded based on how well developed the response is. This would be equal to the marks awarded for question 3 on Paper 3.  You may want to award marks as follows: 7: 8 - 9 marks; 6: 6 - 7 marks - and then all other marks are equal to an IB grade - that is, 5 marks is a 5.

Suggested approaches to the questions are below.

Students should write this as a paragraph and not just list ideas.  The IB requires a "well-developed argument."  Making a list will mean that only mid-range marks will be awarded.

Triangulation

The study has used data triangulation in that there are 15 schools represented.  This means that there are several different perspectives represented in the sampling of the data.  That being said, the sample is restricted to a certain part of the United States.  A broader sampling of data may give a more holistic picture of attitudes toward community service.  In addition, only the students were interviewed.  Interviewing parents and/or teachers may give richer data for analysis and confirm or challenge the perceptions of the students. Only choosing to look at students at the end of their high school career may also have an effect on the findings.  Since they are perhaps nostalgic as they end their time in high school, the results may be less credible than if students still in the process of high school were interviewed.

Researcher triangulation should be used - both in the carrying out of the interviews and the interpretation of the data.  This limits potential researcher bias.  It also confirms that different researchers reached the same conclusions when looking at the same data.

Finally, by only using one-on-one interviews, it is possible that demand characteristics or interviewer effects played a role in the findings.  Method triangulation would increase the credibility of the study.  Using questionnaires would allow the researchers to see if they had the same responses when the participants were anonymous.  In addition, running focus groups would potentially result in different responses as participants listen to the thoughts of their peers.  If the same results are obtained using different methods, then we know it was not the choice of face-to-face interviews that led to the results.

Member checking

A key aspect of credibility is that the results reflect the intention and meaning of the participants' data. It is important that the researchers take the interpretation of their data back to the participants (members) and asks them if they feel that their feelings/opinions have been fairly represented.  This is also a chance for the participants to correct misperceptions or clarify their responses.

The credibility of the researcher

When evaluating the credibility of this study, one would want to know what the experience of the researcher is.  Are they a teacher? A psychologist?  And do they have experience working with teenagers?  One would want to know that if several researchers would used, was there training for how to carry out the semi-structured interviews?

Goal: To explain the different types of bias in research.  Discuss how a researcher may avoid bias in a study.

Presentation: The presentation examines three key sources of bias: selection (sampling) bias, participant bias, and researcher bias.  For each type of bias, strategies for avoiding the bias are discussed. Nothing is added to your grade book.

Checking for understanding: A set of multiple-choice questions to check for understanding of key vocabulary relevant to question 3 on Paper 3.  The questions are automatically scored and entered into your grade book.  Explanations of the correct responses are provided for the students.

There are three final assessments for this unit.  One for each of the three qualitative methods, each with a different question 3. You may choose to give one or all three of the final assessments over time. Each assessment is one hour.

If you want students to do this remotely, you can assign this task using the "exam mode" which will limit their time to one hour.

Mock Paper 3 A

This stimulus piece is based on this study. Details have been altered in order to align with IB assessment practice.

Markscheme

1a. Identify the method used and outline two characteristics of the method

1b. Describe the sampling method used in the study.

1c. Suggest an alternative or additional research method giving one reason for your choice

2. Describe the ethical considerations that were applied in the study and explain if further ethical considerations could be applied.

3. Discuss the possibility of generalizing the findings of the study.

There are three final assessments for this unit.  One for each of the three qualitative methods, each with a different question 3. You may choose to give one or all three of the final assessments over time. Each assessment is one hour.

If you want students to do this remotely, you can assign this task using the "exam mode" which will limit their time to one hour.

Mock Paper 3 A

This stimulus piece is based on this study. Details have been altered in order to align with IB assessment practice.

Markscheme

1a. Identify the method used and outline two characteristics of the method

1b. Describe the sampling method used in the study.

1c. Suggest an alternative or additional research method giving one reason for your choice

2. Describe the ethical considerations that were applied in the study and explain if further ethical considerations could be applied.

3. Discuss how a researcher could ensure that the results of the study are credible.

There are three final assessments for this unit.  One for each of the three qualitative methods, each with a different question 3. You may choose to give one or all three of the final assessments over time. Each assessment is one hour.

If you want students to do this remotely, you can assign this task using the "exam mode" which will limit their time to one hour.

Mock Paper 3 A

The third paper 3 is focused on a case study.

This stimulus piece is based on this study. Details have been altered in order to align with IB assessment practice.

Markscheme

1a. Identify the method used and outline two characteristics of the method

1b. Describe the sampling method used in the study.

1c. Suggest an alternative or additional research method giving one reason for your choice

2. Describe the ethical considerations in reporting the results and explain ethical considerations that could be taken into account when applying the findings of the study.

3. Discuss how the researchers in the study could avoid bias.