Pilot Testing a French and Kinyarwandan Game to Improve Vaccine Attitudes and Misinformation Resilience in Rwanda

John Cook¹, Laetitia Nyirazinyoye², Michael Mugisha², Angelique Uwamahoro², Rukundo Jean Claude³, Chelsey Lepage⁴, Angus Thomson⁴, Kathryn L. Hopkins⁵, Wendy Cook⁶

1. Melbourne Centre for Climate Change, University of Melbourne, Melbourne, Australia
2. School of Public Health, College of Medicine and Health Sciences, University of Rwanda, Kigali, Rwanda
3. UNICEF, Kenya Country Office, Nairobi, Kenya
4. Iminy, Lyon, France
5. Sabin Vaccine Institute, Washington DC, USA
6. Wendy Cook Design, Melbourne, Australia

OPEN ACCESS

PUBLISHED: 31 March 2025

CITATION: Cook, J., et al., 2025. Pilot Testing a French and Kinyarwandan Game to Improve Vaccine Attitudes and Misinformation Resilience in Rwanda. Medical Research Archives, [online] 13(3).
https://doi.org/10.18103/mra.v13i3.6421

COPYRIGHT:© 2025 European Society of Medicine. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

DOI https://doi.org/10.18103/mra.v13i3.6421

ISSN 2375-1924

ABSTRACT

Vaccine misinformation causes negative impacts such as decreased vaccine acceptance and reduced intent to get vaccinated. The association between susceptibility to vaccine misinformation and vaccine hesitancy underscores the need for interventions that increase public resilience against misinformation.

One promising intervention is psychological inoculation, where recipients receive a “weakened form” of misinformation to build immunity to real-world misinformation. Misinformation is delivered in weakened form by exposing its misleading techniques.

Cranky Uncle Vaccine is an interactive, digital game that applies inoculation theory, explaining facts about vaccines as well as the misleading techniques used to cast doubt on the facts. We document a pilot study testing the effectiveness of an East African version of Cranky Uncle Vaccine conducted in Rwanda.

Two translations of the game were made available with participants choosing either a Kinyarwandan-speaking (n = 535) or French-speaking (n = 499) version. Among Kinyarwandan-speaking players, the average age was 27.8 years with more males (52.7%) than females (46.9%). Among French-speaking players, the average age was 29.9 years with more males (55.3%) than females (44.7%). Among both languages, participants were highly educated (median education level “Some/all university”).

Before and after playing the game, participants filled out surveys measuring vaccine attitudes and discernment between facts and fallacies. French-speaking participants showed significant improvement in general vaccine attitude and vaccine importance but while the change in intent to get vaccinated was positive, it was not statistically significant.

Kinyarwandan-speaking players showed significant improvement in general vaccine attitude, vaccine importance, and intent to get vaccinated. The game was most effective among players who expressed vaccine hesitancy in the pre-game survey with 55% of hesitant French-speaking participants and 71% of Kinyarwandan-speaking participants switching to being likely to get vaccinated.

These results show that a digital game employing an inoculation approach is effective in building public resilience against vaccine misinformation and boosting vaccine acceptance, particularly among those expressing vaccine hesitancy.

Keywords: vaccination; Rwanda; misinformation; inoculation; games

Introduction

Misinformation about vaccines can negatively impact vaccine acceptance. National surveys spanning five countries found a clear link between susceptibility to misinformation and both vaccine hesitancy and a reduced likelihood to comply with health guidance measures.¹ An even wider survey spanning 40 countries found a strong association between perceived believability of COVID-19 misinformation and vaccination hesitancy.² People exposed to online misinformation around COVID-19 vaccines are less likely to accept a vaccine, with scientific-sounding misinformation associated with a stronger decline in vaccination intent.³ Agreement with COVID-19 misinformation is associated with lowered trust in scientific institutions and decreased confidence in governments.⁴

Health-related misinformation is particularly problematic in lower-to-middle-income countries (LMICs) with poorer regions showing a substantially higher prevalence of misinformation and greater susceptibility to believing COVID-19 misinformation. The impact of misinformation varies across countries and communities, having a disproportionate impact on marginalized populations.⁵ Misinformation literacy—defined as the skill to distinguish true from false information—varies across countries. A cross-national survey in Kenya, Uganda and Rwanda found that Kenyans and Rwandans were better at detecting false COVID-19 statements, while Ugandans were better at detecting true messages.⁶

That said, Rwanda is lauded as a best-case example among LMICs as the first low-income country in the world to introduce human papillomavirus (HPV) vaccination into its national program in 2011.⁷ The campaign to boost vaccination rates among girls in grade six achieved 93% coverage, with its success due to school-based vaccination, community engagement, and a nationwide sensitization campaign.⁸ Over a five-year period, the Rwandan government rolled out four vaccines (against pneumococcus, HPV, rotavirus, and measles & rubella), achieving over 90% coverage for each. This was achieved through extensive, cross-sectoral planning at least one year before introducing each vaccine, and mobilization of local leaders, development partners, civil society organizations, and community health workers.⁹

A survey of Rwandan adults found a high rate of vaccination acceptance, with respondents expressing COVID-19 vaccine acceptance and general vaccine acceptance at 91.6% and 98.2%, respectively.¹⁰ The high coverage rate for childhood immunisation in Rwanda (98% in 2015) has been attributed to several factors, such as community health workers sensitising communities, strong political will at the government level, decentralising service delivery approaches to the district and village level in order to tailor appropriate approaches for local populations, involving community leaders and leaders at all levels of government, and strong relationships with development partners.¹¹ Rwanda has been an exemplar for educating its population regarding the benefits of vaccination with vaccination coverage rates rivaling the higher levels seen within high-income countries.¹²

Proactive strategies have been a key feature of Rwandan success in bolstering vaccination acceptance. It is in this spirit that our study focuses on a proactive intervention designed to build public resilience against vaccine misinformation. There are a number of structural features of misinformation necessitating a pre-emptive approach. Misinformation spreads faster and deeper than accurate information¹³ and is difficult to dislodge once it takes hold.¹⁴,¹⁵ The more people are exposed to misinformation, the more they perceive it as accurate¹⁶ and are more likely to promote it.¹⁷ In response to these challenges, pre-emptive strategies have grown in prominence, seeking to prevent damage before misinformation spreads. One way to achieve this is by building critical thinking skills and resilience against misinformation through psychological inoculation.¹⁸

Inoculation theory

Inoculation theory is a branch of psychological research that applies the metaphor of vaccination to knowledge. Just as exposing people to a weakened version of a virus conveys immunity to the actual virus, exposing people to a dose of weakened misinformation can build a person’s cognitive immunity, protecting against future misinformation.¹⁹,²⁰ Specifically, misinformation is delivered in weakened form by explaining the rhetorical techniques and logical fallacies used to distort facts, an approach known as logic-based or technique-based inoculation.²¹

There are two elements to an inoculation message. First, the recipient is warned of the threat of being misled, putting them cognitively on guard against misinforming content. The threat warning motivates recipients to protect themselves from being deceived.²² Second, the inoculation contains counterarguments that explain how the misinformation misleads, such as explaining different logical fallacies. Inoculation has been found to be effective across a range of topics including health,²³ climate change,²⁴ and political misinformation.²⁵

As well as technique-based inoculation, another common approach is fact-based inoculation, which exposes how the misinformation is misleading through factual explanations.²¹ This approach has also been labeled as topic-based²⁶ or issue-based.²⁷ One limitation of fact-based inoculation is that addressing a specific fact restricts the generalizability of the intervention.²⁸ Another limitation is that misinformation has the potential to cancel out factual explanation.²⁹–³¹

In contrast, an advantage of technique-based inoculation is it has the potential to generalize—teaching people a misleading technique can help them spot misinformation across multiple topics.²⁴ Another advantage of the technique-based approach is it is effective when presented along with misinformation while fact-based inoculations can be cancelled out by misinformation.³¹

However, one danger of inoculating interventions is that warning people about the danger of being misled might cause them to also become distrustful of factual information.³² Some interventions targeting misinformation have had the inadvertent effect of reducing perceived reliability of factual information.¹⁸,³³ It’s not sufficient for a misinformation intervention to reduce susceptibility to misinformation—the intervention should increase discernment, defined as people’s ability to accurately distinguish facts from misinformation.³⁴

Another challenge of technique-based inoculation is the sheer number of misleading fallacies used in misinformation. A useful framework for explaining misleading techniques is the FLICC framework, a taxonomy of techniques and fallacies used in misinformation.³⁵ However, conveying this large taxonomy to the public presents an education and communication challenge.

Past inoculation studies have relied on text or video messages passively consumed by recipients. Videos targeting vaccine misinformation are effective in increasing vaccine intent and resistance to misinformation.³⁶ In addition, a recent, innovative approach has been to actively involve recipients in an inoculating intervention through digital games. This offers an engaging and interactive way to convey the comprehensive and detailed list of fallacies in the FLICC taxonomy.

Cranky Uncle Vaccine game

Digital games have been shown to be effective in improving players’ ability to discern facts from misinformation.²⁷,³⁷ The climate misinformation game Cranky Uncle incorporates research into technique-based inoculation²⁴ and critical thinking³⁸ as well as the use of persuasive humour.³⁹ The game has been adopted by hundreds of educators across the world.⁴⁰

The Cranky Uncle Vaccine game was subsequently co-designed in Rwanda, Uganda, and Kenya, narrowing the focus to vaccine misinformation.⁴¹ Cranky Uncle Vaccine contains the two elements of an inoculation intervention. The first element is a threat warning, where the player is warned about different tricks used to mislead (Figure 1a). This occurs in the onboarding stage at the start of the game.

The second element is counterarguments, in this case explanations of the different techniques employed to distort or cast doubt on facts (Figure 1b). These occur throughout the game as the player navigates through explanations of ten common fallacies used in vaccine misinformation.

Figure 1. Two elements of an inoculation intervention, found in Cranky Uncle Vaccine.

Pilot tests measured the game’s effectiveness in Uganda and Kenya (co-designed East African characters shown in Figure 2),⁴² as well as Ghana.⁴³ In these pilot studies, Cranky Uncle Vaccine was found to be effective in improving players’ discernment—the ability to distinguish facts from misinformation. Further, both studies found improvement in players’ attitudes towards vaccines with more than half of players who were unlikely to get vaccinated before the game switching to being likely to get vaccinated after completing the game (58% in Uganda/Kenya, 53% in Ghana). This strong result shows the powerful effect of the combination of vaccine facts and inoculating explanation of misinformation fallacies in an engaging, interactive intervention. However, the effectiveness of the game had yet to be tested in non-English languages.

Figure 2. Co-designed characters in the East African version of Cranky Uncle Vaccine. From left to right: younger man, younger woman, healthcare professional, Cranky Uncle, older woman, professional man with disability.

Materials and Methods

We used convenient sampling to invite 1034 Rwandans—535 Kinyarwandan-speaking and 499 French-speaking—to participate in the research. The sample size (N~1000) was calculated using One-Way Repeated Measures ANOVA power analysis assuming small effect size on some of the more difficult to move dependent variables (e.g., f = .08), power 0.95, and significance level of 0.05. Data was collected from 4 December 2023 to 29 January 2024.

Participants were recruited through partner organisations and institutions, including Youth Centres, University faculty, WhatsApp groups of staff and students (public and private), community members, and public members available on email or WhatsApp. Study participants were also encouraged to share the invitation link with family members and friends, so that additional participants were recruited through chain referral (snowballing) from those who previously completed the game. In addition, recruitment involved personal contacts, telephone, and participants approached in-person. A link to an explanatory statement and consent form was shared through these networks. Those who consented to participate were redirected to a page to download the game onto their phones or play on their computers via a browser.

Participants were at least 18 years old, residing in Rwanda, literate in French or Kinyarwandan, and had access to a smartphone or computer with internet access. Participants were excluded if they indicated a location outside Rwanda, if they couldn’t speak French or Kinyarwandan, or if they withdrew their consent. As players were anonymous, there was no way to track whether there was linguistic overlap between groups (such as a participant playing both translated versions of the game). Location was determined in two ways—by asking players to enter details of their location and using location data from the player’s phone. A sum of 5,000 (3.5 USD) Rwandan Francs was sent to each of the participants as compensation for their internet expenses and time. Reimbursement was provided once the participant completed the post-game survey.

Prior to the onset of data collection, a narrative explanation was included within the mobile app describing the purpose of the study, procedures to be followed, and the risks and benefits of participation. At the start of the game, players were invited to participate in research. If they indicated yes, they were shown a consent form. Consent was granted by clicking the Next button on the Informed Consent Form. To withdraw during or after gameplay, participants followed the Consent link on the About screen within the game—then unchecked “Allow collection of anonymous game data.” Only data that had been consented by the participant was included in research analysis.

Measures

Our research approach was primarily quantitative. Participants filled out a pre-game survey embedded within the game, then played the game until they completed explanations of ten misinformation tricks, then filled out a post-game survey. The measures used in this experiment and described below were the same as those used in previous Cranky Uncle Vaccine pilot studies.⁴²,⁴³ All the questions except for demographic and feedback items were repeated in both the pre-game survey and post-game survey.

DEMOGRAPHICS. In the pre-game research survey, participants were asked demographic questions concerning age, gender, region, and education level (categories for gender and education level are shown in Q2 and Q4 in Table 1).

VACCINE ATTITUDES. Participants answered three questions on a four-point scale, measuring whether

the participant was personally for or against vaccination, agreed with the statement “I feel that it is important that I get vaccinated,” and was likely to get recommended vaccinations for themselves or their children. The first two items were drawn from the validated Vaccination Trust Indicator.⁴⁴ We used 4-point Likert scales to measure vaccine knowledge, acceptance/intent, and misinformation resistance.

GAME FEEDBACK. Two open-ended questions were included in the post-game survey, asking what features stood out to players and what changes they would like to see made in the game. This data was content analysed for quantitative analysis, identifying the most common themes.

FACT RELIABILITY. Participants were shown two factual statements about vaccines and asked to rate how true or false they thought they were on a four-point scale. To obtain the two fact items, we first conducted a literature review of research studying vaccine misinformation (a detailed summary is provided in the description of the East Africa co-design workshops).⁴¹ From this literature review, scripts were created explaining the ten most common fallacies, which included relevant vaccine facts for each fallacy. From the scripts, two factual items were chosen to reflect the broad themes expressed in the fact-based content in the game. The factual statements were “Vaccines are among the most thoroughly tested medical products available” and “Vaccines help protect us against many serious diseases that can pass from person to person.”

FALLACY RELIABILITY. Participants were shown six misleading statements, each representing a different fallacy: natural is best (“Natural herbs boost the immune system better than vaccines”), false cause (“Someone got sick after a vaccine so the vaccine must be to blame”), evil intent (“The COVID-19 virus was intentionally created and spread for profit reasons”), conspiracy theory (“The COVID-19 vaccine contains a microchip to track people”), pick and choose (“Vaccines aren’t safe as a study was published mentioning safety concerns”), and personal attack (“Drug companies earn large amounts of money from vaccines so vaccines are untrustworthy”). The fallacy items were identified using the same process as the two fact items, selected as common arguments taken from the game script which was derived from a literature review of research studying vaccine misinformation.

GAME FEEDBACK. Participants who completed the game and the post-game survey answered two open-ended questions soliciting qualitative feedback about the game: “What stood out the most to you in the game?” and “What changes would you like made to the game?” Answers to both questions were content analyzed by two annotators, using the major themes identified in an earlier pilot study.⁴²

STATISTICAL ANALYSIS. To explore the game’s effectiveness, paired t-tests were conducted on the dependent variables of interest—the three vaccine attitude measures, two fact reliability measures, and six fallacy reliability measures—collected prior and after playing the game. Further, the area under a ROC curve (derived from the fact and fallacy measures) was calculated as a single measure of discernment and a paired t-test was conducted to explore any change in discernment from before to after the game. Analyses were conducted separately for French-speaking and Kinyarwandan-speaking participants.

Table 1. Research questions (English version)

Q1
Can you share your age in years? (pre-game only)

Q2
Please share your gender. (pre-game only)
Options:

• Female

• Male

• Other

• Prefer not to say

Q3
Please share your country in which you live and if you can, the county where you currently live. (pre-game only)

Q4
Please share your highest level of education completed. (pre-game only)

• No formal education

• Some/all of primary school

• Some of secondary school/high school

• Completed secondary school/high school

• Some/all college/diploma level

• Some/all university/degree level

• Some/all postgraduate degree level

Q5
Thinking about vaccination in general, would you say you are personally…

• Strongly for vaccination

• Somewhat for vaccination

• Somewhat against vaccination

• Strongly against vaccination

Q6
Please respond to the following statement:
“I feel that it is important that I get vaccinated”.

• Strongly agree

• Somewhat agree

• Somewhat disagree

• Strongly disagree

Q7
How likely are you to get recommended vaccinations for yourself or your children, if applicable?

• Very likely

• Somewhat likely

• Somewhat unlikely

• Very unlikely

How true do you think the following statements are?

Options for Q8 to Q15:

• True

• Mostly true

• Mostly false

• False

Q8 (Fact #1)
Vaccines are among the most thoroughly tested medical products available.

Q9 (Natural is best)
Natural herbs boost the immune system better than vaccines.

Q10 (False cause)
Someone got sick after a vaccine so the vaccine must be to blame.

Q11 (Evil intent)
The COVID-19 virus was intentionally created and spread for profit reasons.

Q12 (Fact #2)
Vaccines help protect us against many serious diseases that can pass from person to person.

Q13 (Conspiracy theory)
The COVID-19 vaccine contains a microchip to track people.

Q14 (Pick & choose)
Vaccines aren’t safe as a study was published mentioning safety concerns.

Q15 (Personal attack)
Drug companies earn large amounts of money from vaccines so vaccines are untrustworthy.

Q16
What stood out the most to you in the game? (post-game only)

Q17
What changes would you like made to the game? (post-game only)

Results

Participants filled out a pre-game survey embedded within the game (median time 3.1 minutes), then played the game until they completed explanations of ten misinformation tricks (median time 27.2 minutes), then filled out a post-game survey (median time 4.2 minutes).

For French-speaking players, demographic questions concerned age (Mean M = 29.9, Standard Deviation SD = 9.5), gender (44.7% female, 55.3% male), and education level (median education level “Some/all university”). Kinyarwandan-speaking participants also answered demographic questions: age (Mean M = 27.8, Standard Deviation SD = 9.2), gender (46.9% female, 52.7% male), and education level (median education level “Some/all university”).

T-tests showed no significant difference between the French-speaking and Kinyarwandan-speaking participants on gender (t(1031) = .467, p = .64, d = 0.029) but showed a significant difference between the two groups with regard to age (t(1021) = 3.481, p < .001, d = 0.217) and education (t(1030) = −2.129, p = .03, d = 0.132).

The wording of the questions are given in Table 1 and demographic data is shown in Table 2.

Table 2. Demographics of French-speaking and Kinyarwanda-speaking participants

a) Age, b) Gender, c) Education levels

Main effects on vaccine attitudes and misinformation discernment

Figure 3 shows pre-game and post-game vaccine attitudes.

For French-speaking players, paired t-tests showed significant improvement from pre-game to post-game in general vaccine attitude (t(498) = -2.6, p = .008, d = .12) and vaccine importance (t(498) = -3.4, p < .001, d = .15). While intent to get vaccinated increased, the change wasn’t significant (t(498) = -1.6, p = .11, d = .07).

For Kinyarwandan-speaking players, paired t-tests showed significant improvement from pre-game to post-game in:

• General vaccine attitude (t(534) = -5.9, p < .001, d = .26)

• Vaccine importance (t(534) = -2.7, p = .004, d = .13)

• Intent to get vaccinated (t(534) = -2.6, p = .009, d = .11)

Results are shown in Figure 3.

Figure 3. Vaccine attitudes before (red) and after (blue) playing the game, measuring whether participants are generally for or against vaccines, perceived importance of vaccines, and likelihood of getting recommended vaccinations.

Among the French-speaking participants expressing vaccine hesitancy in the pre-game survey, selecting unlikely or very unlikely to get vaccinated (n = 29 or 5.8%), just over half of these participants (n = 16 or 55%) switched to likely or very likely to get vaccinated in the post-game survey.

Similarly among the Kinyarwandan-speaking participants, among those expressing hesitancy before the game (n = 21 or 3.9%), nearly three quarters (n = 15 or 71%) switched to being likely to get vaccinated. Signal detection theory offers a technique for exploring changes in players’ discernment between factual and misinforming statements. ROC (Receiver Operating Characteristic) analysis examines discrimination between accurate and false information, with the area under the curve (AUC) representing a single measure of discernment.

For French-speaking participants, a paired samples t-test showed a decrease from pre-game AUC values (M = .6997, SD = .3174) to post-game AUC values (M = .676, SD = .339) with the change being statistically significant (t(498) = -2.09, p = .036, d = -.09, CI [-0.046393800, -0.001535391]).

For Kinyarwandan-speaking participants, a paired samples t-test showed an increase from pre-game AUC values (M = .7424, SD = .2976) to post-game AUC values (M = .7627, SD = .2931) with the increase in AUC being marginal (t(534) = 1.666, p = .09, d = .07, CI [-0.003644129, 0.044298335]).

Figure 4 shows the ROC curves summarizing participants’ discrimination between facts and misinformation.

Figure 5 shows the pre-game and post-game perceived reliability of vaccine facts and fallacies for both countries (see Appendix for wording of research questions).

French-speaking participants showed:

• No change in perceived reliability for fact #1 (t(498) = 0, p = 1, d = 0)

• No change for fact #2 (t(498) = -.13, p = .897, d = .01)

• Significant decrease in reliability ratings for the false cause fallacy (t(498) = 3.22, p = .001, d = .14)

• Marginal increase in reliability for personal attack (t(498) = -1.71, p = .087, d = .08)

However, there was no significant decrease in reliability for the natural is best fallacy.

(t (498) = 1.44, p = .15, d = .06), evil intent (t (498) = 1.23, p = .218, d = .06), pick and choose (t (498) = .499, p < .617, d = .002), or conspiracy theory (t (498) = -.513, p = .608, d = -.02).

Kinyarwandan-speaking participants showed no change in perceived reliability for fact #1 (t (534) = -1.373, p = .17, d = -.06) or fact #2 (t (534) = 1.038, p = .299, d = .04). They showed a significant decrease in their reliability ratings for the natural is best fallacy (t (534) = 2.23, p = .026, d = .1), false cause (t (534) = 10.129, p < .001, d = .44), evil intent (t (534) = 4.319, p < .001, d = .19), and personal attack (t (534) = 3.482, p < .001, d = .15). However, there was no significant decrease in reliability for pick and choose (t (534) = 1.221, p < .223, d = -.05) or conspiracy theory (t (534) = -1.328, p = .185, d = .06).

Figure 5. Perceived reliability of vaccine facts: fact 1, fact 2, and vaccine fallacies: natural is best, false cause, evil intent, pick & choose, conspiracy theories, and personal attack. Red shades represent pre-game measures, blue shades represent post-game measures.

Moderating effect of pre-existing vaccine attitudes

One limitation with our pilot study design was the non-representativeness of the participant sample, with high education levels and high vaccine acceptance in the pre-game survey. Consequently, we conducted further analysis to explore whether the game’s effectiveness varied depending on players’ pre-existing vaccine attitudes.

A pre-game vaccine attitude variable was created by averaging general vaccine attitude, vaccine importance, and vaccine intent (questions 5, 6, and 7 in the pre-game survey). Linear regression indicated a significant relationship between pre-existing vaccine attitudes and change in general vaccine attitude (t (1032) = -21.5, p < .001). Figure 6a shows the change in general vaccine attitude for participants split into high vaccine acceptance (pre-game average vaccine attitude > 2) and low vaccine acceptance (pre-game average vaccine attitude ≤ 2).

The change in vaccine importance was significantly associated with pre-existing vaccine attitudes (t (1032) = -19.6, p < .001) with the relationship visualised in Figure 6b. Similarly, the change in vaccine intent was significantly associated with pre-existing vaccine attitudes (t (1032) = -15.6, p < .001), shown in Figure 6c.

Fact reliability, calculated by averaging the two fact items, was significantly associated with pre-existing vaccine attitudes (t (1032) = -5.9, p < .001). Fallacy reliability was calculated by averaging the six fallacy items. This was the one dependent variable that didn’t show a significant relationship with pre-existing vaccine attitudes (t (1032) = 1.6, p = .1).

Figure 6. Pre-game and post-game values for participants with low or high pre-game vaccine acceptance for

Content analysis of player feedback

Player feedback invited participants to describe highlights in the game and request changes. Qualitative feedback was content analysed by two annotators, with interrater reliability calculated using Cohen’s kappa.

Interrater reliability for the highlight answers showed substantial agreement (kappa = .68), while answers for changes answers showed very good agreement (kappa = .85). In cases of disagreement, the two annotators deliberated and decided on consensus codings.

Table 3 lists the highlight themes in order of prevalence, with example quotes from each category.

Table 3. Themes identified in response to the question “What stood out the most to you in the game?”, percentage of participants in each theme, and examples.

In the post-game survey, participants were asked “What changes would you like made to the game?” (samples of answers shown in Table 4). For the Kinyarwanda version, this was translated to “Ni izihe mpinduka wifuza ko uyu mukino wakuzanira?” which could be interpreted as “What changes would you like this game to bring you?” While some participants answered this question by listing what they would like to see changed in the game (as per the original intent of the question), others answered this question by listing what impact the game had or might have had on themselves or others.

Consequently, we created a new category for our content analysis capturing “Impact of the game” answers, which we then subdivided into what personal impact the game had on the player’s perceptions/behaviour or what changes they would like to see the game have on society more broadly.

Table 4. Themes identified in response to the question “What changes would you like made to the game?”, percentage of participants in each theme, and examples.

Discussion

This Rwanda-based study pilot-tested the effectiveness of the Cranky Uncle Vaccine game in improving vaccine attitudes and discernment between vaccine facts and misinformation in two non-English languages—French and Kinyarwandan. We found that in this context, Cranky Uncle Vaccine was an effective tool for improving vaccine attitudes and building public resilience against vaccine misinformation.

In particular, we found that the game’s effectiveness was greatest with those who began the game with low levels of vaccine acceptance. Among these participants, the game showed a strong improvement in vaccine attitudes and agreement with vaccine facts, although for French-speaking participants, there was only a marginal reduction in agreement with vaccine misinformation. This result has implications for which segments of the community should be targeted for this style of intervention.

One result from this study that contrasted with previous Cranky Uncle Vaccine pilot studies conducted in Kenya, Uganda, and Ghana was that agreement with vaccine facts showed no significant change for both French-speaking and Kinyarwandan-speaking participants.

While an ideal outcome is increased agreement with facts (as seen in English-speaking pilot studies), the researchers avoided the problem of reducing trust in accurate information, which has sometimes occurred in misinformation interventions.

Another unique insight from this study arose inadvertently from the misinterpretation of one of the feedback survey questions, with some participants mistakenly thinking they were being asked what changes the game had on themselves or what changes they would like the game to have on others. This happy accident yielded qualitative insights into the personal impact of the game, with players reporting that the game had successfully shifted their perceptions and understandings in comments such as “Changing perceptions about things I don’t know much about” and “The game made me realise the misunderstandings I had about vaccines”. Players reported holding erroneous beliefs prior to playing the game then shifting to more accurate beliefs: “Before playing this game, I thought that vaccination was no good, but now I know that without vaccination, there could be a lot of deaths.” While inoculation is conceived as a pre-emptive strategy, it can also be effective for those who have already encountered misinformation,⁴⁵ an approach described as therapeutic inoculation.⁴⁶

As well as changes in perceptions and attitudes regarding vaccines, participants also reported changes in behaviour as a result of playing the game, in comments such as “I will change the way I take care of my health”. Others went further, describing changes to family behaviour: “I was inspired to get vaccinated and to encourage my relatives to do the same” and “Having my children vaccinated based on the national calendar”. The most common recurring changes in behaviour involved how they talked to others about vaccines after playing the game: “Having adequate information about vaccines so that I can explain their benefits to others”, “Acquiring knowledge to educate others”, and “Helping me convince others that vaccination is important”.

This proactive frame of mind indicated that the game was successful in increasing player’s counterarguments against vaccine misinformation, one of the expected outcomes of an inoculation intervention. Passively holding counterarguments in mind was indicated in comments such as “The game opened my mind to the tactics used by people who deny the benefits of vaccines.” However, some participants went further and displayed an increased intent to talk to others about vaccines, reporting that after playing the game, they knew “…how to respond when I meet someone who denies vaccines”, and that they now had “…adequate information about vaccines so that I can explain their benefits to others.” This increased sense of empowerment after an inoculation intervention is known as post-inoculation talk, a by-product of inoculation where recipients are more likely to talk about controversial topics.⁴⁷,⁴⁸

The other desired outcome from an inoculation intervention is increased perceived threat of being misled. Players reported an increased awareness of the misleading tricks used in vaccine misinformation: “The game opened my mind to the tactics used by people who deny the benefits of vaccines” and “The tactics used by cranky [sic] uncles are the same tactics used in real life.” Another goal of the game was to improve misinformation discernment, which was not only indicated in the quantitative data but also in the qualitative feedback with one player reporting that “I’ve learnt to distinguish between true and false information about the vaccine.”

Among the requested changes, the most common category was “no changes”. As roughly equivalent percentages requested more content (7.4%) or less content (7.2%), this indicates the amount of content in the game strikes a good balance. Nevertheless, there were some criticisms, with some negative comments reflecting the pilot study design. One comment was “provide means to play the game. Such as internet and communication bundles.” For the pilot study, the game was made available on smartphones but there are currently efforts to make the game available on other platforms such as text-

based chatbots and audio-based Interactive Voice Recognition (IVR). In particular, a version of the game is in development for the Internet of Good Things (IoGT) which offers free data access on low-end phones, improving the game’s accessibility.

Some players also requested other translations with comments such as “It would be better to use all languages (Swahili, English)” or “An English version of the game.” The pilot study was designed so that players were only able to access French or Kinyarwandan versions of the game but publicly, the game is available in English, Swahili, French, and Kinyarwandan.

A recurring comment was a request for similar content or interventions on other topics: “Give us more games”, “Teach us more, even on other topics”, “Telling us about other diseases.” The critical thinking nature of the game means that the logic-based approach can readily be adapted into interventions to counter misinformation in other topics. This is evidenced by the fact that Cranky Uncle Vaccine was originally adapted from the Cranky Uncle game, an intervention focused on climate misinformation.⁴⁰

Some updates to the game have already been implemented since the Rwandan pilot study. For example, one comment was “For some questions, you can’t go back. If you make a mistake and move to the next, you can’t go back to the previous question.” After the Rwandan pilot was completed, a back button was added to the game.

Some participants requested “Developing a better understanding of how to respond to vaccine detractors.” This could take the form of an extra module in the game that trains players on how to have conversations with people who have concerns or hold misconceptions about vaccines. Such a module could form part of training for health professionals, educators, and communicators.

Another limitation in current misinformation games is an exclusive focus on building resilience against misinformation and have yet to include interventions that would empower players to have conversations about controversial topics like vaccination and climate change.

Communication strategies have been developed for clinicians when addressing vaccine hesitancy and refusal, such as rapport building and eliciting and addressing concerns.⁴⁹ However, deploying such training is resource-intensive. A productive area of future research is exploring whether such training frameworks could be deployed in a digital game, offering a powerful way to deploy the training at scale.

One limitation of our study was that we only established the efficacy of Cranky Uncle Vaccine in one east African country. However, this study adds to a growing body of evidence showing efficacy of the game now in Kenya, Uganda, and Ghana. We also show increased generalizability by testing the game in two additional languages other than English. As our analysis showed little difference in effectiveness between French and Kinyarwandan, this also indicates the game’s effectiveness is conveyed across different languages.

These outcomes will continue to be explored in ongoing pilot studies being conducted in Pakistan (with Urdu and Roman Urdu translations) and Australia (with an Arabic translation).

Another limitation of this study was the use of convenience sampling which resulted in a non-representative participant sample that was highly educated and showed high vaccine acceptance prior to playing the game. We attempted to circumvent this limitation by analysing the impact of the game on different segments of the participants, finding that the game was most effective with those who started the game with low vaccine acceptance. Nevertheless, conducting a study with a representative sample would offer a more robust assessment of the game across different population segments.

As the research survey is embedded in the game as part of a broader cross-country program, this imposes a degree of inflexibility with regards to amending the research survey relative to past pilot studies. Consequently, past pilot studies have acknowledged the lack of measurement of important inoculation concepts such as perceived threat and counter-arguments.²⁷,⁵⁰ However, a fortunate accident from translation of the research question yielded qualitative insights into these concepts, with participants reporting both increased perceived threat and increased counter-arguments. Further, participants reported an increased feeling of empowerment after playing the game, intending to talk more about vaccines with their social networks.

Conclusion

This study broadens the evidence base for the effectiveness of Cranky Uncle Vaccine in improving vaccine attitudes and misinformation resilience, by expanding to another country and two non-English languages. We add to the growing evidence for digital games as a powerful and scalable tool to counter misinformation about vaccines (as well as other topics). Given that over 80% of misinformation intervention research has focused on Global North countries,⁵¹ we also fill this research gap by testing a misinformation intervention in the Global South.

While digital games such as Cranky Uncle and Cranky Uncle Vaccine have shown to be effective in countering misinformation, there’s an urgent need for research on the game’s effectiveness in other formats and in other contexts. Smartphones are not always widely adopted in non-WEIRD (Western, Educated, Industrial, Rich, Democracies) countries. A text-only chatbot game would be ideally suited to low-end phones, making the game available to difficult-to-reach audiences. An audio-only game could reach illiterate segments of the adult population which varies in East African countries from 17% (Kenya) to 21% (Rwanda). African phone users have to pay for data use but IoGT offers free data access on low-end phones, so an IoGT game offers a low-tech version of the full-featured smartphone game.

Our results also show that inoculation interventions such as digital games have the potential to convey multiple benefits. Cranky Uncle Vaccine is most effective with people who are vaccine hesitant, improving perceptions and attitudes about vaccines, including the intent to get vaccinated. However, it should not be overlooked that the game is also effective with people who are already accepting of vaccines by empowering them to talk about vaccines with their friends and family. In this context, the game is not about “preaching to the choir” but instead, it “teaches the choir to sing”, an important benefit that has already been observed in other topics such as climate change.⁵²

Author Contributions

JC, AT, LN, MM contributed to research design and ethics application. WC contributed to intervention content design and image creation. MM, LN and AU contributed to data collection and processing. JC and WC content analysed qualitative survey data. JC conducted literature review and data analysis. JC led the first draft of the manuscript; and KH, LN, MM, AU contributed to reviewing and further revisions of the manuscript. All authors provided final approval of the paper prior to submission.

Declaration of Interest

AT and CL work for Irimi Company which offers consulting services in behavioral public health.

Data Availability

The datasets generated during and/or analyzed during the current study are available in the OSF repository:
https://osf.io/rvuzf/?view_only=a45a8ff792124ef0b0b7c00bee27cd2

Ethical Approval

Ethical clearance for this study was obtained from the Rwanda National Ethics Committee (RNEC) before commencement of the study.

References