RESEARCH

 

Reducing unconscious bias in postgraduate specialist assessment in South Africa

 

 

B Mrara^I; O Oladimeji^II; A J Rantloane^III; V C Burch^IV

^IMB BCh, DA (SA), FCA (SA), Cert Crit Care (SA); Department of Anaesthesia, Faculty of Medicine and Health Sciences, Walter Sisulu University, Mthatha, South Africa
^IIMBBS, MSc, MPA, PhD; Department of Public Health, Sefako Makgatho Health Sciences University, Ga-Rankuwa, South Africa
^IIIMB ChB, MMed; Department of Anaesthesiology, University of Pretoria, Gauteng, South Africa
^IVMB BCh, MMed, PhD, FCP (SA), FRCP (London); Directorate of Education and Assessment, Colleges of Medicine of South Africa, Johannesburg, Gauteng, South Africa

Correspondence

 

 

---

ABSTRACT

Unconscious bias involves making decisions, interpreting situations and forming assumptions about people based on past experiences, stereotypes, ingrained thoughts and deep-seated ideas. In diverse multicultural societies, unconscious bias has been extensively studied for its impact on social inequity including access to quality healthcare. In addition, unconscious bias can influence assessment outcomes in medical practitioner training, potentially hindering advancement towards establishing a diverse health workforce. This paper reflects on perspectives on unconscious bias in postgraduate assessments utilising a South African model as an illustration. The authors propose some practical strategies to address unconscious bias in specialist training and assessment.

Keywords: Unconscious bias; educational assessment; stereotypes.

---

 

 

Unconscious bias, also known as implicit bias, refers to making automatic decisions, interpretations and assumptions about people and situations based on past experiences, stereotypes, deep-seated thoughts and ingrained ideas.^[1] Characteristics such as race, gender, age, language, disability and nationality can shape unconscious bias. Globally, unconscious bias significantly affects the quality of care given to patients of different racial backgrounds.^[2] Furthermore, unconscious bias may impact postgraduate specialist training and assessment, as these processes are managed by faculty and examiners who may have inherent biases.^[3,4] Yeo et al.^[3] observed disparities in pass rates by race, ethnicity, sex and family status in the American Board of Surgeons examinations, noting higher success rates for single women, white and non-Hispanic candidates. The disparities were significant in the oral compared with the written examinations, suggesting possible implicit bias among the examiners. Differences in language proficiency and other factors are also possible explanations for the disparities. If unchecked, unconscious bias in assessment will hinder the achievement of a diverse health workforce that is essential to reducing inequality, as espoused by the Sustainable Development Goals.^[5]

Diagnosing and quantifying unconscious bias can be challenging.^[6] One of the tests used for this purpose is the Implicit Association Test (IAT), which measures associations between concepts, to uncover hidden biases.^[7] The IAT is a computer-based test that is taken in seven steps where concepts such as race and gender are paired with attributes such as kindness and logic. Different combinations are presented to the test-taker. The response time in pairing the concepts to the attributes is used as a measure of unconscious bias. Faster responses are regarded as reflecting authentic internal biases, while slower responses reflect disagreeable associations. The IAT has faced criticism for construct validity, complexity and situational intra-individual score variability.^[8] For example, the responses may be slowed by factors such as the cognitive complexity of the seven-step task and the capability of the test-taker to switch between categories. Furthermore, results could also reflect cultural knowledge rather than internal biases. It is not clear how the test should be positioned in discussions about unconscious bias - whether its primary purpose is to raise awareness, support unconscious bias training or function as a diagnostic tool.^[9] Nevertheless, it remains the most widely used diagnostic tool for unconscious bias in the medical literature. Using the IAT as a validated tool may invoke feelings of guilt and defensiveness in those who are found to have unconscious biases, potentially diminishing constructive interventions.^[10] Sukhera et al.^[11] recommend that the IAT should not be framed as the definitive metric of unconscious bias owing to its limitations. Instead, it should be considered a small component of broader efforts to recognise and address hidden biases in individuals and institutions.

Post-apartheid South Africa (SA) promised change and equality across all spheres of life and sectors of society, including the health professions.^[12,13] However, the legacy of apartheid lingers on in the form of racial group disparities in socioeconomic status, access to healthcare and life expectancy. ^[14,15] Furthermore, decades of segregation and living apart are likely to have affected people's attitudes toward other races and given rise to unconscious biases. This predisposition to unconscious bias is fuelled in part by a lack of knowledge and experience of other racial groups as well as socio-political hierarchies, as described by Emberton.^[16] Laws against racism and discrimination such as the "Promotion of Equality and Prevention of Unfair Discrimination Act 4 of 2000"^[17] were promulgated to compel professionals to actively distance themselves from explicit racial bias and discrimination. Sadly, these laws do not address unconscious biases. This is illustrated in a 2016 qualitative study by Thackwell et al.,^[18] which explored the experiences of black African medical specialist trainees at predominantly or historically white institutions. Participants reported instances of individual racial discrimination, institutional racism as well as racial and gender bias.

Medical training for doctors and specialists was previously segregated, with restrictions on admission of black, Indian and coloured individuals to specific medical schools. However, over the years, there has been significant growth in the number of doctors and specialists of colour, and this has positively contributed to diversifying the healthcare workforce in SA. Notwithstanding this progress, many discrepancies persist. A 2015 audit of SA medical student demographics noted that the demographics of SA medical doctors were significantly skewed in favour of white doctors, falling considerably short of the country's demographics.^[12] An audit of records from the Health Professions Council of South Africa in 2019 found that up to 69% of surgical specialists were classified as white.^[19] Four years later, the 2023 register of medical specialists showed the percentage of black surgeons to be approaching 50% of the total. However, in other disciplines such as anaesthesiology, the percentage of white specialists remains above 60%.^[20] These inequities are likely to affect the composition of university academic staff responsible for specialist training and assessment, as observed by Thackwell et al.^[18] in their qualitative study.

The Colleges of Medicine of South Africa (CMSA) is the national specialist certification examining body since 1954.^[21] The CMSA currently has 29 constituent colleges that conduct postgraduate examinations for medical and dental specialists, and it has the sole authority to conduct capstone specialist examinations in SA. The examinations are conducted by panels comprising academic staff from universities that train medical specialists. While CMSA has made significant strides in ensuring diverse examiner panels, there is still a long way ahead. In some constituent member colleges examiner panels are still predominantly white, owing to the composition of the academic departments. This means that unconscious bias may be an important consideration in the decision-making processes regarding specialist examinations.

The transition to diverse panels will potentially encourage dialogue and research on implicit bias and reduce its impact. CMSA has also made efforts to reduce bias by standardising assessment opportunities using objective examination formats, thereby potentially bridging gaps between historically advantaged and disadvantaged universities.

 

Implications of unconscious bias in postgraduate specialist assessment

Unconscious bias of examiners may affect both written and performance examinations. Written examination bias may be reduced by the de-identification of scripts, the use of single best-answer questions with automated machine marking or short-answer questions with concise constructed responses, which minimise the influence of language nuances. Performance examinations remain at risk of unconscious bias because examiner judgments inherently involve a subjective element.^[22,23] Even when using structured assessment processes such as the Objective Structured Clinical Examinations (OSCEs), which include marking memoranda, scoring rubrics and checklists to guide examiner decisions, the overall impression of examiners may still be considered and incorporated as a global rating score.^[22] Several other factors can compromise the objectivity of OSCE assessments including examiner experience and comparing candidate performance, as noted by Zimmermann et al.^[23]

Some examples of unconscious bias that may affect decisions in performance examinations include ageism, physical appearance and gender. Other examples are explained below.

Affinity bias refers to being drawn to people of a similar background. ^[24]

In the examination context, it could manifest as examiners being friendly and 'coaching' candidates of the same demographics or language, potentially giving them an unfair advantage.

Conformity bias is the tendency to change one's beliefs to fit in with others. This bias is exemplified by junior examiners being influenced by more experienced examiners on the judgment of candidate performance, potentially leading to unfair outcomes.^[25]

Contrast bias refers to judging two issues while comparing them, instead of judging and assessing them independently. An example is an unconscious comparison of two candidates, based on language accents instead of establishing a criterion-referenced standard to make pass or fail judgments.^[26]

In addition, the amplification cascade has been described as an important negative consequence of unconscious bias. It is the amplification of small differences in performance, translating them into large differences in grades, sometimes with awards, often favouring the well-represented trainees and perpetuating a narrative of excellence in the favoured group.^[27]

 

Measures to address unconscious bias

Addressing unconscious bias in postgraduate medical training and assessment requires a multifaceted approach involving examiners, candidates and the respective training and examining institutions such as SA universities and the CMSA. For all institutions, it is crucial to prioritise diversity and inclusion when appointing academic staff and CMSA examination panels by actively seeking and supporting faculty from diverse backgrounds including both historically advantaged and disadvantaged universities. Additionally, diversity and inclusion training^[28] should be integrated into curricula and examiner training.

Additional practical measures to address unconscious bias are presented in Table 1. These measures are a combination of views supported in the literature and empirical concepts developed by the authors. There is a concerning dearth of interventional studies addressing unconscious bias in postgraduate medical education and assessment.

Ongoing and expanded implementation of these interventions by local medical schools and the CMSA will continue to address unconscious bias in postgraduate medical training and assessment.

 

Conclusion and recommendations

Addressing and reducing the impact of unconscious bias in postgraduate specialist training and assessment is a crucial part of reducing the lingering effects of historical inequality, which subtly influences the education and assessment environment, potentially leading to disparities in training outcomes. Strategic measures such as providing regular bias-awareness training for assessors, implementing standardised assessment processes and diversifying the pool of trainers and examiners, are essential to continue to improve the fairness of postgraduate training and assessment. These strategies can help foster a more equitable and inclusive environment for health professions education in SA.

A commitment to transparency in training and assessment processes and open discussions about unconscious bias can help build trust and further enhance fairness. As SA continues to grapple with its past and strives for a more equitable future, health professions education and assessment processes must reflect this shift. We should strive to create an environment that not only eliminates bias but actively promotes diversity and inclusion. This is not only a moral imperative but also a necessary step to ensure the highest quality of care for all patients in a diverse society.

Declaration. None.

Acknowledgements.

Author contributions. BM conceived the idea. BM and OO initiated the discussion to publish the concept. Subsequently, BM and OO created the initial draft. VB and AR contributed technical insights and provided guidance. All authors participated in the critical review process of this piece and approved the final version.

Funding. BM's research-protected time was partially supported by a Discovery Grant (No 038208). OO's research-protected time was partially supported by the South African Medical Research Council (SAMRC) through its Division of Research Capacity Development under the Research Capacity Development Initiative from funding received from the South African National Treasury and the National Research Foundation Grant (132385), Incentive Funding for Rated Researchers (IPRR). The content and findings reported or illustrated are the sole deduction, view and responsibility of the researcher and do not reflect the official position and sentiments of the funders.

Data availability statement. Not applicable.

Conflicts of interest. None.

 

References

1. Greenwald AG, Banaji MR. Implicit social cognition: Attitudes, self-esteem and stereotypes. Psychol Rev 1995;102(1):4-27. https://doi.org/10.1037/0033-295x.102.1.4        [ Links ]

2. FitzGerald C, Hurst S. Implicit bias in healthcare professionals: A systematic review. BMC Med Ethics 2017;18(1):19. https://doi.org/10.1186/s12910-017-0179-8        [ Links ]

3. Yeo HL, Dolan PT, Mao J, Sosa JA. Association of demographic and program factors with American Board of Surgery qualifying and certifying examinations pass rates. J Am Med Assoc Surg 2020;155(1):22-30. https://doi.org/10.1001/jamasurg.2019.4081        [ Links ]

4. Dayal A, O'Connor DM, Qadri U, Arora VM. Comparison of male vs female resident milestone evaluations by faculty during emergency medicine residency training. J Am Med Assoc Intern Med 2017;177(5):651-657. https://doi.org/10.1001/jamainternmed.2016.9616        [ Links ]

5. United Nations. Transforming our world: The 2030 agenda for sustainable development. 2015. https://wedocs.unep.org/20.500.11822/9814 (accessed May 2023).         [ Links ]

6. Arif SA, Schlotfeldt J. Gaps in measuring and mitigating implicit bias in healthcare. Front Pharmacol 2021;12:633565. https://doi.org/10.3389/fpl        [ Links ]

7. Greenwald AG, McGhee DE, Schwartz JL. Measuring individual differences in implicit cognition: The implicit association test. J Pers Soc Psychol 1998;74(6):1464-1480. https://doi.org/10.1037//0022-3514.74.6.1464        [ Links ]

8. Schimmack U. The implicit association test: A method in search of a construct. Perspect Psychol Sci 2021;16(2):396-414. https://doi.org/10.1177/174569        [ Links ]

9. Sukhera J, Wodzinski M, Rehman M, Gonzalez CM. The implicit association test in health professions education: A meta-narrative review. Perspect Med Educ 2019;8(5):267-275. https://doi.org/10.1007/s40037-019-00533-8        [ Links ]

10. Maina IW, Belton TD, Ginzberg S, Singh A, Johnson TJ. A decade of studying implicit racial/ethnic bias in healthcare providers using the implicit association test. Soc Sci Med 2018;199:219-229. https://doi.org/10.1016/j.socscimed.2017.05.009        [ Links ]

11. Sukhera J, Wodzinski M, Rehman M, Gonzalez CM. The implicit association test in health professions education: A meta-narrative review. Perspect Med Educ 2019;8(5):267-275. https://doi.org/10.1007/s40037-019-00533-8        [ Links ]

12. van der Merwe LJ, van Zyl GJ, St Clair Gibson A, et al. South African medical schools: Current state of selection criteria and medical students' demographic profile. S Afr Med J 2015;106(1):76-81. https://doi.org/10.7196/SAMJ.2016.v106i1.9913        [ Links ]

13. Clarke M, Bassett C. The struggle for transformation in South Africa: Unrealised dreams, persistent hopes. J Contemp Afr Stud 2016;34(2):183-189. https://doi.org/10.1080/02589001.2016.1202501        [ Links ]

14. Yach D. Health status and its determinants in South Africa. Afr Health 1994;(Spec No):5-8.         [ Links ]

15. Kon ZR, Lackan N. Ethnic disparities in access to care in post-apartheid South Africa. Am J Pub Health 2008;98(12):2272-2277. https://doi.org/10.2105/AJPH.2007.127829        [ Links ]

16. Emberton M. Unconscious bias is a human condition. Perm J 2021;25(20):199. https://doi.org/10.7812/TPP/20.199        [ Links ]

17. https://www.justice.gov.za/legislation/acts/2000-004.pdf (accessed 29 May 23).         [ Links ]

18. Thackwell, N, Swartz L, Dlamini S, et al. Race trouble: Experiences of black medical specialist trainees in South Africa. BMC Int Health Hum Rights 2016;16:1-6. https://doi.org/10.1186/s12914-016-0108-9        [ Links ]

19. Tiwari R, Chikte U, Chu KM. Estimating the specialist surgical workforce density in South Africa. Ann Glob Health 2021;87(1):83. https://doi.org/10.5334/aogh.348        [ Links ]

20. Rantloane A (Medical and Dental Board, Health Professions Council of South Africa (HPCSA)) personal communication. Register of medical specialists, with permission from HPCSA. 23 January 2024.         [ Links ]

21. The Colleges of Medicine of South Africa. https://www.cmsa.co.za (accessed 1 April 2024).         [ Links ]

22. Hyde S, Fessey C, Boursicot K, MacKenzie R, McGrath D. OSCE rater cognition - an international multi-centre qualitative study. BMC Med Educ 2022;22(1):6. https://doi.org/10.1186/s12909-021-03077-w        [ Links ]

23. Zimmermann P, Kadmon M. Standardised examinees: Development of a new tool to evaluate factors influencing OSCE scores and to train examiners. GMS J Med Educ 2020;37(4):Doc40. https://doi.org/10.3205/zma001333        [ Links ]

24. Bapuji H, Chrispal S, Vissa B, et al. Local, yet global: Implications of caste for MNEs and international business. J Int Bus Policy 2023;6(2):201-234. https://doi.org/10.1057/s42214-022-00146-9        [ Links ]

25. Padalia D. Conformity bias: A fact or an experimental artifact?. Psychol Stud 2014:59;223-230. https://doi.org/10.1007/s12646-014-0272-8        [ Links ]

26. Yeates P, O'Neill P, Mann K, Eva KW. 'You're certainly relatively competent': Assessor bias due to recent experiences. Med Educ 2013;47:910-922. https://doi.org/10.1111/medu.12254        [ Links ]

27. McClintock AH, Fainstad T, Jauregui J, Yarris LM. Countering bias in assessment. J Grad Med Educ 2021;13(5):725-726. https://doi.org/10.4300/JGME-D-21-00722.1        [ Links ]

28. Devine PG, Ash TL. Diversity training goals, limitations and promise: A review of the multidisciplinary literature. Annu Rev Psychol 2022;73:403-429. https://doi.org/10.1146/annurev-psych-060221-122215        [ Links ]

29. Woreta FA, Gordon LK, Knight OJ, Randolph JD, Zebardast N, Pérez-González CE. Enhancing diversity in the ophthalmology workforce. Ophthal 2022;129(10):e127-e136. https://doi.org/10.1016/j.ophtha.2022.06.033        [ Links ]

30. Gandhi M, Johnson M. Creating more effective mentors: Mentoring the mentor. AIDS Behav 2016;20 (Suppl 2):294-303. https://doi.org/10.1007/s10461-016-1364-3        [ Links ]

31. Matava CT, Alam F, Kealey A, Bahrey LA, McCreath GA, Walsh CM. The influence of resident and faculty gender on assessments in anesthesia competency-based medical education. Can J Anaesth 2023;70(6):978-987. https://doi.org/10.1007/s12630-023-02454-x        [ Links ]

32. Kozato A, Patel N, Shikino K. A randomised controlled pilot trial of the influence of non-native English accents on examiners' scores in OSCEs. BMC Med Educ 2020;20(1):268. https://doi.org/10.1186/s12909-020-02198-y        [ Links ]

33. Royce CS, Morgan HK, Baecher-Lind L, et al. The time is now: addressing implicit bias in obstetrics and gynecology education. Am J Obstet Gynecol 2023;228(4):369-381. https://doi.org/10.1016/j.ajog.2022.12.016        [ Links ]

 

 

Correspondence:
B Mrara
bmrara@wsu.ac.za

Received 9 September 2024
Accepted 14 November 2024