EDITORIAL

 

Workplace-based assessment - a new era of surgical training competency assessment in South Africa

 

 

D Nel; L Cairncross; V Burch; E Burch; L Green-Thompson; ZM Koto; OD Montwedi; J Fagan; E Jonas

On behalf of the leaders of the South African Committee ofMedical Deans, College ofMedicine of South Africa, and the College of Surgeons - working together to facilitate the implementation of Workplace-based assessment in South Africa

Correspondence

 

 

Surgical training has traditionally been based on the Halstedian apprenticeship model.¹ In this model, certification is dependent on completing training under the guidance of a specialist surgeon and passing final written and oral examinations that primarily assess theoretical knowledge. In order to train specialists who are fit for purpose,² it is crucial that we move beyond the assessment of biomedical knowledge only and build capacity to test the full spectrum of competencies required for daily clinical practice, including surgical capability.³ In addition, changes to modern healthcare environments and greater numbers of trainees have resulted in the apprenticeship model no longer being a sustainable strategy for training surgeons. Furthermore, increasing pressures on operating room availability, ethical concerns of trainees training on patients, and escalation of litigation for medical errors require the integration of innovative strategies into training programmes and examining bodies to optimise training and trustworthy certification of competent surgeons.⁴

 

Competency-based medical education

An early attempt to measure competence, specifically the ability to perform procedures, was through the use of a logbook, which was introduced in the 1990s. However, as logbooks only record exposure and activity, rather than competence, they have limited content validity.¹ Being present at a procedure does not mean that a trainee is necessarily competent to perform that procedure or task independently.

Competency-based medical education (CBME), which came to the fore in the last 20 years, aims to ensure that graduates attain the minimum standards for unsupervised practice in their field.⁵ While traditional approaches to training and assessment focused on lists of knowledge objectives, in CBME outcomes guide all curriculum and assessment decisions. Furthermore, the primary method of confirming competence for unsupervised practice is through workplace-based assessment (WBA), which has been defined as "the assessment of working practices based on what trainees actually do in the workplace, and predominantly carried out in the workplace itself."⁶ By providing the opportunity for direct assessment of what trainees actually do in the clinical environment, WBA has greater face validity than other assessment methods that attempt to measure real-world competence. Evidence for its validity, acceptability and educational impact in surgery is growing, and it is increasingly being adopted globally.⁷

Rollout of assessment in the workplace is underpinned by a clear understanding of the entrustable professional activities (EPAs) of a discipline, the "units of professional practice (tasks or bundles of tasks)"⁸ that can be entrusted to an individual once they have demonstrated the necessary competence to execute them. EPAs, first described by Ten Cate almost 20 years ago,⁹ make use of rating scales anchored in entrustment rather than arbitrary numbers or percentages.¹⁰ A wide range of tools is currently being used to assess EPAs in the workplace, including the mini-clinical evaluation exercise (Mini-CEX), Ottawa clinic assessment tool (OCAT), case-based discussion (CBD), objective standardised assessment of technical skill (OSATS), and the Ottawa surgical competency in the operating room evaluation (O-SCORE).¹¹ Rather than being prescriptive of content, most of these tools provide flexible frameworks that can be populated according to the needs and realities of the healthcare systems within which training takes place. Furthermore, changes over time can be made to adapt training to contemporary healthcare requirements and changing realities and challenges in health systems.

An important aspect of WBA in the context of CBME is that the individual assessments, also termed observations, are low stakes and are primarily intended to be formative. Thus, assessment in the workplace is used for learning, and not just as an assessment of learning.¹²¹³ Key to this formative strategy is the delivery of repeated and timely feedback to the trainee after observation of an activity by supervisors. High stakes decision making, such as to determine progression through training and eligibility to undertake specialist and subspecialist certification examinations offered by the Colleges of Medicine of South Africa (CMSA), is then assessed by a group of faculty surgeons, often referred to as a 'competence committee'. Such committees typically meet 3-6 monthly and review trainees' portfolios of learning, including WBA observations and other relevant documentation, to evaluate progression towards competence across the prescribed domains of surgical practice.

Workplace-based assessment in South Africa

The development and implementation of a WBA strategy for postgraduate training of specialists and subspecialists in South Africa, is being actioned by a national WBA steering committee, led by the South African Committee of Medical Deans and supported by the CMSA. This complex task includes defining appropriate and achievable EPAs for specialist training, the development of guidelines and adoption of suitable WBA observation tools, building capacity for implementation in individual training institutions, facilitating the integration of WBA into existing university-based training programmes, and identifying or developing a technology platform for national use. This generic approach will need customisation for each discipline.

The Steering Committee is aiming for a process of stepwise implementation of WBA across all specialties over a 12-24-month period. One of the early steps will require specialist colleges and departments to form working groups to determine lists of nationally agreed-upon EPAs for their discipline. This process requires consultation with a wide range of stakeholders including clinicians with educational expertise. Although the surgical colleges will agree on the list of EPAs, the development of WBA and institutional implementation will be the responsibility of the universities. It is anticipated that trainees undertaking specialist certification examinations offered by the CMSA will require institutional verification of competence for the specified EPAs of their discipline based on WBA.

WBA is not a fad, or an onerous addition to the already heavy workload shouldered by busy clinicians, but rather a catalyst that moves us beyond the realms of theoretical learning and assessment to achieving more well-rounded, empathetic and competent specialists. The fundamentals of WBA design in South Africa are being developed in an inclusive manner to ensure that the final outcome for implementation is acceptable, appropriate and feasible for all stakeholders, including trainees, trainers, training institutions and examining bodies. Change is not always comfortable or easy, but we urge the university surgical community at large to embrace this initiative which provides an invaluable opportunity for the implementation of modern assessment methods, thus reaffirming South African universities as training institutions of excellence, producing world-class clinicians. We are truly at the dawn of a new era of surgical training competency assessment in South Africa.

ORCID

D Nel https://orcid.org/0000-0002-3265-1049

L Cairncross https://orcid.org/0000-0001-5368-9882

L Green-Thompson https://orcid.org/0000-0002-2950-9527

ZM Koto https://orcid.org/0000-0002-9274-2508

J Fagan © https://orcid.org/0000-0001-9621-1248

Ε Jonas © https://orcid.org/0000-0003-0123-256X

 

REFERENCES

1. Balasundaram I, Aggarwal R, Darzi LA. Development of a training curriculum for microsurgery. Br J Oral Maxillofac Surg. 2010;48(8):598-606. https://doi.org/10.1016/j.bjoms.2009.11.010.         [ Links ]

2. Sathekge MM. Work-based assessment: a critical element of specialist medical training. S Afr Med J. 2017;107(9):12059. https://doi.org/10.7196/SAMJ.2017.v107i9.12655.         [ Links ]

3. Health Professions Council of South Africa. Core competencies for undergraduate students in clinical associate, dentistry and medical teaching and learning programmes in South Africa. [Internet]. Available from: https://www.hpcsa-blogs.co.za/wp-content/uploads/2017/04/MDB-Core-Competencies-ENGLISH-FINAL-2014.pdf.         [ Links ]

4. Nestel D, Dalrymple K, Paige JT, Aggarwal R. Advancing surgical education: theory, evidence and practice. Springer Nature. 2019. https://doi.org/10.1007/978-981-13-3128-2.         [ Links ]

5. Frank JR, Snell LS, Cate OT, et al. Competency-based medical education - theory to practice. Med Teach. 2010;32(8):638-45. https://doi.org/10.3109/0142159X.2010.501190.         [ Links ]

6. Beard J. Workplace-based assessment: the need for continued evaluation and refinement. Surgeon. 2011;9(Suppl 1):S12-13. https://doi.org/10.1016/j.surge.2010.11.014.         [ Links ]

7. Aryal K, Hamed M, Currow C. The usefulness of work-based assessments in higher surgical training: a systematic review. Int J Surg. 2021;94:106127. https://doi.org/10.1016/j.ijsu.2021.106127.         [ Links ]

8. Ten Cate O, Taylor DR. The recommended description of an entrustable professional activity: AMEE Guide No. 140. Med Teach. 2021;43(10):1106-14. https://doi.org/10.1080/0142159X.2020.1838465.         [ Links ]

9. Ten Cate O. Entrustability of professional activities and competency-based training. Med Educ. 2005;39(12):1176-7. https://doi.org/10.1111/j.1365-2929.2005.02341.x.         [ Links ]

10. Ten Cate O, Chen HC, HoffRG, et al. Curriculum development for the workplace using entrustable professional activities (EPAs): AMEE Guide No. 99. Med Teach. 2015;37(11):983-1002. https://doi.org/10.3109/0142159X.2015.1060308.         [ Links ]

11. Hackney L, O'Neill S, O'Donnell M, Spence R. A scoping review of assessment methods of competence of general surgical trainees. Surgeon. 2022:S1479-666X(22)00027-0. https://doi.org/10.1016/j.surge.2022.01.009.         [ Links ]

12. Bok HG, Teunissen PW, Favier RP, et al. Programmatic assessment of competency-based workplace learning: when theory meets practice. BMC Med Educ. 2013;13:123. https://doi.org/10.1186/1472-6920-13-123.         [ Links ]

13. Norcini J, Burch V. Workplace-based assessment as an educational tool: AMEE Guide No. 31. Med Teach. 2007; 29(9):855-71. https://doi.org/10.1080/01421590701775453.         [ Links ]

 

 

Correspondence:
email: danielnel87@gmail.com