RESEARCH METHODOLOGY AND STATISTICS

 

The implementation of the South African Orthopaedic Registry (SAOR): factors that improve usage

 

 

Hannah T Bussio^I, ; Xan Swart^II; Megan O'Connor^III; Sivesonke Khanyile^I; Odette Koch^IV

^IDivision of Orthopaedics, Dr George Mukhari Academic Hospital, Sefako Makgatho Health Sciences University, Pretoria, South Africa
^IIDivision of Sports Sciences, University of Pretoria, Pretoria, South Africa
^IIIDivision of Orthopaedics, Harry Gwala Regional Hospital, University of KwaZulu-Natal, Pietermaritzburg, South Africa
^IVDivision of Orthopaedics, 1 Military Hospital, University of Pretoria, Pretoria, South Africa

 

 

---

ABSTRACT

BACKGROUND: Registries have been created to accrue comprehensive clinical data for evaluation, with purported benefits for clinical governance, public health and scientific investigation. As has been the experience internationally, orthopaedic surgeon contribution to data collection for the South African Orthopaedic Registry (SAOR) has been moderate. Due to this response, several awareness initiatives were implemented by the South African Orthopaedic Association (SAOA) to encourage surgeons to engage with SAOr. This study aimed to evaluate the efficacy of these initiatives.
METHODS: This retrospective study evaluated SAOR's operational data between 1 August 2019 and 30 June 2022. For each month, the number of new surgeons registering, new patients captured, and new registry pathways initiated were evaluated. These monthly increases were evaluated relative to the implementation of several awareness initiatives. Multiple linear regression was performed to determine if any initiative significantly increased recruitment rates. The Strengthening the Reporting of Observational studies in Epidemiology (STROBE) checklist was used for reporting.
RESULTS: Thirty-five months of SAOR operational data were analysed. The cumulative number of surgeons enrolled to use the registry by June 2022 was 108, the number of patients captured 9 992, and the number of pathways initiated 9 585. Thirty-three awareness initiatives of four types took place in the period from January 2021 until June 2022. No interventions increased the recruitment of surgeons. Group promotional sessions and virtual group demonstrations significantly increased the number of patients captured on the registry. Group promotional sessions significantly increased the number of pathways initiated.
CONCLUSION: The SAOR was established in 2019 by the SAOA with the intent to improve orthopaedic practice in South Africa. Several awareness initiatives have been implemented to engage surgeon contributions to the registry, none of which increased enrolment of new surgeons, but some of which increased the number of patients captured and pathways initiated. Ways to improve surgeon engagement with registries should be the focus of future research.
Level of evidence: Level 4

Keywords: registry, clinical, patient, South Africa, orthopaedics

---

 

 

Introduction

The terms 'patient registry', 'clinical registry', 'clinical data registry', 'disease registry' and 'outcomes registry' are used interchangeably to denote the collection of data within the healthcare setting, herein referred to as 'registry'.¹ The Agency for Healthcare Research and Quality (AHRQ) defines a registry as 'an organized system that uses observational study methods to collect uniform data (clinical and other) to evaluate specified outcomes for a population defined by a particular disease, condition, or exposure, and that serves one or more predetermined scientific, clinical, or policy purposes'.¹ Registries are distinct from other forms of medical data collection on account of the focused collection of predetermined data variables for individuals with a defined common feature of interest, taking place across multiple sites.²⁴ Registries have been created to accrue comprehensive clinical data for evaluation, with purported benefits for the clinical governance of patient care, public health and scientific investigation.⁵⁷ Traditionally, registries were developed by organisations or researchers who were responsible for the collection of data and analysis for various utilities.¹ As registries have evolved, patients have become more involved in the establishment of registries and capturing their own data, in what are termed 'patient-powered registries'.¹ Gliklich et al. summarised the types of registries into three variants according to the purpose of data collection: 1) those related to monitoring patients with a specific disease or condition, 2) registries concerned with administered patient care that monitor the investigation, management and outcomes of treatment, and 3) those where a healthcare product is the focus of interest.⁸

Orthopaedic registries largely fall into the latter group, and are concerned with implantable medical devices such as those used in joint replacement surgery.⁹ The first joint replacement registry is credited to the Mayo Clinic, established in 1969.⁹ Numerous benefits for the collection and analysis of registry data for orthopaedic joint replacement surgery have subsequently been elucidated. While not comparable to costly, large scale, randomised controlled trials in terms of scientific rigour, Robertsson reasoned that registries allow identification of subtle differences in implant efficacy and early detection of failure, paramount to monitoring the outcomes of implantable products.^10,11 In some instances, registry findings have identified products performing poorly, and provided justification for their removal from the market.^12,13 Furthermore, registries hold a wealth of information that can be used to examine research hypotheses and test associations.¹¹ Information such as this can lead to changes in practice and improved patient care and outcomes.¹⁴ Recognising the value of large volume data, several countries have sought to develop national joint replacement registries.^15,16 In some instances, joint registries have been expanded to include coverage for implantable devices in other orthopaedic subspecialties.¹⁷

Following the success of numerous international joint registries, members of the South African Orthopaedic Association's (SAOA) executive committee established the South African National Joint Registry (SANJR) in 2012.¹⁸ As has been the experience internationally, barriers to the growth of the registry were experienced, including sustainable funding for registry maintenance, surgeon compliance with data contribution, and poor data quality.¹⁸ A European initiative outlined several of the key issues with registries based on survey feedback from a number of European medical registries.¹⁹ Finding consistent funding sources was the lead issue raised among the survey participants.¹⁹ Several issues regarding the data itself were also raised, including the responsibilities for data entry, data ownership and the quality or completeness of the data captured.¹⁹ The need for evidence-based guidelines for orthopaedic management in South Africa during the COVID-19 pandemic highlighted once more the beneficial role of registries: for researching outcomes in order to enhance orthopaedic practice and patient care, not only for arthroplasty, but for all orthopaedic specialties.²⁰ Subsequently, the South African Orthopaedic Registry (SAOR) was initiated, funded by the SAOA.

The SAOR utilises Pro Registry™ software on a licence purchased by the SAOA, offered by Amplitude™ Clinical Services Limited incorporated registry software (Worcestershire, England). All paid-up surgeon members of the SAOA (collectively termed surgeons) have access to the registry. The data entered on the registry is hosted on a secure server. The surgeons have access to information they input and may request access to other surgeon data by agreement between surgeons or by formal request to the SAOR steering committee. The previous SANJR joint replacement data was carried over. The SAOA has assigned the SAOR steering committee members to be data controllers, responsible for reviewing the purpose for and ethical permissions of all data requests. Ethical oversight of the SAOR is held by Stellenbosch University Health Research Ethics Committee (HREC). The new database incorporates additional avenues for data collection for each orthopaedic specialty, and includes a patient portal where patient-reported outcome measures can be captured. Furthermore, at the time that a patient's details are input into the registry, they can be initiated on one or more pathways by the treating surgeon. Pathways are predefined information groups with specific data points for collection relevant to a specific orthopaedic condition, for example, a hip replacement pathway or a carpal tunnel pathway. Pathway data collection represents longitudinal data capture for several visits, admissions or surgeries.

Acknowledging the limitations of the former SANJR, and, working on the assumption that the updated registry format would itself not improve registry utility, several initiatives were implemented to raise awareness and encourage user engagement of the SAOR by orthopaedic surgeons nationally. This cross-sectional study aimed to evaluate the efficacy of these initiatives by analysing if there was a significant change in the rate of utilisation of the registry following their implementation. This was assessed by determining if there was a significant change in the monthly number of new surgeons, new patients, or new pathway inputs, following the implementation of each of the initiatives.

 

Methods

This retrospective cross-sectional study evaluated SAOR's operational data between 1 August 2019 and 30 June 2022. For each month, the number of new surgeons registered, the number of new patients captured, and the number of new registry pathways initiated were recorded. These numbers were evaluated following implementation of several awareness initiatives, to determine if the initiatives resulted in heightened engagement with the registry. The Strengthening the Reporting of Observational studies in Epidemiology (STROBE) checklist was used for reporting.²¹

Following appropriate applications to the Stellenbosch University HREC, and Sefako Makgatho University Research Ethics Committee (SMUREC) and approval, the deidentified requested data was released for analysis. This data was provided by Amplitude™ in the form of monthly reports. The reports quantify the total or cumulative number of surgeons, patients and pathways (collectively termed participants). Also, the number of new surgeons, new patients and new pathways (collectively termed new recruitments) were reported at monthly intervals. The number of new surgeons was identified by the number of new surgeon enrolments to the registry, for data input and access. The number of new patients were identified by the number of new patient demographic profiles entered, prior to initiation of a patient on a pathway. Similarly, the number of new pathways initiated for each month were recorded. These new recruitments did not necessarily have completed records; there is no set minimum data requirement for surgeon or patient profiles or initiation on a pathway. Repeated patient entry is prevented by an alert that appears when attempting to capture the details of a patient already in the database. Repeated pathways are permissible for a single patient to allow for multiple orthopaedic conditions in the same patient.

The awareness initiatives (collectively termed interventions) consisted of four varieties. Broadcast emails were sent to all SAOA members to encourage registry utilisation. Virtual group sessions were held. These were online interactive demonstrations on registry use delivered by the software developers and members of the SAOR steering committee, which included question and answer components. Similarly, there were face-to-face group demonstration sessions of the registry, and question-answering sessions conducted by members of the SAOR steering committee. Finally, group promotion sessions comprised SAOA branch meetings, congress in-person presentations or advertisements, delivered by members of the SAOR steering committee or SAOA members.

Data analysis encompassed review of the total number of participants, the recruitment rate or monthly increase in number of participants (new recruitments per month), and the cumulative recruitment of participants. The estimated increase of recruitments per month, related to each intervention, were evaluated, and comprised the outcome of interest. This was performed comparing the period preceding the initiation of interventions (August 2019 until December 2020) and the period following this (January 2021 until June 2022), termed the pre- and post-intervention periods. All analyses were conducted in R version 4.2.2 (R Core Team, Vienna, Austria).²² Bivariate analysis using a paired t-test examined if there existed a statistically significant difference between the pre-intervention period and the post-intervention period in terms of the monthly increase in number of participants per month. This analysis was intervention agnostic. Multiple linear regression concluded the analysis. The four interventions were modelled simultaneously to quantify the extent to which each component influenced the rate increase per month. Alpha was considered significant if it was less than (not equal to) 0.05.

 

Results

Description of participants and interventions

Thirty-five months of SAOR operational data were analysed. As at the end of June 2022, the cumulative number of surgeons enrolled to use the registry were 108, the number of patients captured 9 992 and the number of pathways initiated 9 585 (Figure 1). A total of 31 interventions took place at irregular intervals starting from February 2021. These comprised five broadcast emails, 12 virtual group demonstration sessions, one face-to-face group demonstration and 13 group promotion sessions.

Efficacy of interventions

There was a significant increase in the monthly number of patients and pathways captured in the period post initiation of interventions, signifying an increased rate of recruitment as compared to the preceding period that was absent of any interventions (Table I). No change in surgeon recruitment rate was observed following the implementation of the interventions.

Multiple linear regression models estimated if the monthly increase in recruitments was related to the individual intervention types. Group promotion sessions conducted at SAOA branch meetings and the SAOA congress were significantly associated with increased initiation of pathways. Similarly, both virtual group demonstrations and the group promotion sessions were associated with a significant increase in patients captured. No individual intervention increased the number of surgeons enrolling to utilise the registry (Table II). Figure 2a depicts the timing and number of interventions relative to the cumulative increase in patients and pathways following the interventions, while Figure 2b depicts the same relative to the cumulative number of surgeons.

 

Discussion

This study aimed to assess the efficacy of various awareness initiatives led by the SAOR steering committee aimed at promoting awareness of the SAOR and enhancing surgeon engagement. We found that none of the previously conducted interventions increased the numbers of surgeons enrolling to use the registry, but that virtual group demonstrations and group promotional sessions increase the number of patients entered in the registry; and that group promotional sessions also significantly increase the number of pathways initiated.

It is concerning that no interventions improved the enrolment of surgeons to use the registry, but it is perhaps not unexpected. Lee, in an editorial regarding the role that registries can play in patient safety, explained that surgeons need to 'buy in' to the process.²³ By this he referred to the surgeons being willing to share their data (including complications) for potential scrutiny. He explained that data sharing could benefit patient care but also the individual surgeons who would be able to compare their results to the average, identify substandard and above-average performance, and attend to these areas accordingly.²³ Secondly, Lee acknowledged the need for resources for data entry and collation.²³ The South African healthcare system faces numerous challenges, including lack of resources and staff attrition, which likely contribute to surgeons focusing on the immediate delivery of care with less time available to attend to quality improvement.²⁴ Furthermore, in the era of electronic health records, which have previously been elucidated as burdensome to physicians and contributory to the prevalence of burnout, it is foreseeable that orthopaedic surgeons do not want to take on additional administrative tasks.^25,26

Despite these challenges, increasing surgeon engagement with the SAOR should be pursued, considering the myriad benefits that registries offer. A systematic review conducted by Mandavia et al., entitled 'What are the essential features of a successful surgical registry?', identified several approaches to engaging clinician participation and completion of records.²⁷ The SAOR currently applies several of these approaches, and several more can still be employed. Features of the SAOR known to increase engagement include the fact that it is hosted on an online software platform which is available on multiple browsers.²⁷ In addition, the SAOR has the capacity to generate printable operative notes and compile graphical or data summaries of the surgeons' practice directly from the data entered, both of which may be perceived as clinically valuable to surgeons.²⁷ A registry platform that can be utilised for research has also been shown to increase participation, and is another strength attributable to the SAOR.²⁷ It can be argued that the SAOR is too comprehensive and not as intuitive as other registries, and that these drawbacks may limit surgeon engagement. While the SAOR platform design permits the entry of an exhaustive number of variables for any detail the surgeon deems pertinent to a patient, it must be emphasised that there are no minimum data restrictions, and surgeons should consider that successful registries feature a succinct predefined set of pertinent variables to speed the data entry process.²⁷ Additional proposals for consideration to improve surgeon engagement include the issue of certificates based on the level of contribution each surgeon has made, as have been implemented in other national registries.²⁸ However, most effective according to previous investigation, is the implementation of a mandatory system of data completion to maintain professional council or association membership.^27,29

While the interventions did not increase the number of surgeons, there was an indication that group promotion sessions and virtual group demonstration sessions increased the number of patients captured and the number of pathways initiated. This is an encouraging confirmation that while interventions do not achieve the desired surgeon engagement, they may stimulate already registered surgeons to utilise the registry more effectively. This may mean that, prior to the interventions, surgeons were not aware of the full utility of the registry, or that surgeon interest in the registry was renewed through promotion. In both cases, ongoing demonstrations and promotions show benefit for continued engagement and support of contributing surgeons.

Two potential limitations should be considered when interpreting the results. First, the investigation aimed to assess the recruitment rate of new surgeons, patients and pathways relative to awareness initiatives. The recruitment of patients and pathways particularly did not necessarily represent completed data entry for each record but were rather an indicator of the initiation of a new record for a patient or a pathway. This means that the patient and pathway numbers reported herein likely over-estimate the clinically valuable data within the registry. The quality of data captured was, however, not the focus of investigation in this study. Secondly, the analysis for efficacy of each intervention assumed that the effect would happen in the same month as the intervention. This assertion was made based on the consideration that the effect was likely greatest in the period immediately post the intervention. This analysis was further complicated by the fact that interventions took place on individual days but recruitment rates were calculated by month. This means that our findings may or may not reflect the real effect, but are likely to be a fair representation of complex human behavioural psychology.

 

Conclusion

The SAOR, representing the updated SANJR inclusive of all orthopaedic subdisciplines and a patient portal, was established in 2019 by the SAOA with the intent to improve orthopaedic practice in South Africa. Several awareness initiatives have been implemented to engage surgeon contributions to the registry, none of which increase enrolment of new surgeons, but some of which increase the number of patients captured and pathways initiated. Future research directed at improving registry utilisation, that explore mandated use and incentives for data capture, is suggested.

Acknowledgements

The authors wish to acknowledge the contributions of Prof. Marilize Burger in the maintenance of the ethical approval for data collection for the South African Orthopaedic Association Registry. They also wish to acknowledge the members of the SAOR steering committee for approval to conduct this project.

Ethics statement

The authors declare that this submission is in accordance with the principles laid down by the Responsible Research Publication Position Statements as developed at the 2nd World Conference on Research Integrity in Singapore, 2010. The study complied with the South African Department of Health ethics guidelines (2015), as well as Stellenbosch University Health Research Ethics Committee and Sefako Makgatho University Research Ethics Committee principles. The overseeing ethics committee for all data collection and release for the South African Orthopaedic Association Registry is Stellenbosch University Health Research Ethics Committee (HREC), reference number B20/08/003 and project ID 15535. However, prior to commencement of this research, the appropriate ethical approval was obtained from the Sefako Makgatho University Research Ethics Committee (SMUREC), reference SMUREC/M/200/2022: PG and project ID 0491.

All procedures were in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975, as revised in 2008.

Declaration

The authors declare authorship of this article and that they have followed sound scientific research practice. This research is original and does not transgress plagiarism policies.

Author contributions

HTB: protocol development, data collection, manuscript preparation, manuscript revision

XS: protocol development (statistics), data analysis, manuscript preparation

MO: study conceptualisation, protocol development, manuscript revision and review

SK: protocol review, manuscript review

OK: protocol development, data collection, manuscript preparation and manuscript review

ORCID

Busslo HT https://orcld.org/0000-0001-9933-4198

Swart X https://orcld.org/0000-0002-9555-4254

O'Connor M https://orcld.org/0000-0001-8864-4916

Khanylle S https://orcld.org/0000-0002-0799-770X

Koch O https://orcld.org/0000-0003-1871-9569

 

References

1. Workman T. Defining patient registries and research networks. Engaging patients in information sharing and data collection: the role of patient-powered registries and research networks. US: Agency for Healthcare Research and Quality, 2013. PMID: 24156118.         [ Links ]

2. Donaldson L. Registering a need. BMJ. 1992;305:597-98. https://doi.org/10.1136/bmj.305.6854.597        [ Links ]

3. Newton J, Garner S. Disease registers in England. Institute of Health Sciences. University of Oxford 2002.         [ Links ]

4. Drolet BC, Johnson KB. Categorizing the world of registries. J Biomed Inform. 2008;41:1009-1020. https://doi.org/10.1016/j.jbi.2008.01.009        [ Links ]

5. Heger M. A registry of registries? The US backs the idea for patients. Nat Med. 2011;17:4-4. https://doi.org/10.1038/nm0111-4a        [ Links ]

6. Genetic and Rare Disease Information Centre, https://rarediseases.info.nih.gov/about (2023, accessed 27 April 2023).         [ Links ]

7. Pop B, Fetica B, Blaga ML, et al. The role of medical registries, potential applications and limitations. Med Pharm Rep. 2019;92:7-14. 20190115. https://doi.org/10.15386/cjmed-1015        [ Links ]

8. Gliklich RE, Dreyer NA, Leavy MB. Registries for evaluating patient outcomes: a user's guide. 2014. PMID: 24945055.         [ Links ]

9. Delaunay C. Registries in orthopaedics. J Orthop Surg Res. 2015;101:S69-S75. https://doi.org/10.1016/j.otsr.2014.06.029        [ Links ]

10. Robertsson O. Knee arthroplasty registers. J Bone Joint Surg Br. 2007;89-B:1-4. https://doi.org/10.1302/0301-620x.89b1.18327        [ Links ]

11. Inacio MC, Paxton EW, Dillon MT. Understanding orthopaedic registry studies: a comparison with clinical studies. J Bone Joint Surg Am. 2016;98:e3. https://doi.org/10.2106/bjsN.01332        [ Links ]

12. Maloney WJ. National joint replacement registries: has the time come? JBJS. 2001;83:1582-85.         [ Links ]

13. Furnes O, Lie SA, Havelin LI, et al. Exeter and charnley arthroplasties with Boneloc or high viscosity cement. Comparison of 1,127 arthroplasties followed for 5 years in the Norwegian Arthroplasty Register. Acta Orthop Scand. 1997;68:515-20. https://doi.org/10.3109/17453679708999017        [ Links ]

14. De Steiger R. A critical examination of the Australian Orthopaedic Association National Joint Replacement Registry: Improving outcomes of hip and knee replacement. The University of Adelaide, Australia, 2018.         [ Links ]

15. Herberts P, Ahnfelt L, Malchau H, et al. Multicenter clinical trials and their value in assessing total joint arthroplasty. In: Clinical Orthopaedics and Related Research 1989, pp.48-55.         [ Links ]

16. Lucht U. The Danish Hip Arthroplasty Register. Acta Orthop Scand. 2000;71:433-39. https://doi.org/10.1080/000164700317381081        [ Links ]

17. Gressler LE, Devlin V, Jung M, et al. Orthopedic Coordinated Registry Network (Ortho-CRN): advanced infrastructure for real-world evidence generation. BMJ Surg Interv Health Technol. 2022;4:e000073. 20221111. https://doi.org/10.1136/bmjsit-2020-000073        [ Links ]

18. De Vos J. South African National Joint Registry: First Annual Report. 2016 2016. South Africa: South African National Joint Registry.         [ Links ]

19. Methodological guidelines and recommendations for efficient and rational governance of patient registries. In: Zaletel M and Kralj M, (eds.). Cross-border patient registries initiative Ljubljana: National Institute of Public Health, 2015, p. 232.         [ Links ]

20. Bernstein B. The Art of War'. SA Orthop J. 2021;20(2):65-65.         [ Links ]

21. Von Elm E, Altman DG, Egger M, et al. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. BMJ. 2007;335:806-808. https://doi.org/10.1136/bmj.39335.541782.AD        [ Links ]

22. The R Project for Statistical Computing. In: Team RC, (ed.). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing, 2022.         [ Links ]

23. Lee MJ. Safety in surgery: the role for registries. Clin Orthop Relat Res. 2013;471:2743-45. 20130606. https://doi.org/10.1007/s11999-013-3093-5        [ Links ]

24. Aikman N. The crisis within the South African healthcare system: A multifactorial disorder. S Afr J Bioethics Law. 2019;12:5.         [ Links ]

25. Gaffney A, Woolhandler S, Cai C, et al. Medical documentation burden among US office-based physicians in 2019: a national study. JAMA Intern Med. 2022;182:564-66. https://doi.org/10.1001/jamainternmed.2022.0372        [ Links ]

26. LI C, Parpla C, Srlharan A, et al. Electronic medical record-related burnout In healthcare providers: a scoping review of outcomes and interventions. BMJ Open. 2022;12:e060865. https://doi.org/10.1136/bmjopen-2022-060865        [ Links ]

27. Mandavia R, Knight A, Phillips J, et al. What are the essential features of a successful surgical registry? a systematic review. BMJ Open. 2017;7:e017373. https://doi.org/10.1136/bmjopen-2017-017373        [ Links ]

28. Lenguerrand E, Whitehouse M, Beswick A, et al. Revision for prosthetic joint infection following hip arthroplasty: Evidence from the National Joint Registry. Bone Jt. Res. 2017;6:391-98.         [ Links ]

29. Mangera A, Parys B. BAUS Section of Endourology national Ureteroscopy audit: setting the standards for revalidation. J Clin Urol. 2013;6:45-49. https://doi.org/10.1177/1875974212465536        [ Links ]

 

 

Received: August 2023
Accepted: November 2023
Published: May 2024

 

 

Editor: Prof. Marilize Burger, Stellenbosch University, Cape Town, South Africa
Funding: No funding was received for this study.
Conflict of interest: HT Bussio, X Swart and S Khanyile declare they have no conflicts of interest that are directly or indirectly related to the research.
O Koch and M O'Connor are/have been members of the South African Orthopaedic Association and its steering committee for the SAOR. Release of data for the purpose of the study was unanimously approved by members of the steering committee, following the standard operating procedure guidelines for data release, as outlined in the approved protocol for SAOA Registry data collection by Stellenbosch University Health Research Ethics Committee (HREC), reference number B20/08/003 and project ID 15535.
* Corresponding author: bussiohannah@gmail.com