RESEARCH

 

Development and validation of an instrument for procedure-related death notification

 

 

F K Orr^I; C Keyes^II; A Alli^III

^IMB ChB, DA (SA), DCH (SA); Department of Anaesthesia and Perioperative Medicine, Faculty of Health Sciences, University of Cape Town, South Africa
^IIBSc, PDSS, BHSc Hons, MSc (Med); Department of Forensic Medicine and Pathology, Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, South Africa
^IIIMB BCh, MMed, FCA (SA), Cert Crit Care (SA); Department of Anaesthesia, Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, South Africa

Correspondence

 

 

---

ABSTRACT

BACKGROUND. Anaesthesia-related mortality is an important, potentially avoidable cause of perioperative mortality. A procedure-related death notification (PRDN) instrument is completed by relevant medical practitioners after a procedure-related death and is used to audit practice and identify areas of care that require improvement. It is also used in medicolegal investigations when establishing cause of death, and in the case of litigation. The current South African (SA) PRDN instrument, designated the GW7/24 form, contains both surgical and anaesthetic sections and is considered to be outdated, inadequate and in need of revision.
OBJECTIVES. To develop and validate a revised anaesthetic section of the SA PRDN instrument that can be used for procedure-related deaths in future and be used to update the GW7/24 form for epidemiological, forensic or academic use.
METHODS. Lynn's two-stage model was utilised. After an extensive literature review, a provisional PRDN instrument was developed. This provisional instrument was debated and reviewed at a peer group discussion in which 6 local experts took part. These experts were anaesthetic and forensic pathology specialists who specifically have expert knowledge on procedure-related deaths. A revised PRDN instrument was developed, which was then rated by 8 national experts using a Likert scale. The content validity index (CVI) for each item and for the instrument as a whole was then established. Items with a CVI <0.88 were removed to formulate the final PRDN instrument.
RESULTS. The provisional PRDN instrument consisted of 14 domains and 66 items. The revised PRDN instrument consisted of 13 domains and 65 items, of which 3 items with a CVI <0.88 were removed. The final PRDN instrument, after minor revisions based on suggestions from the 8 national experts, consisted of 18 domains and 79 items. Every item on the form was declared relevant and important by the national experts, with the final instrument scoring an overall CVI of 1.
CONCLUSIONS. A comprehensive, updated and validated anaesthetic section of the SA PRDN instrument was developed. This could be used as a government and anaesthesiology society-endorsed template when updating the current GW7/24 form.

---

 

 

Anaesthesia-related mortality is an important, potentially avoidable cause of perioperative mortality. The current worldwide anaesthesia-related mortality rate has decreased in recent years to <1 death per 10 000 anaesthetics.^[1] Among the reasons for this drop are better monitoring techniques,^[1-3] improved postoperative monitoring and care in the recovery room,^[2] better training of anaesthesiologists^'[3,4] and the adoption of safety standards and practice guidelines.^[1-3] These improvements can be attributed to the identification of problem areas by conducting mortality and morbidity audits.^['5]

A procedure-related death notification (PRDN) instrument is completed by relevant medical practitioners after an anaesthetic-related death and can be used to audit practice and identify areas of care that need improvement. It is also used in medicolegal investigations when establishing cause of death, and in the case of litigation. The current South African (SA) PRDN instrument, the GW7/24 form (Report of Person Whose Death is Associated with the Administration of an Anaesthetic or a Diagnostic or Therapeutic Procedure), originally published in the 1970s, is anecdotally considered to be outdated, inadequate and in need of revision.

 

Objectives

To develop and validate a new anaesthetic section of the SA PRDN instrument that can be used for any procedure-related death in the future, for epidemiological, forensic or academic purposes.^[6] The revision of the surgical section of the SA PRDN instrument was not within the scope of this study.

 

Methods

Approval to conduct the study was obtained from the Human Research Ethics Committee (Medical) (ref. no. M160684) and the Postgraduate Committee of the University of the Witwatersrand, Johannesburg.

The study comprised a two-stage process of instrument development and validation using Lynn's methodology, consisting of a development stage and a judgement-quantification stage.^[7]

Development stage

The development stage comprised three steps: domain identification, item generation and instrument formation. An extensive literature review was performed (domain identification) and a provisional PRDN instrument was produced (item generation). The literature review focused on extracting factors that are considered to be important in anaesthesia- and procedure-related deaths. Both locally and internationally published anaesthesia morbidity and mortality audits and PRDN instruments were included in the literature search. These extracted factors were then included in the provisional PRDN instrument. The instrument formation step involved a review of the provisional PRDN instrument, which was debated in a peer group discussion consisting of 6 experts.

According to Lynn,^[7] the number of experts who should be used in this stage often depends on the availability of accessible and agreeable experts, but a minimum of 5 would provide an adequate level of control to rule out chance agreement. The maximum number of experts has not been established, but is unlikely to be more than 10. Six experts participated in the peer group discussion in this study: 4 anaesthesiologists and 2 forensic pathologists. The experts in each stage were chosen in consultation with senior anaesthetic and forensic pathology consultants. They were chosen based on their knowledge of procedure-related deaths and the medicolegal processes surrounding these deaths in SA and elsewhere. They consisted of anaesthesiologists and forensic pathologists working in SA who were willing to participate in the study. This group of experts were only used in the development stage and were not involved in the later judgement-quantification stage.

The provisional PRDN instrument and information sheet were sent to each participating expert in advance. All 6 invited experts participated in the discussion, which was chaired by the principal investigator and held at Charlotte Maxeke Johannesburg Academic Hospital, Johannesburg. The aim of the discussion was to refine the provisional PRDN instrument in order to improve its content validity. During the discussion, the experts were asked to grade each item according to the following Likert scale:

1. Not important

2. Of little importance

3. Important but needs alteration

4. Important and succinct.

A score of 3 or 4 meant that the item was important and should remain on the form, and a score of 1 or 2 meant that it should be removed. Each item on the provisional PRDN instrument was debated until complete consensus was reached. The experts were also encouraged to make suggestions for the inclusion of additional items. After the peer group discussion, the provisional PRDN instrument was refined to produce a revised PRDN instrument to be used in the judgement-quantification stage. The revised instrument was emailed to the experts to confirm that the discussed changes had been made.

Judgement-quantification stage

The judgement-quantification stage involved two stages: validation by rating each item in the revised RPDN instrument, and calculating the content validity index (CVI) of each item and of the instrument as a whole.

The number of experts needed in this stage follows the same principles as in the development stage.^[7] When 6 or more participants are used, one expert can be in disagreement with the others and the instrument will still be assessed as content valid; however, if there are 5 or fewer participants, all the participants must agree. In this study, 10 experts were invited to participate, of whom 8 responded. During this stage the revised PRDN instrument, along with an information sheet describing the study and giving instructions on how to rate the items, was emailed to each participating expert around SA. None of the experts who participated in the judgement-quantification stage was involved in the previous development stage.

The content validity of each item in the revised RPDN instrument was independently rated by each of the 8 experts according to the same Likert scale used in the development stage, as described by Lynn.^[7] The CVI for each item is the proportion of experts who rated that item as content valid by giving it a score of 3 or 4 on the Likert scale. The proportion of experts who need to agree on each item in order for it to be real agreement v. chance was determined by setting the standard error of the proportion as 7 out of 8 experts agreeing (0.88). If fewer than 7 out of 8 of the experts scored the item with a 3 or 4, the item was removed from the form. The CVI was then applied to the entire form by determining the proportion of total items judged as being content valid.^[7,8]

 

Results

Development stage

The demographics of the 6 local experts present at the development stage peer group discussion and the 8 national experts who responded to the email in the judgement-quantification stage are shown in Table 1.

A provisional PRDN instrument was developed during the development stage through a thorough literature review of 11 journal articles^[1,3,4,9-16] related to anaesthesia-related mortality and morbidity. The provisional PRDN instrument consisted of 14 domains, namely demographics, hospital, anaesthetist, preoperative assessment, anaesthetic technique, induction, airway (if applicable), ventilation, monitoring and equipment, drugs, fluids, preoperative care, cardiac arrest (if applicable), and cause of death. The 14 domains contained 66 items in total. The provisional PRDN instrument is available as a supplementary file (http://www.samj.org.za/public/sup/14083-1.doc).

This provisional instrument was debated at the peer group discussion and a revised PRDN instrument was developed. A total of 13 items were added, 11 items were changed or revised, and 8 items were omitted and replaced with 2 questions that were more comprehensive, all to provide more clarity. The revised PRDN instrument contained 13 domains with 65 items in total. The revised PRDN instrument is available as a supplementary file (http://www.samj.org.za/public/sup/14083-2.doc).

Judgement-quantification stage

There were only 3 items that the 8 experts did not agree on, and these items were removed from the form. The CVI for the revised PRDN instrument as a whole was 0.96, making it content valid. Although the revised PRDN instrument achieved an adequate score to enable it to stand alone as a content-valid instrument, it was decided to improve this form even further. The national experts made several useful comments and suggestions that were taken into consideration and incorporated into the final PRDN instrument. Alterations included 10 additions and 6 modifications, and the 3 items with a CVI <0.88 were removed. The final PRDN instrument, which consisted of 18 domains and 79 items, was then re-rated by the national experts and achieved a CVI of 1 (Table 2).

 

Discussion

A death due to anaesthesia is a tragic event for the patient, their family, and the medical practitioners involved. Although the anaesthesia-related mortality rate in SA has decreased, there is still a lot that can be done to make anaesthesia safer. To identify which areas of practice need improving, and where resources need to be allocated, accurate audits of mortality and morbidity need to be carried out. To do this adequately, proper documentation describing the perioperative events is required. One such document is the anaesthetic chart, and another is the PRDN instrument that is completed in the event of a death.

A comprehensive PRDN instrument is also needed in medicolegal situations. Forensic pathologists need the information in order to establish the cause of death and to decide whether an inquest should be held. Medical practitioners use this form when defending themselves in the case of an inquest or formal prosecution. There are several differences between the anaesthetic sections of the current GW7/24 form and the final PRDN instrument developed by the present study.

Under the domain 'Hospital', an option was added to specify the level of hospital as primary healthcare facility, district, regional, tertiary or other. This information is important in SA, as there is a difference in anaesthesia-related mortality rates between different hospital levels. There is a need to collect accurate, region-specific anaesthetic information in order to identify where resources need to be allocated and how to strengthen anaesthetic services at primary, district and regional levels.^[17]

The domain 'Anaesthetist' included a question asking for more detail on the experience level of the anaesthetist and whether there was assistance available to them. Community service officers in district and regional centres in SA, with little or no anaesthetic experience, are often expected to anaesthetise patients alone. Studies in both Australia and Scotland have shown that there is a link between inexperienced anaesthetists and anaesthesia-related mortality.^[18-20] Illustrating this in SA would lead to better training and support of doctors working in rural areas, potentially improving care to rural populations.

The domain 'Monitoring and equipment' was an addition to the final PRDN instrument. It was suggested at the peer group discussion that a tick-box format would suit this domain and would be an efficient way of collecting important information that is not included in the current GW7/24 form. It has been shown in the literature that there is an association between the use of pulse oximetry and end-tidal carbon dioxide monitoring and a reduction in respiratory-related damaging events.^[21] Although it is not easy to prove that specific monitors reduce mortality, it is important to observe their effect on patient outcome and other safety-related measures, especially as new monitoring techniques are continually being developed.^[22]

Part C of the GW7/24 form, completed by the surgeon, asks for details about conditions such as shock and haemorrhage - how they affected the course of the procedure and how they were managed. It was considered by both local and national experts that the fluid and drug management of shock and haemorrhage during a procedure is usually the responsibility of the anaesthetist and should therefore be included in the anaesthetic part of a PRDN instrument. The domains 'IVI access' and 'Fluids' were added and ask for specific details on fluid and blood management. Incorrect blood management has been particularly highlighted in the international literature as a cause of anaesthesia-related morbidity and mortality, with delayed blood transfusion causing hypotension, hypovolaemic shock and postoperative myocardial infarction in high-risk patients.^[1,13]

Adequate monitoring and facilities during the postoperative period, including continued monitoring in the intensive care unit (ICU) for high-risk patients, have been shown to reduce perioperative mortality rates.^[1,15,23] The Royal Australasian College of Surgeons Anaesthetic Case Form^[24] includes detailed questions about recovery room management and a section on the use of intensive care and high-care unit resources. The GW7/24 form does not include any questions about the postoperative period, so it was decided that a postoperative care domain should be added to the final PRDN instrument, and that it should include a question about ICUs. In SA, ICU beds are scarce resources, resulting in many eligible patients being denied ICU services. The information obtained from question 4.2 in the final PRDN instrument, 'Was ICU requested? If denied, please state reasons why', may help to motivate for more ICU beds in the future.

The objective of this study was to develop and validate a revised anaesthetic section of the SA PRDN instrument. The final PRDN instrument produced in this study should be submitted to the South African Society of Anaesthesiologists (SASA) as a template for procedure-related mortality documentation. In collaboration with academic anaesthetic centres in SA, SASA can use this instrument as a society-endorsed document when lobbying the National Department of Health to update the current GW7/24 form.

Study limitations

This study is limited by the fact that the final PRDN instrument was not evaluated in a real-world setting. The new instrument should be piloted in a hospital or region to assess efficacy, usability and relevance. For example, a qualitative study could be performed to assess both anaesthetist and forensic pathology satisfaction with the instrument over a defined period of time.

In addition, only Part D of the GW7/24, relating to the anaesthesia, was revised in this study. Parts A, B and C of the form should also be updated in collaboration with surgical and forensic pathology colleagues.

 

Conclusions

The current SA PRDN form, the GW7/24, is outdated and inadequate, and needs to be revised. The final PRDN instrument developed in this study is comprehensive and validated and will adequately capture all the information required for epidemiological, forensic and academic purposes. This new instrument is a suitable replacement for Part D of the current GW7/24 form.

Declaration. None.

Acknowledgements. None.

Author contributions. FKO: design of work, data collection and analysis, drafting and revision of article. CK: revision and editing of final article. AA: conception of work, revision, editing and approval of article.

Funding. None.

Conflicts of interest. None.

 

References

1. Braz LG, Braz DG, da Cruz DS, Fernandes LA, Modolo NSP, Braz JRC. Mortality in anesthesia: A systematic review. Clinics (Sao Paulo) 2009;64(10):999-1006. https://doi.org/10.1590/S1807-59322009001000011        [ Links ]

2. Lienhart A, Auroy Y, Pequignot F, et al. Survey of anaesthesia-related mortality in France. Anesthesiology 2006;105(6):1087-1097. https://doi.org/10.1097/00000542-200612000-00008        [ Links ]

3. Gottschalk A, van Aken H, Zenz M, Standl T. Is anesthesia dangerous? Dtsch Arztebl Int 2011;108(27):469-474. https://doi.org/10.3238/arztebl.2011.0469        [ Links ]

4. Li G, Warner M, Lang B, Huang L, Sun L. Epidemiology of anesthesia-related mortality in the United States, 1999 - 2005. Anesthesiology 2009;110(4):759-765. https://doi.org/10.1097/ALN.0b013e31819b5bdc        [ Links ]

5. Lee L, Domino K. The Closed Claims Project: Has it influenced anesthetic practice and outcome? Anesthesiology Clin North Am 200230(3):485-501. https://doi.org/10.1016/S0889-8537(02)00006-8        [ Links ]

6. Vilas-Boas V, Mingotte P, Freitas M. Telephone call for post-discharge surveillance: Validation and application of tool for video-assisted surgery. Rev Bras Enferm 2015;68(5):899-905. https://doi.org/10.1590/0034-7167.2015680519i        [ Links ]

7. Lynn M. Determination and quantification of content validity. Nurs Res 1986;35(6):382-386. https://doi.org/10.1097/00006199-198611000-00017        [ Links ]

8. Grant S, Davis L. Selection and use of content experts for instrument development. Res Nursing Health 1997;20(3):269-274. https://doi.org/10.1002/(sici)1098-240x(199706)20:3<269::aid-nur9>3.0.co;2-g        [ Links ]

9. Gibbs N. National anaesthesia mortality reporting in Australia from 1985 - 2008. Anaesth Intensive Care 2013;41(3):294-301. https://doi.org/10.1177/0310057X1304100304        [ Links ]

10. Lunn J, Hunter A, Scott D. Anaesthesia-related surgical mortality. Anaesthesia 1983;38(11):1090-1096. https://doi.org/10.1111/j.1365-2044.1983.tb12486.x        [ Links ]

11. Clinical Excellence Commission, Australia. Activities of the Special Committee Investigating Deaths Under Anaesthesia, 2014. http://www.cec.health.nsw.gov.au/incident-management/mortality-review-committees/scidua/resource-list (accessed 26 October 2016).         [ Links ]

12. Special Committee Investigating Deaths Under Anaesthesia, Australia. Case Record of Death in Association with Sedation and/or Anaesthesia. http://www.cec.health.nsw.gov.au/_data/assets/pdf_file/0004/259357/scidua-questionnaire-template.pdf (accessed 30 May 2016).         [ Links ]

13. Lienhart A, Auroy Y, Péquignot F, et al. Survey of anesthesia-related mortality in France. Anesthesiology 2006;105(6):1087-1097. https://doi.org/10.1097/00000542-200612000-00008        [ Links ]

14. Harrison GG. Death attributable to anaesthesia, A 10-year survey (1967 - 1976). Br J Anaesth 1978; 50:1041-1046. https://doi.org/10.1093/bja/50.10.1041        [ Links ]

15. Arbous MS, Grobbee DE, van Kleef JW, et al. Mortality associated with anaesthesia: A qualitative analysis to identify risk factors. Anaesthesia 2001;56(12):1141-1153. https://doi.org/10.1046/j.1365-2044.2001.02051.x        [ Links ]

16. Tiret L, Hatton F, Desmonts JM, Vourc'h G. The implications of a national study of risk of anaesthesia. Health Policy 1988;9(3):331-336. https://doi.org/10.1016/0168-8510(88)90022-X        [ Links ]

17. Patel N, Peffer M, Leusink A, Singh N, Smith M. Surgery and anaesthesia in the South African context: Looking forward. S Afr Med J 2016;106(2):135-136. https://doi.org/10.7196/SAMJ.2016.v106i2.10529        [ Links ]

18. Lundgren A. Peri-operative deaths in two major academic hospitals in Johannesburg, South Africa. PhD dissertation. Johannesburg: University of the Witwatersrand, 2011. http://wiredspace.wits.ac.za/xmlui/bitstream/handle/10539/11663/ACL%20FINAL%20COMPLETED%20THESIS.pdf?sequence=1&isAllowed=y (accessed 21 June 2017).         [ Links ]

19. McFarlane HJ, van der Horst N, Kerr L, McPhillips G, Burton H. The Scottish Audit of Surgical Mortality: A review of areas of concern related to anaesthesia over 10 years. Anaesthesia 2009;64(12):1324-1331. https://doi.org/10.1111/j.1365-2044.2009.06125.x        [ Links ]

20. Warden J, Horan B. Deaths attributed to anaesthesia. Anaesth Intensive Care 1996;24(1):66-73. https://doi.org/10.1177/0310057X9602400112        [ Links ]

21. Department of Anesthesiology and Pain Medicine, Seattle. Closed Claims Project and its registries, 2016. http://depts.washington.edu/asaccp/projects/anesthesia-closed-claims-project (accessed 13 September 2016).         [ Links ]

22. Chilkoti G, Wadhwa R, Saxena A. Technological advances in perioperative monitoring: Current concepts and clinical perspectives. J Anaesthesiol Clin Pharmacol 2015;31(1):14-24. https://doi.org/10.4103/0970-9185.150521        [ Links ]

23. National Confidential Enquiry into Patient Outcome and Death, London, 2002. http://www.ncepod.org.uk/ncepod.html (accessed 26 October 2016).         [ Links ]

24. Royal Australasian College of Surgeons. Northern Territory Audit of Surgical Mortality. https://www.surgeons.org/research-audit/surgical-mortality-audits/regional-audits/ntasm (accessed 6 January 2020).         [ Links ]

 

 

Correspondence:
F K Orr
franceskateorr@gmail.com

Accepted 2 July 2019