Services on Demand
Article
Indicators
Related links
- Cited by Google
- Similars in Google
Share
SA Orthopaedic Journal
On-line version ISSN 2309-8309
Print version ISSN 1681-150X
SA orthop. j. vol.13 n.4 Centurion Dec. 2014
Interrater reliability assessment of pre-reduction MRI features identifying hazardous disc disruption in distractive-flexion cervical spine injuries
Dr MA FlemingI; Dr T Westgarth-TaylorII; H CarraraIII; Dr S CandyIV; Professor R DunnV
IMBBCH, FcOrth (SA), MMED; Senior Orthopaedic Registrar, Groote Schuur Hospital, University of Cape Town
IISenior Radiology Registrar, Groote Schuur Hospital, University of Cape Town
IIIStatistician, University of Cape Town
IVBSc HED, MBChB, FCRad (Diag) SA; Head of Radiology Department, Groote Schuur Hospital, University of Cape Town
VProfessor of Orthopaedics, Groote Schuur Hospital, University of Cape Town
ABSTRACT
AIM: This study assessed the interrater reliability of MRI when radiologists and orthopaedic surgeons assess intervertebral disc injury characteristics in distraction flexion (DF) injuries in the cervical spine. The most reliable MRI features of intervertebral disc injury are identified for future use
METHODS: Pre-reduction MRI scans of 110 consecutive DF injuries were reviewed independently by a radiology and an orthopaedic surgical team. All cases were managed at a single tertiary referral unit over a ten-year period. Variables included for assessment were: disc herniation (posterior to the inferior vertebrae or above the level of the superior vertebrae's endplate), disc disruption, posterior longitudinal ligament disruption and disc containment. A double data entry method was used. Cohen's kappa value was used to determine interrater reliability
RESULTS: Perfect agreement was never achieved between the two teams. The variables that had the highest interrater agreement were posterior disc prolapse and impression of containment. When disagreement occurred, the radiology team would tend to define the lesions as more severe than the orthopaedic team
CONCLUSION: MRI assessment of disc injury characteristics carries moderate to fair interrater reliability at best. We conclude that the treating surgeon should review scan images personally prior to choosing a treatment algorithm, not relying solely on a written report. Posterior herniation and disc containment carry the highest interrater reliability
Key words: cervical disc prolapse, MRI interrater reliability, distraction flexion, unifacet, bifacet subluxation
Introduction
Eismont et al. identified the risk of secondary cord injury in distraction flexion (DF) injuries management in 1991.' Pre-reduction MRI has been proposed to identify disc injuries that may compress the cord following spinal reduction causing a secondary cord injury. The necessity for pre-reduction MRI has been disputed by some due to accessibility issues and resultant delays. Hart and Vaccaro2 debate the issue whether MRI will identify patients requiring open discectomy prior to reduction as opposed to the view that the clinical risk of awake closed reduction is too small to justify MRI-related delays to closed reduction and thus indirect decompression. In many countries, MRI access is limited due to distance and cost.
Although the use of sensitive MRI scanning may detect disc pathology, its clinical correlation has been called into question.3 Vaccaro et al. assessed the timing and influence of MRI on management of DF injuries.4 They found inconsistency between surgeons with regard to their use of MRI, as well as treatment modalities between the groups. Orthopaedic surgeons were found to be much more likely to perform closed reduction having reviewed an MRI.
In addition to inconsistent MRI use, there is frequently a difference of opinion between clinical and radiological staff as to the status of the disc, thus making clinical decisions difficult.
Aim
The aim of this study was to compare the interpretation of pre-reduction MRIs of DF injuries between radiologists and orthopaedic surgeons, and to determine consensus regarding the 'dangerous' disc. A 'dangerous' disc was defined as an uncontained disc, herniating posteriorly, that may be drawn into the spinal canal during closed reduction.
Although the use of sensitive MRI scanning may detect disc pathology, its clinical correlation has been called into question
Methods
Following approval by the Institutional Ethic committee (091/2011), 110 consecutive DF injury cases were identified over a ten-year period from the senior author's database.
MRI scans (DICOM data) were collated from the hospital archive. The scans included sagittal and axial images in T1, T2 weighted sequences and sagittal STIR sequences.
These scans were independently reviewed by a radiology and an orthopaedic team. The radiology team comprised an MRI-trained senior radiology registrar and head consultant of radiology and the other, a senior orthopaedic registrar and senior spinal orthopaedic surgeon.
A meeting was held prior to the start of the scan review, where consensus was achieved on the variables to assess and the definition thereof as below. These two teams were blinded to the clinical scenarios and worked independently.
The data obtained from each team was independently accrued and entered into Epidata statistical software. A double entry system was employed to reduce entry error.5 All data was categorical except disc heights.
Interrater reliability/agreement was determined using the Cohen's kappa value. This statistical measure is used when comparing two researchers' observations. It is a measure used to determine the agreement while excluding the predicted agreement due to chance.6 Values of 0.6 to 1 show almost perfect agreement. Values of 0.4-0.6 are considered moderate agreement; 0.2-0.4 describes fair agreement; less than 0.2 indicates poor agreement.
MRI variables assessed
1. The posterior longitudinal ligament (PLL) was assessed as disrupted, intact or unsure.
2. Disc heights were measured in millimetres at the midpoint on mid-sagittal view.
3. Disc disruption was measured by the presence of an intervertebral disc signal on T2-weighted images.
We then defined lines on the sagittal sequences to indicate disc herniation. These included:
4. Posterior vertebral body line - a vertical line extended cephalad from the posterior body of the vertebral body caudal to the injury (Figure 1)
5. The inferior vertebral body line - a horizontal line extended posteriorly from the inferior border of the vertebral body superior to the injured disc (Figure 2)
6. Corner-to-corner line - a line from the cephalad vertebrae's postero-inferior corner to the inferior vertebrae's postero-superior corner (Figure 3)
Disc containment. This was a subjective assessment as to whether after assessing all the MRI sequences, the reviewer thought the disc was contained by the annulus or not.
Results
Figure 4 summarises the degree of agreement for each variable. The interrater agreement was never excellent as determined by Cohen's kappa values. There was moderate agreement on the posterior line, inferior line and containment assessment, with fair agreement on the rest. The manner of disagreement was however consistent. Radiologists reported a more severe degree of injury and were more often unsure of the appearance of the disc than the orthopaedic team.
Table I highlights this pattern with reference to the PLL data. The large numbers in blue blocks represent the agreed assessment between the radiologists and orthopaedic surgeons, whereas the white blocks represent the disagreements. The rows represent the orthopaedic assessment and the columns the radiologists. Thus, in row one, the orthopaedic assessment was 33 'intact' cases. The radiologist agreed in only four cases, calling 20 of these disrupted and nine unsure.
The numbers indicating disagreement are larger to the right superior area of the matrix. This trend persisted with all variables tested. The weight of disagreement may be interpreted as radiologists assessing an injury to be more severe than the orthopaedic team when disagreement was present.
Discussion
The requirement of pre-reduction MRI in DF injuries has developed since early case reports of secondary spine injury inflicted during closed reduction by uncontained cervical discs. Literature highlights these isolated case reports as preventable by identifying uncontained/ prolapsed discs with MRI prior to reduction.
The reported incidence of potentially 'dangerous' disc lesions in DF injuries is high, ranging from 15-77%7-10 with a markedly increased incidence in locked bifacet injuries.9 In our study, the agreed incidence of uncontained disc lesions was 63%.
The senior author questions to what extent this expensive and time-consuming investigation alters our management acutely although accepts its use in well-resourced environments where immediate access is available. MRI requisition delays initiation of closed reduction within the confines of our state service and even after hours in the private sector. Delay in the cord-injured patient may preclude the only chance a patient may have to gain some recovery from severe spinal cord injuries.11
Until recently this rationale made good clinical sense and was supported in animal studies;1213 however, recent evidence indicates the clinical benefits from reduction within 24 hours.11 It is with this in mind that early reduction should be prioritised above the possible (but rare) risk of secondary cord injury during reduction and subsequent disc migration and cord compression, provided the patient is alert and able to communicate during closed reduction.
Although MRI scanning has been employed in this context for many years, there have been no clear guidelines to which disc injuries will cause secondary cord injury during closed reduction. Additionally, the low interrater agreement while reviewing these scans in this study indicates difficulties in relying on MRI reporting.
In the context of DF injuries, the high incidence of these MRI-identified lesions coupled with the very low incidence of secondary cord injury during closed reduction, suggests that to act on every MRI-identified disc prolapse would result in a cumulative clinically unacceptable time delay before reduction. Thus delay due to insistence on pre-reduction MRI or based on a potential disc at risk on MRI is likely to cause a greater cumulative neurologic deficit to the cohort from failed recovery than the potential of secondary cord injury due to the rare postreduction disc compression.
This forms the first part of a larger study that combines clinical data and decision-making to identify whether in fact the presence of these disc lesions altered our clinical approach.
In this study our most reliable interrater variable to ascertain the presence of a hazardous disc lesion was posterior protrusion of the disc and the clinical impression as to whether the disc was contained or not.
There is a possible selection bias in that our cohort had pre-reduction MRIs. There is an unproven chance that if every patient had an MRI there would be a higher incidence of serious pathology since there is a trend among our surgeons to reduce patients with a complete spinal cord lesion without pre-reduction MRI. This is based on the philosophy of potential gain with nothing to lose.2
The use of Cohen's kappa value has been charged with being a conservative statistical tool6 but it has become the standard in the orthopaedic literature when comparing two parties' agreement.
Our most reliable interrater variable was posterior protrusion of the disc and the clinical impression as to whether the disc was contained or not
Conclusion
Interrater agreement of MRI disc injury variables in DF injuries in this study was never excellent. The best agreement was found in the assessment of posterior disc prolapse and the researchers' overall impression of containment. Radiologists tended to report a more severe disc injury than orthopaedic surgeons.
For these reasons we advocate that the treating surgeon personally assess the MRI scan prior to clinical decisionmaking rather than relying on a report.
Having identified a small subset of patients where there was in fact consensus that the injured disc was potentially hazardous in terms of closed reduction, the authors have initiated a follow-up study correlating MRI data with clinical management and outcome.
References
1. Eismont FJ, Arena MJ, Green BA, D M, Green A. Extrusion of an intervertebral disc associated with traumatic subluxation or dislocation of cervical facets. Case report: Trauma of an intervertebral subluxation disc associated with cervical facets or dislocation of five. J Bone Joint Surg Am. 1991;73:1555-60. [ Links ]
2. Hart RA, Vaccaro AR, Nachwalter RS. Controversies in spine cervical facet dislocation: when is magnetic resonance imaging indicated? Spine (Phila Pa 1976). 2002;27(1):116-18. [ Links ]
3. Benzel EC, Hart BL, Ball P a, Baldwin NG, Orrison WW,Espinosa MC. Magnetic resonance imaging for the evaluation of patients with occult cervical spine injury. J Neurosurg [Internet]. 1996 Nov;85(5):824-29. Available from:http://www.ncbi.nlm.nih.gov/pubmed/8893720 [ Links ]
4. Grauer JN, Vaccaro AR, Lee JY, Nassr A, Dvorak MF, Harrop JS, et al. The timing and influence of MRI on the management of patients with cervical facet dislocations remains highly variable: a survey of members of the Spine Trauma Study Group. J Spinal Disord Tech. 2009 Apr;22(2):96-99. [ Links ]
5. Reynolds-Haertle RA, McBride R. Single vs. Double data entry in CAST. Control Clin Trials [Internet]. 1992 Dec;13(6):487-94. Available from: http://www.sciencedirect.com/science/article/pii/019724569290205E [ Links ]
6. Kilem Gwet, Gwet K. Inter-rater reliability: dependency on trait prevalence and marginal homogeneity. Stat Methods Inter-Rater Reliab Assess [Internet]. 2002 [cited 2013 Jun 30];2:1-10. Available from: http://en.wikipedia.org/wiki/Cohen's_kappa [ Links ]
7. Vaccaro AR, Falatyn SP, Flanders AE, Balderston RA, Northrup BE, Cotler JM. Magnetic resonance evaluation of the intervertebral disc, spinal ligaments, and spinal cord before and after closed traction reduction of cervical spine dislocations. Spine (PMa Pa 1976) 1999; 24(12):1210-17. [ Links ]
8. Berrington N, Staden J, Willers J, van der Westerhuizen J. Cervical intervertebral disc prolapse associated with traumatic facet dislocations. Surg Neurol. 1993;40(5):395-99. [ Links ]
9. Doran SE, Papadopoulos SM, Ducker TB, Lillehei KO. Magnetic resonance imaging documentation of coexistent traumatic locked facets of the cervical spine and disc herniation. J Neurosurg [Internet]. Journal of Neurosurgery Publishing Group; 1993 Sep 27 [cited 2013 Jul 2];79(3):341-45. Available from: http://thejns.org.ezproxy.uct.ac.za/action/showCitFormats?doi=10.3171/jns.1993.79.3.0341 [ Links ]
10. Mahale Y, Silver J, Henderson N. Neurological complications of the reduction of cervical spine dislocations. J Bone Joint Surg Br. 1993;75:403^09. [ Links ]
11. Fehlings MG, Vaccaro A, Wilson JR, Singh A, W Cadotte D, Harrop JS, et al. Early versus delayed decompression for traumatic cervical spinal cord injury: results of the Surgical Timing in Acute Spinal Cord Injury Study (STASCIS). PLoS One [Internet]. 2012 Jan [cited 2013 May 29];7(2):e32037. Available from: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=3285644&tool=pmcentrez&rendertype=abstract [ Links ]
12. Tarlov IM. Spinal cord compression studies III. Time limits for recovery after gradual compression in dogs. Neurol Psychiatry. 1954;7l(5):588-97. [ Links ]
13. Carlson GD, Gorden CD, Oliff HS, Pillai JJ, LaManna JC. Sustained spinal cord compression Part I: Time-Dependent Effect on Long-Term Pathophysiology. J Bone Jt Surg [Internet]. 2003;85(1):86-94. Available from: http://dx.doi.org/ [ Links ]
Correspondence:
Professor R Dunn
Groote Schuur Orthopaedic Department
Tel: +27(0)214045108
Fax: +27(0)214472709
Email: robert.dunn@uctac.za
This article is also available online on the SAOA website (www.saoa.org.za) and the SciELO website (www.scielo.org.za). Follow the directions on the Contents page of this journal to access it.