RESEARCH

 

Evaluation of the diagnostic accuracy of the HemoCue device for detecting anaemia in healthy school-aged children in KwaZulu-Natal, South Africa

 

 

T P Gwetu^I; M K Chhagan^II

^IMB ChB, MPH; Department of Public Health, School of Clinical Medicine, College of Health Sciences, Nelson R Mandela School of Medicine, University of KwaZulu-Natal, Durban, South Africa
^IIFCPaed, PhD; Department of Public Health, School of Clinical Medicine, College of Health Sciences, Nelson R Mandela School of Medicine, University of KwaZulu-Natal, Durban, South Africa

Correspondence

 

 

---

ABSTRACT

BACKGROUND: The prevalence of anaemia in school-aged children is reported to be high (>10%), yet neither the onset of anaemia nor the disease causing it is easily established. Any form of anaemia, even if mild, can compromise children's health and survival. This study was conducted to generate data to support or reject use of the HemoCue device as a potential point-of-care method for haemoglobin (Hb) assessment in field and primary healthcare settings.
OBJECTIVE: To assess the validity of the HemoCue in relation to the gold-standard laboratory method.
METHODS: A cross-sectional study of children aged 6 - 8 years, analysing the diagnostic accuracy of the HemoCue in determining Hb levels in venous blood. Agreement between the HemoCue and laboratory techniques was evaluated using the Bland-Altman plot. The intra-class correlation coefficient was used to assess within-subject variability of measured Hb.
RESULTS: A trend of underestimation of Hb values was noted. The mean Hb with the HemoCue was 11.70 g/dL and that with the laboratory method 12.19 g/dL. The mean difference between the two methods was 0.49 g/dL, with a standard deviation of 0.77 g/dL (95% confidence interval -0.59 - -0.38). Discrepancies >1 g/dL were identified in 14.1% of cases. Bias increased with increasing Hb values.
CONCLUSION: The HemoCue was found to be comparable to the standard laboratory method for determining Hb concentrations in school-aged children. Its usefulness for screening healthy children was demonstrated, although a full blood count is recommended if anaemia or iron deficiency is suspected.

---

 

 

Anaemia frequently occurs in association with an underlying condition and is linked to poor clinical outcomes. In school-aged children in sub-Saharan Africa, the prevalence of anaemia is reported to be high (affecting 25.4%).^[1] There is, however, a notable paucity of data relating to haemoglobin (Hb) levels, onset and duration of anaemia, and the underlying illness causing the anaemia in school-aged children. This is a significant deficiency, as any anaemia, even if it is mild, can compromise growth and development, irrespective of the underlying condition causing it. The Hb level is endorsed by the World Health Organization (WHO) as the universal indicator of anaemia at individual and population levels.^[2] However, most reliable ways of measuring it require some automated equipment that is not readily available in primary healthcare centres or resource-poor settings.

An accurate diagnostic test is essential for identifying suboptimal Hb levels and monitoring the response to interventions. The HemoCue method has some key advantages over the routinely used laboratory method. The HemoCue device measures Hb in a small sample of undiluted capillary or venous blood, is portable and user friendly, does not require access to refrigeration or electricity, gives immediate digitally displayed results, and is relatively inexpensive. This study assessed the HemoCue as an alternative point-of-care (POC) test for diagnosing anaemia and as a potential vehicle to improve coverage for anaemia screening programmes. We also determined a practical initial diagnostic approach that could assist in selecting an appropriate management strategy.

 

Methods

Design and participants

We carried out a cross-sectional survey in a rural community living in an isolated mountainous part of western KwaZulu-Natal Province (KZN), South Africa. This study was an ancillary investigation in a larger longitudinal study measuring various health indicators and psychosocial needs of preschool and school-aged children. Data collection was integrated into the usual services at the research site as far as possible. One hundred and eighty-four consecutive children attending follow-up over a 4-month period were invited to take part in the current study.

Diagnostic testing

Hb values of children in this study were determined using two methods: the conventional laboratory method and the HemoCue device. Samples for the laboratory method and the HemoCue were collected simultaneously from the cephalic vein of seated participating children using a Vacutainer needle. Hb determination with the HemoCue was conducted at the research site by the principal investigator (TPG), who had received training on the standardised use of a portable HemoCue Hb-201+ system (Angelholm, Sweden). The child's venous blood in the Vacutainer was used to fill a HemoCue cuvette and placed on the HemoCue. The result was read within 10 minutes and recorded to two decimal places. The device was calibrated each morning using the standard guideline provided. For every tenth participant, two HemoCue cuvettes were filled and the Hb values of the samples compared for quality control purposes. The HemoCue device readings were interpreted without knowledge of the results of the reference standard or clinical examination. Corresponding blood samples were sent for analysis according to standard protocol to a local accredited laboratory that measured Hb values using an automated analyser, the Siemens Advia 2120 machine. Anaemia cut-off values used were based on WHO recommen-dations.^[2] For the outcome measure of anaemia, Hb <11.5 g/dL was used. Anaemia was classified as mild (Hb 11.0 - 11.4 g/dL), moderate (8.0 - 10.9 g/dL) or severe (<8.0 g/dL).

Data management and analysis

Data were entered daily into a predesigned electronic database using the SPSS Statistics for Windows package, version 22.0 (IBM, USA), and cleaned regularly. SPSS was used to perform the statistical analysis. The distribution of Hb values was assessed using descriptive statistics and graphical plots. The paired Student's f-test was used to compare mean Hb differences between the HemoCue device and the laboratory method. The intra-class correlation coefficient (ICC) and Pearson's correlation coefficient were computed to assess the agreement and correlation between the Hb values provided by the two instruments. The Bland-Altman limits of agreement method was also applied to the numerical data to assess agreement between the two methods.^[3] The Bland-Altman analysis was demonstrated in MS Excel. The sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) with 95% confidence intervals (CIs) were used to determine diagnostic accuracy of the categorical data. Clinical intervention for anaemia is generally provided for cases with moderate and severe anaemia. Hence the assessment of HemoCue performance takes into consideration its ability to detect clinically significant anaemia with Hb values that are <10.9 g/dL according to the laboratory method.

Ethical considerations

Ethical approval for this study was obtained from the official review board of the University of KwaZulu-Natal Biomedical Research Ethics Committee. Informed consent was obtained from each child's parent or caregiver before inclusion in the study. Children with anaemia were referred to their local health centre for further management.

 

Results

One hundred and eighty-four children participated in the study. The participants' ages ranged from 72 to 102 months (mean 83.7 (standard deviation (SD) 7.3)). There were more boys (108/184, 58.7%) than girls (76/184, 41.3%). The children all had symptoms of anaemia, and none were reported to be taking iron supplements. The prevalence of anaemia using the laboratory method was 43/184 (23.4%) (mean Hb 12.17 g/dL, range 7.80 - 19.20; 95% CI 10.06 - 14.52). Anaemia was predominantly mild (24/43, 55.8%) or moderate (18/43, 41.9%). Severe anaemia was rare (1/43, 2.3%). According to the HemoCue device, the mean Hb was 11.70 g/dL (range 6.60 - 15.50; 95% CI 9.45 - 14.13) and the prevalence of anaemia was 41.3% (76/184); 1/76 children (1.3%) had severe, 45/76 (59.2%) moderate and 30/76 (39.5%) mild anaemia.

Accuracy

The HemoCue device was generally precise and showed good correlation with the reference laboratory method (Figs 1 and 2); 158/184 (85.9%) of Hb values measured with the HemoCue were within 1 g/dL of those measured with the laboratory reference method, and 182/184 (98.9%) were within 2 g/dL.

 

 

The sensitivity for anaemia of the HemoCue device was 93%, meaning that only 7% of children with anaemia were missed by the device. The NPV was 97%, indicating that children with an optimal HemoCue reading were probably not anaemic. Specificity was good (94%), but the PPV was low (52.6%), indicating that among the children with suboptimal HemoCue readings only 52.6% actually had the condition (Table 1).

 

 

Precision

The HemoCue device demonstrated good reproducibility. Generally no differences were observed between repeat procedures on the same sample of blood at the same time. Sequential samples in the same patient did not differ significantly. The mean difference for repeat HemoCue readings using the Bland-Altman method was 0.1 g/dL (95% CI 0.1 -0.2). The 95% limits of agreement were -1.4 -1.5 g/dL. Instrument error was therefore ruled out as a substantial contributor of error due to unreliability.

The Bland-Altman mean difference for Hb values between the HemoCue device and the laboratory method was 0.47 g/dL (95% CI 0.36 - 0.59), with 95% limits of agreement ranging from -1.07 to 2.02 (Table 2). The SD (precision 0.77) and Pearson correlation coefficient (r=0.79) were high and significant (p=0.000). The ICC was 0.88 (95% CI 0.84 - 0.91). This was statistically significant (p=0.00). The ICC is calculated on a range of 0 - 1, with 1 representing complete reliability without any measurement errors and 0 reflecting unreliability.

 

 

Variation was noted in mean Hb values depending on anaemia status. The mean Hb for anaemic participants was 0.32 g/dL (95% CI -0.51 - -0.14) with a Pearson correlation coefficient of r=0.81 and SD 0.60, and statistically significant (p=0.001). The standard error (SE) was low (0.092). The mean difference in Hb values among the non-anaemic participants was 0.52 g/dL (95% CI -0.66 - -0.39). The correlation among the non-anaemic participants was lower than among those who were anaemic (r=0.64, SD 0.81 and SE 0.068; p=0.000).

 

Discussion

This study showed that the HemoCue device was less accurate than the laboratory method and that its variability was significant. The hypothesis that the device is equivalent to the laboratory method has to be rejected. Nevertheless, it must be kept in mind that automated laboratory Hb analysis and the HemoCue are not competing techniques but complementary methods for Hb assessment. Laboratory analysis represents the gold-standard reference method for Hb assessment. An acceptable mean difference between the HemoCue and the laboratory method would be<1 g/dL;^[4] our results therefore show a significant difference (14.1%; p<0.05) between the two methods when monitoring Hb values. The relevance of this limit should be interpreted with caution, however, as a 1 g/ dL error will have different consequences for interpretation of Hb values depending on the circumstances. In evaluating the results of this study, one also needs to keep in mind the usefulness of the HemoCue as a POC anaemia screening tool that is portable, displays Hb results rapidly within 1 minute, and uses a tiny blood sample.

The HemoCue tended to underestimate Hb values compared with the laboratory method, which may result in overestimation of anaemia. Although the device gave lower Hb values, these measurements varied widely. Discrepancies in Hb values of >1.0 g/dL were identified in 14.1% of participants and differences of >2 g/dL in 1.1%. Outliers of >2 g/dL are significant because they could result in altered clinical management, with children being overtreated or overinvestigated. Although such large differences in Hb values were infrequent, we recommend that clinical decisions should not be based solely on HemoCue results.

While the differences noted could not definitely be attributed to any individual factor, they may be due to factors such as biological variation, collection mistakes during sampling, environmental factors or device error. Previous studies have generally reported a difference between the means, although most did not reach statistical significance. The range in the mean difference of Hb values in this study was 0.1 - 0.5 g/dL (SD 0.37 - 1.1). A trend towards underestimation of Hb values has frequently been reported,^[5-7] although some studies found no difference,^[8] some overestimation,^[9] and another both under- and overestimation, Neufield et al.^[10]reporting underestimation of Hb values with capillary samples for concentrations <12.5 g/dL and overestimation at higher values.

The HemoCue device showed good sensitivity in detecting anaemia in the population studied (Table 2). Children with optimal Hb concentrations (>11.5 g/dL, the anaemia cut-off) are more likely to be misclassified than those with low values. The accuracy of the HemoCue was not affected by the presence of anaemia, although bias increased in non-anaemic participants with increasing Hb values. The mean difference in Hb values among non-anaemic children was 0.52 g/dL and that among anaemic children 0.32 g/dL. The sensitivity of the HemoCue for detecting anaemia was 93%, and the NPV was 97.22%. These values are near the top of the range reported in earlier studies, where sensitivity ranged from 75% to 91%, specificity from 88% to 100%, and PPVs from 75% to 86%.^[11] Differences in anaemia detection and estimates of severity could influence statistical estimates in screening initiatives and surveys. When screening for anaemia, a high sensitivity as opposed to a high specificity is desirable, because poor outcomes are likely with untreated chronic anaemia. Although the HemoCue did not match the laboratory method, it can be expected to be useful as a screening tool and a reliable predictor of the need for a laboratory full blood count (FBC) in resource-poor populations. An FBC may be essential when anaemia is identified or iron deficiency is suspected in the presence of a normal HemoCue Hb reading. It is unlikely that clinically important anaemia will go unnoticed, either by the clinician or by the HemoCue. The HemoCue would therefore seem to be a useful screening method for anaemia in appropriate situations such as in primary healthcare, routine surveillance, field survey operations, or evaluation of interventions for iron nutrition.

The HemoCue device was not expected to match the laboratory gold-standard method, but simply to provide an alternative technique in prescribed circumstances. The Clinical Laboratories Improvement Act of 1988 recommends that the estimated percentage difference between a number of devices measuring Hb levels in a laboratory setting should be about 7% of the actual value.^[12] We found a very high coefficient of variation (1.63) for the HemoCue, which was higher than that documented by other researchers investigating the performance of the device under fieldwork conditions.^[11] The mean difference for Hb values was 0.47 g/dL and the 95% limits of agreement between the HemoCue and laboratory Hb values were -1.07 - 2.02 g/dL, indicating that the two measurement methods agreed closely in estimating small values but disagreed with larger values (Fig. 2). This finding suggests that for any child in this study population, the Hb assessments could have differed between -1.07 and 2.02 g/dL. This is too wide-ranging to be clinically acceptable -it would mean, for example, that a child with an Hb value of 10 g/dL according to the HemoCue would be expected to have a laboratory measurement between 8.93 and 12.02 g/dL. Both the ICC and Pearson's correlation coefficient were determined, because it is possible to have a high degree of correlation while agreement is poor. We demonstrated a significant correlation (r=0.85; p=0.001) and a significant bias of 0.47 g/dL (SD 0.77) between the HemoCue and the laboratory method in a field setting (Table 2). Accordingly, data obtained from the field can be assumed to be of the same level of reliability as data obtained in laboratory settings. The diagnostic precision of the HemoCue in laboratory settings is widely acknowledged,^[11] but its performance outside the laboratory setting has been questioned. Morris et al.^[13] suggested the use of repeat sampling to minimise the effect of unreliability. However, this is unlikely to be practical during fieldwork and in settings where large numbers are surveyed.

 

 

Study limitations

This study analysed blood samples obtained from healthy children, so the results are limited to the screening of well children and should not be translated into a setting where acutely ill children need a reliable result and speedy intervention. The HemoCue test is widely conducted with a finger-prick capillary sample. In children, Hb levels between venous and capillary samples have been reported to vary, with the largest discrepancies being in the neonatal period and in acutely ill children. The largest difference between the two sample types has been observed in anaemic children.^[10,11]

Recommendations

Studies have shown that iron deficiency anaemia can result in impaired psychomotor and behavioural development, which may not be reversible.^[14] National screening of all children for anaemia during infancy and later when they start school may be a logical and productive approach to anaemia control. Screening of children around the stage of school entry gives an adequate time interval for anaemia of chronic disease to become apparent. The feasibility of routine large-scale screening of children for anaemia has not been investigated. Further studies are required to address this significant challenge. Until such screening investigations are implemented, the National Department of Health needs to highlight and endorse the appropriate execution of the Integrated Management of Childhood Illness programme for identification and management of anaemia in children. Our study suggests that the HemoCue test is comparable to the reference laboratory method as a practical technique for detecting anaemia in school-aged children. For the device to be used widely and successfully, the interpretation of Hb values may need to be adjusted. The definition of anaemia in children may be improved by basing it on an Hb range linked to negative potential health outcomes. This approach to defining anaemia may, however, result in larger numbers of children being categorised as anaemic. Further research is required before recommending this diagnostic approach, with studies that regulate confounding variables such as comorbidities.

 

References

1. De Benoist B, McLean E, Egli I, Cogswell M. Worldwide Prevalence of Anaemia 1993-2005: WHO Global Database on Anaemia. Geneva: World Health Organization, 2008.         [ Links ]

2. World Health Organization. Haemoglobin Concentrations for the Diagnosis of Anaemia and Assessment of Severity. Vitamin and Mineral Nutrition Information System. Geneva: WHO, 2011. http://www.who.int/vmnis/indicators/haemoglobin.pdf (accessed 20 March 2015).         [ Links ]

3. Altman DG. Practical Statistics for Medical Research. London: Chapman & Hall, 1991.         [ Links ]

4. Radtke H, Polat G, Kalus U, Salama A, Kiesewetter H. Haemoglobin screening in prospective blood donors: Comparison of different blood samples and different quantitative methods. Transfus Apher Sci 2005;33(1):31-35. [http://dx.doi.org/10.1016/j.transci.2004.11.004]        [ Links ]

5. Lara MAM, Kandulu J, Chisuwo L, Bates I. Evaluation and costs of different haemoglobin methods for use in district hospitals in Malawi. J Clin Pathol 2005;58:56-60. [http://dx.doi.org/10.1136/jcp.2004.018366]        [ Links ]

6. Rechner IJ, Twigg A, Davies AF, Imong S. Evaluation ofthe HemoCue compared with the Coulter STKS for measurement of neonatal haemoglobin. Arch Dis Child Fetal Neonatal Ed 2002;86(3):F188-F189. [http://dx.doi.org/10.1136/fn.86.3.F188]        [ Links ]

7. Cohen AR, Seidl-Friedman J. HemoCue system for haemoglobin measurement: Evaluation in anaemic and non-anaemic children. Am J Clin Pathol 1988;90(3):302-305.         [ Links ]

8. Nkrumah B, Nguah SB, Sarpong N, et al. Haemoglobin estimation by the HemoCue portable haemoglobin photometer in a resource poor setting. BMC Clin Pathol 2011;11:5. [http://dx.doi.org/10.1186/1472-6890-11-5]        [ Links ]

9. Spielmann N, Mauch J, Madjdpour C, Schmugge M, Weiss M, Haas T. Accuracy and precision of haemoglobin point-of-care testing during major paediatric surgery. International Journal of Laboratory Haematology 2012;34(1):86-90. [http://dx.doi.org/10.1111/j.1751-553X.2011.01363.x]        [ Links ]

10. Neufeld L, García-Guerra A, Sánchez-Francia D, Newton-Sánchez O, Ramírez-Villalobos MD, Rivera-Dommarco J. Haemoglobin measured by HemoCue and a reference method in venous and capillary blood: A validation study. Salud Publica Mex 2002;44(3):219-227. [http://dx.doi.org/10.1590/S0036-36342002000300005]        [ Links ]

11. Gwetu TP, Chhagan MK, Craib M, Kauchali S. HemoCue validation for the diagnosis of anaemia in children: A semi-systematic review. Pediat Therapeut 2013;4:187. [http://dx.doi.org/10.4172/2161-0665.1000187]        [ Links ]

12. Medicare, Medicaid and CLIA programs. Regulations implementing the Clinical Laboratory Improvement Amendments of1988 (CLIA)-HCFA. Final rule with comment period. Fed Regist 1992;57:7002-7186.         [ Links ]

13. Morris SS, Ruel MT, Cohen RJ, Dewey KG, de la Briere B, Hassan MN. Precision, accuracy and reliability of haemoglobin assessment with use of capillary blood. Am J Clin Nutr 1999;69(6):1243-1248.         [ Links ]

14. Lozoff B, Jimenez E, Hagen J, Mollen E, Wolf AW. Poorer behavioural and developmental outcome more than 10 years after treatment for iron deficiency in infancy. Paediatrics 2000;105(4):e51. http://www.pediatrics.org/cgi/content/full/105/4/e51 (accessed 10 March 2015).         [ Links ]

 

 

Correspondence:
T P Gwetu
tgwetu@gmail.co.za

Accepted 11 June 2015.