ORIGINAL ARTICLES

 

Field evaluation of a malaria rapid diagnostic test (ICT Pf)

 

 

Devanand Moonasar^I; Ameena E Goga^III; Philip S Kruger^V; Christine La Cock^VI; Rajendra Maharaj^IV; John Frean^VIII; Daniel Chandramohan^II

^IMSc (Clinical Parasitology), DrPH. London School of Hygiene and Tropical Medicine, Keppel Street, London, UK
^IIMBBS, MSc, PhD. London School of Hygiene and Tropical Medicine, Keppel Street, London, UK
^IIIMB ChB, DTMH, DCH, FCPaeds, MSc (MCH), MSc (Epidemiology). Medical Research Council of South Africa, Pretoria
^IVMSc (Entomology), PhD, MSc (Infectious Diseases). Medical Research Council of South Africa, Pretoria
^VDPH. Department of Health and Social Welfare, Limpopo Provincial Government, Polokwane
^VIDepartment of Health and Social Welfare, Limpopo Provincial Government, Polokwane
^VIIMB BCh, MMed, MSc, DTM&H, FACTM, FFTM RCPS (Glasgow). National Institute of Communicable Diseases Control, Johannesburg

 

 

---

ABSTRACT

BACKGROUND: Malaria rapid diagnostic tests (MRDTs) are quick and easy to perform and useful for diagnosing malaria in primary health care settings. In South Africa most malaria infections are due to Plasmodium falciparum, and HRPII-based MRDTs have been used since 2001. Previous studies in Africa showed variability in sensitivity and specificity of HRPIIbased MRDTs; hence, we conducted a field evaluation in Limpopo province to determine the accuracy of the MRDT currently used in public sector clinics and hospitals.
METHODS: A cross-sectional observational study was conducted to determine the sensitivity and specificity of an ICT Pf MRDT. We tested 405 patients with fever with ICT Pf MRDT and compared the results with blood film microscopy (the gold standard).
RESULTS: The overall sensitivity of the ICT Pf MRDT was 99.48% (95% confidence interval (CI) 96.17 - 100%), while specificity was 96.26% (95% CI 94.7 - 100%). The positive predictive value of the test was 98.48 (99% CI 98.41 - 100%), and the negative predictive value was 99.52% (95% CI 96.47 - 100%).
CONCLUSIONS: The ICT Pf MRDT is an appropriate test to use in the field in South Africa where laboratory facilities are not available. It has a high degree of sensitivity and acceptable level of specificity in accordance with the World Health Organization criteria. However, sensitivity of MRDT at low levels of parasitaemia (<100 parasites/µl of blood) in field conditions must still be established.

---

 

 

Malaria can be diagnosed using several approaches including clinical presumptive diagnosis, microscopy, polymerase chain reaction (PCR), enzyme-linked immunosorbent assay (ELISA) and malaria rapid diagnostic tests (MRDTs),¹ of which microscopy remains the 'gold standard'.² Since microscopy requires highly skilled staff, electricity, microscopes and laboratory reagents, its use is restricted to appropriately equipped and staffed laboratories. In district primary health care settings, RDTs are more appropriate for diagnosis because they are easy to use and inexpensive, as electricity or highly skilled staff are not needed.¹ Three key factors affect the accuracy of RDTs: manufacturing standards, end user proficiency in conducting test and interpreting the results, and the environment in which the tests are stored and transported (ideally below 30ºC).²

Three malaria parasite target antigens can be detected by MRDT technology: histidine-rich protein II (HRPII), which is found only in Plasmodium falciparum; and parasite lactate dehydrogenase (PLDH) and aldolase, present in all Plasmodium species.¹ The HRPII antigen detection test is suitable for South Africa because species other than P. falciparum are very rare.³

The World Health Organization (WHO) guidelines recommend that for an MRDT to be deemed 'accurate' it should have a sensitivity of >95% and a specificity of >90% at a level of >100 parasites per µl of blood, compared with microscopy;^2,4 and that microscopy is an acceptable gold standard provided that good techniques, good-quality reagents and - most importantly - well-trained and supervised microscopists are available.

Studies in Africa show variability in sensitivity and specificity of HRPII antigen detection tests.^2,5 Field evaluations of MRDTs in South Africa are sparse; an evaluation of the accuracy of the ICT Malaria Pf card test (ICT Diagnostics, Sydney, Australia) in Mpumalanga province showed a sensitivity and specificity of 98.6% and 97.9% respectively.⁶ Although the overall test accuracy was higher than the WHO acceptable threshold levels, the accuracy of this MRDT to detect parasitaemia of <100 parasites per µl of blood was not assessed.²

Owing to the variability in sensitivity and specificity of HRPII-based MRDTs in Africa, and the fact that the currently used ICT Pf MRDT (Global Diagnostics) was not robustly fieldtested in South Africa, we studied the field accuracy of this MRDT.

 

Methods

Our study was done during the high-transmission season (December 2006 - June 2007) in the Vhembe District, Limpopo Province, which borders southern Zimbabwe. This area was chosen because it had had the highest incidence of malaria in the province for the previous 9 years.

We estimated that 203 true malaria cases are needed to detect a 95% sensitivity with a 95% confidence limit of ±3%, assuming that 50% of suspected malaria cases are true malaria; we therefore enrolled 405 suspected malaria sufferers from 2 clinics for our cross-sectional study. The subjects were consecutively selected cases (male or female) of suspected malaria (defined as presentation with fever or headache or chills), attending study clinics for an initial visit and who consented to participate. Patients attending the study clinics for follow-up visits, severely ill patients needing referral, patients with an obvious non-malarial fever, and pregnant women were excluded.

Nurses were trained on slide preparation and RDT testing procedures (preparation and interpretation of results) before the study. Standard operating procedures for slide making and RDT testing were also posted in the study clinics. On repeat visits to the clinics, we reviewed the slide preparation and RDT testing.

A patient identification number (ID) was used to label the data collection forms, the MRDTs and the thin and thick blood smears. Nurses performed an MRDT and made thick and thin blood films from patients after obtaining written informed consent. Blood films were sent to two specialised trained malaria microscopists at the Limpopo Department of Health (Thulamela Health Center, Vhembe District) for staining and microscopy using standard techniques.⁷

The microscopists were blinded to the MRDT results and read the thin and thick blood films independently. When there was discordance between microscopists' and MRDT results, a medical technologist based at Limpopo Department of Health Malaria reference centre who was highly skilled in malaria microscopy and was blinded to the MRDT and field microscopists' readings read the slides, and this result was taken to be correct.⁵ Cohen's Kappa statistic was used to determine microscopist reliability; a score of >0.8 was considered reliable.

The primary measures to assess the accuracy of MRDTs were sensitivity, specificity and positive and negative predictive values, using microscopy as the gold standard. Data were managed using EPI Data version 3.1 and analysed using STATA 8.1. Mean and standard deviations, or median and inter-quartile ranges, were used to describe continuous variables. The continuous variables of parasitaemia, age and temperature were transformed into categorical variables e.g. parasitaemia was grouped into three categories (<500 parasites/µl, 501 - 5 000 parasites/µl or >5 000 parasites/µl). Sensitivity and specificity were determined for detecting any level of parasitaemia and for each parasitaemia category (<500, 501 - 5 000, >5 000).

Ethical approval was obtained from the London School of Hygiene and Tropical Medicine (reference 5061, 22 November 2006), the Limpopo Health and Social Welfare Development Research Committee (reference 4/2/2) and the University of Limpopo Research Ethics and Publications Committee (MR 123/2006).

 

Results

The median subject age was 24.5 years (range 1 - 81), approximately 84% presented with headache, 47% presented with fever, and 29% presented with chills, and 56% were male (Table I).

 

 

The kappa statistic comparing the two field microscopists' results was 0.95 (p<0.001), indicating good reliability. There were 19 (5%) discordant microscopy results between the first and the second microscopy readers, which were settled by the reference laboratory microscopist. Discordant microscopy results showed that 94% (18 of 19) of the first microscopist's readings were false negatives and 6% (1 of 19) was false positive. Parasite counts were conducted in approximately 52% (10 of 19) of the discordant results. Nine slides were poorly stained or prepared, and parasite counts could not be confirmed on them. Those slides that did have parasite counts showed high-level parasitaemias (range: 560 - >20 000 parasites/µl of blood), implying that these were missed by the first microscopist.

Of the 405 patients tested, 198 (49%) were positive for malaria by ICT Pf, and 191 (47%) were positive by microscopy. Of the 191 positive for Pf on malaria microscopy, 190 were positive by ICT Pf test (p<0.001), representing 1 falsenegative result by MRDT that was not among those that were discordant. Among the 214 patients negative on slide microscopy, 206 were negative by ICT Pf test, representing 8 false-positive results by MRDT.

The overall sensitivity of the ICT Pf malaria test was 99.48% (95% confidence interval (CI) 96.17 - 100%; p<0.001); specificity was 96.26% (95% CI 94.7 - 100%; p<0.001), the positive predictive value of the test was 98.48 (99% CI 98.41 - 100.00%; p<0.001) and the negative predictive value was 99.52% (95% CI 96.47 - 100; p<0.001). The J-index for the test was 0.98 (p<0.001) and the likelihood ratio test (LRT) was 24.75 (positive) and 0.01 (negative) (Table II).

Parasite counts were determined only for 61 slides because thick films were not always correctly prepared by nursing staff. Furthermore, there was only one false-negative ICT Pf result, and therefore there were no substantial observations in this analysis. The median parasitaemia calculated among the 61 slides was 25 680 parasites/µl of blood, ranging from 440 to >20 000 parasites/µl. The sensitivity was 97.7% for all three categories of parasitaemia.

 

Discussion

The incidence of malaria (regardless of diagnostic method) was higher among males, despite the Vhembe district having more female residents. This incidence is probably because the manual labour performed by most males exposes them to mosquitoes, especially at dawn and dusk when mosquito vectors are active.⁸ The living conditions of many workers are also poor and not adequately protected by routine malaria control operations.

There was good agreement between slide readings by the two microscopists. However, the false-negative microscopy results of one microscopist are of concern. Sensitivity of microscopy is expected to decline with low-level parasitaemias, particularly when the parasite density is <100 parasites/µl.^5,9 However, in this study, most false-negative results by a microscopist had a parasitaemia >100 parasites/µl of blood, raising concerns about microscopy quality in the malariaaffected areas of Limpopo province. Malaria microscopy proficiency testing surveys in South Africa have shown generally poor performance in public sector laboratories, with those in malaria transmission areas performing no better than elsewhere.¹⁰ Ongoing training of microscopists, followed by regular slide quality monitoring, should therefore be considered by Limpopo province health authorities.

The high (99.48%) overall sensitivity of the ICT Pf in this study is consistent with other ICT Pf studies.^2,4 It was not possible to assess MRDT sensitivity at a parasite density of <100 parasites/µl of blood because all 61 slides that had a parasite count had >100 parasites/µl of blood. An alternative approach to determining parasite detection levels by the ICT Pf MRDT is to use laboratory dilution of wild-type parasites, similar to that conducted by others.⁵ An evaluation of the ICT Pf MRDT for parasite detection limit, by the Medical Research Council (MRC) of South Africa, using the protocol of Craig et al.,¹¹ showed that the ICT Pf test gave 100% sensitivity at a parasite density of 70 parasites/µl of blood. Although the serial dilution approach is not ideal for determining the MRDT parasite detection limit, this finding gives some indication of ICT Pf sensitivity and specificity at low parasite densities.

There was only one false-negative MRDT result in our study. Reasons for false-negative results include: reduced level of parasites in circulating blood;^5,9 decreased antigenaemia post-treatment of patients; poor end-user interpretation of weak positive results; low levels of antigens at early stages of infection, with presence of only sexual stages of the parasites; a variant HRPII antigen, not captured by the monoclonal antibodies of the ICT Pf testing system; anti-HRPII-Pf antibody potentially blocked immuno-detection by the ICT Pf testing system; and the occurrence of antigen-accelerated HRPII antigen clearance. The only false-negative case had 1 920 parasites/µl of blood and the patient had not taken an antimalairal drug before the MRDT test. Others have reported similar findings.⁵ While MRDT false-negative results were rare in our study, health workers should know that a negative result does not always rule out the likelihood of malaria. However, the possibility of false-negative MRDT results should not be used as an excuse to ignore the MRDT results while making treatment decisions. Patients with an MRDT-negative result need not be treated with an antimalarial, but should be followed up closely and re-examined if they do not improve clinically.

There were 8 false-positive ICT Pf results in our study, possibly because patients previously treated for malaria could still have had circulating antibodies, which can persist for weeks after treatment.^5,9 This was unlikely in our study as patients who had had recent malaria or had recently been on malaria treatment were excluded. Other reasons for falsepositive MRDT results are nonspecific heterophile antibodies produced by recent fever episodes and rheumatoid factor, and the false-positive ICT Pf tests might have been due to problems with microscopy. A highly sensitive test such as the ICT Pf test will yield lower specificities, especially in patients with low parasitaemias.¹ While expert microscopy can detect low-level parasitaemias (10 - 50 parasites/µl), microscopy of blood films in malaria-endemic areas often misses low-density parasitaemias.^2,5,9 Our study had the main limitation of slide preparation by nurses not being ideal, despite training and repeat visits by the research team. A key challenge was that nursing staff rotated among the clinics and hospitals, and often new staff would be present.

The specificity of the test was 96.26%. The relatively lower specificity than sensitivity is similar to other findings.⁵ Having a relatively lower specificity which leads to over-diagnosis and over-treatment of non-malaria cases is considered less serious than having a lower sensitivity.^2,5

 

Conclusions

The ICT Pf test is an appropriate test to use in the field, where laboratory facilities are not available. The test has a high degree of sensitivity and acceptable level of specificity in accordance with WHO criteria. However, this study could not measure the sensitivity of ICT pf to detect parasitaemia <100 parasites/µl of blood. We acknowledge with thanks the assistance of the district and sub-district malaria managers of the Vhembe District in Limpopo Province and the Environmental Health Practitioners for assisting us with co-ordinating the study visits. We are also grateful to the Ernest Oppenheimer Trust for financial assistance in conducting this study.

 

References

1. Moody A. Rapid diagnostic tests for malaria parasites. Clin Microbiol Rev 2002; 15(1): 66-78.         [ Links ]

2. World Health Organization. Malaria Rapid Diagnosis, Making it Work, RS/2003/GE/05(PHL). Geneva: World Health Organization, 2003. http://www.who.int/malaria/cmc_upload/0/000/016/750/rdt2.pdf (accessed 30 June 2008).         [ Links ]

3. Moonasar D, Johnson C, Maloba M, et al. Malaria. In: Ijumba P, Day C, Ntuli A, eds. South African Health Systems Trust Review 2003. Durban: South African Health Systems Trust, 2004: 243-256.         [ Links ]

4. World Health Organization. New Perspectives, Malaria Diagnosis: Report of a Joint WHO/USAID Informal Consultation, W.H.O/MAL/2000.1091. Geneva: World Health Organization, 2000. http://www.wpro.who.int/NR/rdonlyres/3DC6B7D7-711F-4F63-8FF9-A70DBA99DB7E/0/NewPersectives.pdf (accessed 30 June 2008).         [ Links ]

5. Bell D, Peeling RW. Evaluation of rapid diagnostic tests: malaria. Nat Rev Microbiol 2006; 4(9 suppl): S34-38.         [ Links ]

6. Durrheim DN, la Grange JJ, Govere J, Mngomezulu NM. Accuracy of a rapid immunochromatographic card test for Plasmodium falciparum in a malaria control programme in South Africa. Trans R Soc Trop Med Hyg 1998; 92(1): 32-33.         [ Links ]

7. Warhurst DC, Williams JE. Laboratory diagnosis of malaria. J Clin Pathol 1996; 49: 453-538.         [ Links ]

8. National Department of Health. Guidelines for the Prevention of Malaria in South Africa, 2003. Pretoria: National Department of Health, 2003.         [ Links ]

9. Bell D, Wongsrichanalai C, Barnwell JW. Ensuring quality and access for malaria diagnosis: how can it be achieved? Nat Rev Microbiol 2006; 4(9 Suppl): S7-20.         [ Links ]

10. Dini L FJ. Quality assessment of malaria laboratory diagnosis in South Africa. Trans Roy Soc Trop Med Hyg 2003; 97: 675-677.         [ Links ]

11. Craig M. Comparative Evaluation of Three Rapid Malaria Diagnostic Tests. Cape Town: Medical Research Council of South Africa, 12 May 2006.         [ Links ]

 

 

Accepted 30 June 2009.

 

 

Corresponding author: D Moonasar (moonad@health.gov.za)