RESEARCH

 

Publication trends of clinical trials performed in South Africa

 

 

M Sibanda^{I, II}; R S Summers^III; J C Meyer^IV

^IBPharm Hons; Department of Pharmacy, Sefako Makgatho Health Sciences University, Ga-Rankuwa, Pretoria, South Africa
^IIBPharm Hons; Inqaba Biotechnical Industries (Pty) Ltd, Pretoria, South Africa
^IIIBPharm, MSc, PhD; Department of Pharmacy, Sefako Makgatho Health Sciences University, Ga-Rankuwa, Pretoria, South Africa
^IVBPharm, MSc (Med), PhD Department of Pharmacy, Sefako Makgatho Health Sciences University, Ga-Rankuwa, Pretoria, South Africa

Correspondence

 

 

---

ABSTRACT

BACKGROUND. Investigators and sponsors of clinical trials have an ethical obligation to disseminate clinical trial results, whether positive or negative, in a timely manner.
OBJECTIVES. To determine the publication rate and average time to reporting for clinical trials carried out in South Africa (SA) and to explore factors indicating whether a study is published or not.
METHODS. A registry-based quantitative retrospective analysis of 79 SA clinical trials for new medicines registered between January 2008 and December 2010 was performed. The relevant trial identification number in the register was used to track all peer-reviewed publications subsequent to registration. Tracking of clinical trials was done through a systematic literature search of the electronic journal databases of the South African Medical Journal (SAMJ), the Cochrane Library, Public Library of Science Medical Journal (PLoS Medicine) and BioMed Central, all of which are indexed on MEDLINE via PubMed. In addition, a manual search of the Open Access Journal of Clinical Trials databases and reference lists on articles related to the trial medicine was performed.
RESULTS. Of the 79 clinical trials surveyed, 72 were concluded by December 2014. Only 35 (48.6%) of them had the results published in a peer-reviewed journal, the current benchmark for dissemination of trial results. The majority (82.9%) of those published had a positive outcome. Of the 35 trials that were published, 77.1% were published within 2 years. The average time from completion to initial reporting was 22 months. Fewer than half (40.5%) of the clinical trials surveyed were placebo controlled.
CONCLUSION. The absence of complete outcomes data from SA clinical trials warrants utmost attention. The study puts forward a case to the regulatory body and research ethics committees to compel all data from clinical trials to be made accessible to clinicians and the public in general by being published in an easily accessible form and in a timely manner.

---

 

 

The safety and efficacy of a medicine must be evaluated through appropriate studies before it can be registered and made commercially avail-able.^[1] In South Africa (SA), the Medicines Control Council (MCC) is mandated to oversee the registration of new medicines.^[2] In order to be registered, before permission to sell is granted, medicines must demonstrate acceptable levels of safety, quality and efficacy.^[3] These requirements are achieved by clinical trials carried out by the applicant.

In SA, all clinical trials must be authorised, which means that they must be registered in the South African National Clinical Trials Register (SANCTR) before commencement.^[3] In most clinical trials, safety and efficacy are demonstrated by comparing several predefined outcomes in a treatment group and a control group. The control group is either treated with the standard treatment or a placebo.^[4] The comparison of the outcomes from the treatment group and the control group should show whether the new treatment is efficacious and safe and whether it can be adopted as the new standard in the treatment of the disease in question.^[5]

In a medical environment that supports evidence-based medicine, clinicians and patients base their choice of medicines on published clinical trial results.^[1] Ideally, all the clinical information that leads to a medicine's approval should be publicly available to help clinicians make informed decisions about how to treat their patients.^[6]

A full-length peer-reviewed article in a medical journal is the primary method by which clinical trial results are communicated to the scientific community and the general public.^[6] An incomplete view of the safety and efficacy of the medicine compared with other treatments is created when the results of clinical trials are not published, which may cause prescribing bias by clinicians towards this new treatment.^[7] Furthermore, publication of the clinical trial outcome establishes that the research was done in a scientifically transparent and ethical manner. Peer review adds credibility to the findings of the research. Without publication, claims of a medicine's efficacy and safety are only hearsay.^[6]

The SA guidelines for clinical trials prescribe the following procedure in the publication of trial results. ^[3]

'Sponsors and investigators have an ethical obligation to disseminate research results, whether positive or negative, in a timely manner. It is however important that the release of research findings be done in an ethical manner, to ensure that false expectations are not raised in a vulnerable population. Research results should not be prematurely released or published, or unreasonably delayed. It is advisable that the main results should be disseminated, using appropriate communication formats, to the participants and other interested members of the communities in which the study was conducted.'

Despite the above guidelines, the results of some clinical trials remain unpublished and this can contribute to publication bias, a type of reporting in which the results of completed studies are left unpublished.^[8] Sometimes, the decision whether or not to publish the results of clinical trials may depend on the nature and direction of the research findings. Internationally, trials with positive outcome measures are more often published than those with negative outcomes.^[6] This occurs because it is more favourable for pharmaceutical companies, in their marketing efforts, to utilise data in which their medicine performs well. Clinical trial results in which the medicine performs badly against the standard of treatment, or in which there is no clinically significant advantage, are seldom published.^[6]

Publication bias may also occur for other reasons, including editors of scientific journals rejecting research findings that have nonsignificant or negative findings or academics simply not completing the process of publishing their results.^[8] Another type of reporting bias is the time-lag bias where the publication of research findings is delayed depending on the nature of the results.^[8]

Some well-documented reports concerning the concealment and delayed release of outcome data of the medicine rofecoxib (Vioxx; Merck, USA) in the USA raised awareness of the limited and incomplete public access to clinical trial results for FDA-approved medicines. Merck, the manufacturer of rofecoxib, acted unethically by withholding information about rofecoxib's safety from doctors and patients for >5 years, resulting in between 88 000 and 140 000 cases of serious heart disease.^[7] This case resulted in a determined effort to achieve improved disclosure of trial results.^[7]

The antidiabetic medicine rosiglitazone (Avandia; GlaxoSmith-Kline, UK) is another example where safety data obtained during clinical trials were concealed. Post-marketing surveillance showed an increased risk of cardiac events with the use of this medicine, which eventually resulted in its withdrawal from the market after 11 years of use.^[9]

Oseltamivir (Tamiflu; Roche, Switzerland) has been marketed by Roche since 1999 as an antiviral medication for the treatment of influenza virus types A and B. Based on the evidence available at the time, it was included in the World Health Organization (WHO) essential medicines list in 2011.^[10] Prior to the global outbreak of H1N1 influenza in 2009, the USA alone had stockpiled nearly USD1.3 billion worth of the antiviral.^[10] In 2012, the Cochrane Collaboration was finally granted access to evidence which had previously been concealed and had remained unpublished. This evidence did not support the claims that oseltamivir lowered the risk of complications from influenza and that it decreased viral transmission or that the benefits of using this medicine outweighed the risks.^[10] Globally, over USD17 billion of public money was spent on stockpiling a medicine of uncertain benefit, as a result of decisions based on incomplete data.^[10]

A study conducted in the USA on 909 clinical trials in the period 1998 to 2000 showed that over half (57%) of all trials in the USA for FDA-approved medicines remained unpublished >5 years after approval. The majority (364/394; 92.4%) of the trials that were published were published within 3 years of FDA approval. The study also reported that trials with statistically significant positive results and larger sample sizes were more likely to be published. Furthermore, it was evident that selective reporting of trial results exists for commonly marketed drugs.^[11]

The US Food and Drug Agency (FDA) through the Food and Drug Administration Amendments Act of 2007 (FDAAA) made it mandatory for the results of all clinical trials to be published within a year of completion. An audit done in 2012 showed that close to 80% of trials failed to comply with the requirement to publish clinical trial results within 1 year of completion. Notwithstanding this situation, no fines have yet been issued for non-compliance.^[12]

To address the situation of incomplete outcomes data, the AllTrials campaign was initiated by some clinicians in Europe. The AllTrials campaign advocates for the principles of open research including improved transparency of clinical trials outcomes. The campaign summarises its objectives using the mantra All trials registered, all results reported', meaning that all clinical trials should be listed in a clinical trials registry, and their results should always be shared as open data. It argues that 'Evidence Based Medicine would be so much better if we had ALL the evidence.'^[13]

In SA, there is no legal requirement to compel principal investigators and/or their sponsors to publish the results of clinical trials, although they have an ethical obligation to do so. This article describes an attempt to determine the level of clinical trials reporting and publication in SA.

 

Objectives

To determine the publication rate and average time to reporting for clinical trials carried out in SA and to explore the factors that determine whether a study is published or not.

 

Methods

Study design

The study was a registry-based quantitative retrospective analysis of SA clinical trials for new medicinal products approved and registered between January 2008 and December 2010.

Sample selection

The SANCTR is the legally mandated register for clinical trials in SA.^[3] However, it does not meet the criteria for a primary register set out by the WHO or the International Committee of Medical Journal Editors (ICMJE).^[14] As such, SA clinical trial investigators who plan to meet WHO requirements for transparency and publication need to register with both SANCTR and a WHO- or ICJME-recognised register such as the USA ClinicalTrials.gov register. A limitation of the SANCTR is that it often has incomplete trial information on the register for essential data sets. The ClinicalTrials.gov register is a more information-rich database and has data sets such as trial identification number, trial completion date, primary and secondary outcome measures and funding source. The ClinicalTrials.gov website was deemed to be a more useful registry for the purposes of this study and incidentally recorded more SA clinical trials than the SANCTR in the study period (a subset of 1 190 SA trials registered on the ClinicalTrials.gov website v. 665 on the SANCTR website). These 1 190 clinical trials investigating new medicines classified as 'new molecular entities' (medicinal products that have not previously been approved by the MCC for any indication) for safety and/or efficacy were registered in the period January 2008 - December 2010.

From the subset of 1 190 trials and with an alpha error level of 0.1, beta 0.5, a minimum sample size of 65 was required with ~10% precision level.^[15] A sample of 79 clinical trials was then randomly selected for this study from the 1 190 trials registered on the ClinicalTrials.gov website using the random number generation method on Microsoft Excel (USA).

Determination of publications in journals and search strategy

For each of the selected 79 clinical trials, the unique international trial identification number as listed on the registry was used to track all subsequent publications. Tracking of clinical trials was done through a systematic literature search of electronic journal databases of the South African Medical Journal (SAMJ), the Cochrane Library, Public Library of Science Medical Journal (PLoS Medicine), and BioMed Central, all of which are indexed in MEDLINE via PubMed. In addition, a manual search of the Open Access Journal of Clinical Trials databases and reference lists on articles related to the trial medicine was performed. Non-peer-reviewed journals were excluded from the search as the process to ascertain the quality of articles included in such journals was deemed too time-consuming for the purposes of this study and because such articles are not regarded as reliable evidence and are sometimes 'advertorial' in nature.

The tracking of possible publications in the electronic journals was done for the period starting from the registration of each clinical trial as recorded in the register until the date of the data collection.

Clarification of covariates

The variables for every clinical trial tracked were defined and recorded as follows: Whether or not the clinical trial was completed (Yes/No) as stated in the registry and the date of completion of the clinical trial if available. Dates of registration and commencement of the clinical trials were also recorded if available (1st day of the month if only month and year were recorded in the register). The funding source was categorised as 'funded by industry' or 'funded by academia' or 'other'. The study phases (either clinical trial phase I, II, III or IV) were recorded as stated on the register. For the purposes of this study, if the study was considered by the principal investigator to cover more than one phase, the highest phase was recorded.

The study designs were characterised as either 'randomised, double-blind with standard care as comparator' or 'randomised, double-blind, placebo-controlled trial' or 'not clear'. Therapeutic areas of the clinical trial were categorised as HIV and AIDS, respiratory disease, cardiovascular disease, diabetes, childhood diseases (vaccines), reproductive system, central nervous system and other. Location of the study was categorised as 'South Africa only' or a multinational study. Because interventional studies were only considered in this study, the primary outcome measure of each clinical trial was categorised as 'safety', 'efficacy' or 'both safety and efficacy'. A completed study was considered published if the results of the primary outcome were published in a peer-reviewed journal. The earliest publication date was used in the case of multiple publications. Other possible covariates such as subject ages and number of subjects were excluded from the study.

Ethical considerations

Ethical clearance for the study was obtained from the Sefako Makgatho Health Sciences University Research Ethics Committee (MREC/H/228/2014 as amended) before commencement of the data collection.

Data analysis

The ClinicalTrials.gov registry stated whether the trial was completed or not and also stated the trial completion date where applicable. The time to publication (TTP) was calculated by subtracting the trial completion date from the publication date. The mean and standard deviation (SD) were calculated for the TTP. Frequencies and proportions (%) were calculated to describe categorical variables such as funding source, study phases, study design, therapeutic areas, location of the study and primary outcome measure.

 

Results

Characteristics of clinical trials

Table 1 shows summary characteristics for the clinical trials surveyed. The majority (n=72; 91.1%) of the 79 trials surveyed were completed. Seventy studies (88.6%) were funded solely by the industry while only 2.5% were funded by academia. The majority of the clinical trials (60.8%) were phase III studies, followed by phase II trials (26.6%), phase IV (8.9%) and lastly phase I clinical trials (3.8%).

 

 

Forty-three (54.4%) clinical trials were randomised and double-blinded, with standard care as the comparator, against 32 (40.5%) randomised, double-blinded, placebo-controlled trials. All studies surveyed were located in SA (as per inclusion criteria), with 81.0% of them being multicentre studies with study locations in other countries as well. The major primary outcome measure was efficacy (60.8%), followed by safety (17.7%).

Publication of clinical trial results

The publication profiles of SA clinical trials are presented in Table 2. Of the 72 trials concluded only 35 (48.6%) had the results published in a peer-reviewed journal. Twenty-nine (82.9%) of the trials that were published had a positive outcome. Of the subset of 35 trials that were published, 77.1% were published within 2 years. The mean TTP was 21.8 months (SD 17.7).

 

 

Discussion

Most trials in the sample covered by this investigation were completed. However, more than half (52%) of the clinical trials surveyed failed to achieve transparency and did not release their findings into the public domain. The results are comparable with a study published in the USA in 2008 which reported that over half (57%) of 909 trials in the USA for FDA-approved medicines between 1998 and 2000 remained unpublished >5 years after approval.^[11] Another study carried out in the USA in 2014 after the introduction of the FDAAA in 2007 indicated an improvement in publication rates, with 118 out of 400 (29.5%)

clinical trials failing to disseminate their results within 4 years of completion.^[16] An audit done in 2012 showed that 78% of trials failed to comply with the updated requirement to publish results within 1 year.^[12] In Finland, a study done in 1998 on 188 clinical trials by the Finnish National Agency for Medicines showed that selective reporting was prevalent, with a reporting rate of 30.3% (57/188) on clinical trials surveyed.^[17]

Table 3 shows a comparison of the USA (2008 and 2014) and Finland (1998) country studies with this SA study. SA clinical trials carried out in the period 2008 to 2010 had a similar publication rate in 2014 to that for clinical trials carried out in the USA in the period 1998 to 2000 and a slightly better publication rate than the Finnish study carried out in 1998. This situation occurred despite increased awareness and advocacy for all clinical trials to be published.^[13]

 

 

This study suggests that the results of the clinical trial had a bearing on whether the study would be published or not. Twenty-nine (83%) clinical trials with positive findings for the primary outcome were published. The 1998 Finnish study also demonstrated a similar trend; trials with positive outcomes (76%) were reported more often than those with inconclusive or negative outcomes.^[17] The current study shows that 71% of the clinical trials were published within 2 years of completion. The 2008 USA study revealed that a majority (92%) of the trials published were published within 3 years of FDA approval.^[11]

The frequency of placebo-controlled trials (40.5%) requires further interrogation. The Declaration of Helsinki states that trial participants 'should be assured of the best proven diagnostic and therapeutic method' and that no patient should be denied the 'standard of care'.^[18]

Respiratory diseases, HIV and AIDS, cardiovascular diseases and diabetes were the most surveyed illnesses in the clinical trials surveyed with frequencies of 17.7%, 13.9%, 13.9% and 13.9%, respectively. Despite the relatively small sample size and the associated biases, this profile almost matches local disease trends,^[19] and could indicate that the clinical trials carried out here are, in actual fact, relevant to SA's disease profile.

Study limitations

Our study had two main limitations. Firstly, the study was a registry-based analysis of the Clinicaltrials.gov website. The study depended largely on the reliability of the data in this online repository and assumed that the data were complete and accurate. Secondly, similar studies point to an average TTP of 27 months. A decision was therefore taken to offer a 4-year grace period from trial completion to determine TTP. This resulted in the study being limited to clinical trials completed by December 2010. It excluded trials completed in subsequent years. It is possible, as shown in the USA studies published in 2008^[11] and 2014,^[16] that there could be improved publication rates for SA clinical trials completed after 2010. This matter needs to be investigated further and may form the basis of continued study on this topic. Nevertheless, this study did take into consideration publication of results that occurred until December 2014.

As suggested by Saito and Gill,^[16] other risk factors such as funding mechanism, age of participants in clinical trials or sample size could influence publication behaviour of researchers, which warrants further investigation for trials carried out in SA. An interrogation of principal investigators and sponsors of clinical trials that were not published also needs to be carried out to establish the reasons why the investigators had not published the trial results.

 

Conclusion

This study determined the peer-reviewed publication rate and TTP for clinical trials carried out in SA, as well as other factors which affect these variables. The absence of complete outcomes data from clinical trials is scientific misconduct and has the effect of providing an incomplete view of the safety and efficacy of the medicine tested compared with other treatments. This situation may cause prescribing bias by clinicians as a result of overestimation of the benefits of the new treatment and underestimation of the harmful effects, thereby putting patients at risk.

The study puts forward a case to the regulatory body, medical professional bodies and research ethics committees to compel all data from clinical trials to be made accessible to clinicians and the general public by being published in a peer-reviewed journal or in any other easily accessible form, e.g. an open register, as mooted by AllTrials. com. National central repositories within countries, such as the SANCTR, would also be a helpful platform for posting clinical trial results. A model which the SA regulatory agency could emulate is the International Standard Randomised Controlled Trial Number (ISRCTN) of BioMed Central. The ISRCTN is an official primary registry of the WHO registry network. It contains the basic set of data items for clinical trials registration prescribed by the WHO and the ICMJE guidelines. Furthermore, the FDAAA requires compulsory registration of summaries of trial results for clinical trials with ClinicalTrials.gov by responsible parties, within 1 year of completion of the trial.^[20] A similar initiative of including a summary of the results for each clinical trial as an additional data set in the register could be worth consideration by the SANCTR.

On a positive note, the South African Medical Research Council has signed the AllTrials petition.^[21] However, other medical organisations in SA have yet to do so. As a result of AllTrials and similar campaigns, the European Union Parliament passed a regulation (effective May 2016) that requires all clinical trials conducted in Europe to be registered before commencement of the clinical trials, and the results of the trials to be reported within a year of completion.^[22]

On 29 July 2015, the AllTrials campaign was officially launched in the USA. The USA now joins a worldwide campaign 'calling for every clinical trial - past, present, and future - to be registered and the results from it reported'.^[23] Furthermore, investors from Europe and the USA who are in support of the AllTrials campaign and who hold a combined USD35 trillion investment in pharmaceutical companies have added to the advocacy by requesting these companies to disclose trial results.^[23]

 

References

1. McGauran N, Wieseler B, Kreis J, Schúler YB, Kólsch H, Kaiser T. Reporting bias in medical research -a narrative review. Trials 2010;11:37. DOI:10.1186/1745-6215-11-37        [ Links ]

2. Medicines and Related Substances Control Act (MRSA) 101 of 1965. Pharmaceutical Society of Southern Africa (PSSA). Pharmacy Law Compendium. Durban: LexisNexis Butterworth, 2003.         [ Links ]

3. National Department of Health, South Africa. Guidelines for Good Practice in the Conduct of Clinical Trials in Human Participants in South Africa: Clinical Trials Guidelines. Pretoria: NDoH, 2006.         [ Links ]

4. Machin D, Day S, Green S. Textbook of Clinical Trials. 2nd ed. London: Wiley, 2006:58.         [ Links ]

5. Ritter JM, Lewis LD, Mant T, Ferro A. A Textbook of Clinical Pharmacology and Therapeutics. 5th ed. London: Hodder Arnold, 2008:86-91.         [ Links ]

6. Song F, Parekh S, Hooper L, et al. Dissemination and publication of research findings: An updated review of related biases. Health Technol Assess 2010;14(8). DOI:10.3310/hta14080        [ Links ]

7. Topol EJ. Failing the public health - Rofecoxib, Merck and the FDA. N Engl J Med 2004;351:1707-1709. DOI:10.1056/NEJMp048286        [ Links ]

8. Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). The Cochrane Collaboration, 2011. www.cochrane-handbook.org (accessed 21 December 2015).         [ Links ]

9. Cohen D. Rosiglitazone: What went wrong? BMJ 2010;341:c4848. DOI:10.1136/bmj.c4848        [ Links ]

10. Goldacre B. What the Tamiflu saga tells us about drug trials and big pharma. 2014. www.theguardian.com/business/2014/apr/10/tamiflu-saga-drug-trials-big-pharma (accessed 5 November 2015).         [ Links ]

11. Lee K, Bacchetti P, Sim I. Publication of clinical trials supporting successful new drug applications: A literature analysis. PLoS Med 2008;5(9):e191. DOI:10.1371/journal.pmed.0050191]        [ Links ]

12. Prayle AP, Hurley MN, Smyth AR. Compliance with mandatory reporting of clinical trial results on ClinicalTrials.gov: Cross sectional study. BMJ 2012;344:d7373. DOI:10.1136/bmj.d7373        [ Links ]

13. Chalmers I, Glasziou P, Godlee F. All trials must be registered and the results published. BMJ 2013;346:f105. DOI:10.1136/bmj.f105        [ Links ]

14. World Health Organization (WHO). International Standards for Clinical Trial Registries. Geneva: WHO, 2012:4.         [ Links ]

15. Lwanga SK, Lemeshow S. Sample Size Determination in Health Studies: A Practical Manual. Geneva: WHO, 1991:27-29.         [ Links ]

16. Saito H, Gill CJ. How frequently do the results from completed US clinical trials enter the public domain? A statistical analysis of the ClinicalTrials.gov database. PLoS One 2014;9(7):e101826. DOI:10.1371/journal.pone.0101826        [ Links ]

17. Bardy AH. Bias in reporting clinical trials. Br J Clin Pharmacol 1998;46(2):147-150. DOI:10.1046/j.1365-2125.1998.00759.x        [ Links ]

18. World Medical Association (WMA). Declaration of Helsinki. Ethical principles for medical research involving human subjects. As amended in 2000 with notes of clarification added in 2002 and 2004. WMA, 2004. http://www.wma.net/en/30publications/10policies/b3/index.html (accessed 30 August 2015).         [ Links ]

19. Statistics South Africa (SSA). Mortality and causes of death in South Africa, 2013: Findings from death notification SSA, 2013. http://www.statssa.gov.za/publications/P03093/P030932013.pdf (accessed 30 August 2015).         [ Links ]

20. Food and Drug Administration (FDA) Amendments Act of 2007. Public Law No. 110-85. FDA, 2007. http://www.gpo.gov/fdsys/pkg/PLAW-110publ85/pdf/PLAW-110publ85.pdf (accessed 30 August 2015).         [ Links ]

21. Swanepoel S. SA joins global bid to publish all clinical studies, 2013. http://ftp.bhfglobal.com/sa-joins-global-bid-publish-all-clinical-studies (accessed 21 December 2015).         [ Links ]

22. European Commission (EC). Clinical trials - Regulation EU No. 536/2014. 2014. EC, 2014. http://ec.europa.eu/health/human-use/clinical-trials/regulation/index_en.ht (accessed 30 August 2015).         [ Links ]

23. Moody T. US, EU investors ask Big Pharma to release all drug trial results, including failures, 2015. http://arstechnica.co.uk/tech-policy/2015/07/us-eu-investors-ask-big-pharma-to-release-all-drug-trial-results-including-failures/ (accessed 30 August 2015).         [ Links ]

 

 

Correspondence:
M Sibanda
mncengelis@gmail.com

Accepted 25 February 2016.