UPDATE: In June 2020, the World Health Organization (WHO) and the US National Institutes of Health (NIH) halted the hydroxychloroquine arm of the Solidarity trial and the ORCHID study, respectively, after results showed that hydroxychloroquine provided no benefit to hospitalized patients with COVID-19. Previously hydroxychloroquine trials had been temporarily halted over safety concerns following the publication of a multinational registry analysis in May 2020. However, the authors of the analysis later retracted their paper after the veracity of the data they used was questioned and an independent third-party peer review could not be conducted. Hydroxycholoquine trials resumed briefly after the analysis was retracted, but were stopped again after results indicated that hydroxychloroquine was not effective. The US Food and Drug Administration (FDA) has also revoked the Emergency Use Authorization (EUA) it issued for the drug for the treatment of COVID-19.
April 1, 2020. Doctors across the US were recently urged to sign a petition supporting the use of hydroxychloroquine (Plaquenil) in patients with COVID-19, the disease caused by the new coronavirus (SARS-CoV-2).
“Tell Trump to CUT RED TAPE & make hydroxychloroquine available to you and your patients,” the text message read.
Twitter and the media have also been full of praise for Dr. Dider Raoult and the success he reported using hydroxychloroquine and azithromycin (Z-Pak, Zithromax) in 80 COVID-19 patients in France and for Dr. Vladimir Zelenko who added zinc to the same regime and used it to treat 699 patients.
However, critics have called into question the "evidence" these doctors have reported. So are the critics nitpicking or do they have a valid point about the poor quality of the evidence to date?
The ‘gold standard’ for evidence
Large, randomized double-blind placebo-controlled studies are the gold standard of clinical trial design when determining how effective a drug is. Such a design allows us to compare different treatment options. It minimizes bias and confounding factors and gives us confidence that the outcomes we observe are due to the treatment being studied and not some other factors.
Why are there concerns about Raoult’s results? Isn’t it ok to cut corners in a pandemic?
Raoult’s initial study was conducted in 26 patients with COVID-19 in France. Patients were assigned to a control group or a treatment group, which received hydroxychloroquine alone or in combination with azithromycin. The trial has some obvious shortcomings, including its small size and its lack of randomization and blinding, which help to reduce bias. Given the urgency with which an effective treatment for COVID-19 is required, some corner cutting is understandable. However, there are other shortcomings or flaws that are of more concern. For example:
The trial included patients in the control group who refused to give their consent to be treated with hydroxychloroquine and azithromycin. It is not normal to include such patients in a trial and is a concern from an ethics point of view.
The results likely look more promising than they are because the data from 6 of the 26 patients was omitted, including data from 3 patients who became so unwell they were admitted to the intensive care unit (ICU) and from 1 patient who died.
It is unclear why, but the study endpoint was the presence or absence of virus after 6 days in the study and it was not a good choice. The issues with this endpoint are obvious when we consider that one patient who had a negative result on day 6 then went on to record a positive result again two days later. And another patient had a positive result on day 6, but was negative just two days later.
Raoult has published further results which include data from a total of 80 hospitalized COVID-19 patients. However, unlike the first report, there are no results from a control group. This means that we have no comparison, no way of knowing if the treatment made a difference compared with patients who did not receive it.
Why are there concerns about Zelenko’s results? He’s treated 669 patients and none have died
Zelenko is a New York-based doctor, whose practice has treated 669 patients with hydroxychloroquine, azithromycin and zinc in an outpatient setting. He settled upon this regimen after reading the results from Raoult’s study and another study from South Korea investigating hydroxychloroquine and zinc. His practice is treating patients of any age with shortness of breath, those over 60 or who are immunocompromised and displaying mild symptoms. His approach also has a number of shortcomings or flaws. For example:
Zelenko has not published any results at all. He has simply announced his findings on the Internet, so we have no way of analyzing them.
We do not know how many of the 669 patients had COVID-19. Zelenko is reported to have said that he has largely not actually had his patients tested for the new coronavirus because he was worried this would delay treatment and affect its efficacy.
We do not know if the patients actually took the treatment. Unlike in a clinical setting where the patients may be observed taking their medications, these patients treated themselves at home.
So while well-intentioned, Zelenko’s word and approach is not enough to tell us whether or not his regimen is actually effective against COVID-19.
Glimmer of hope
While an earlier study conducted in 30 patients hospitalized with COVID-19 in China did not show hydroxychloroquine provided any benefit, another group of researchers have reported more promising results. These researchers conducted a randomized double-blind controlled trial in 62 patients with COVID-19 at another hospital in China. The average age of the patients was 44.7 years. The trial enrolled patients with relatively mild COVID-19, but with pneumonia evident on chest CT.
All of the 62 patients enrolled in the trial received standard treatment consisting of oxygen therapy, antiviral and antibacterial drugs, and immunoglobulins with or without corticosteroids. The 31 patients randomized to the treatment arm also received hydroxychloroquine for the first 5 days. Neither the patients nor the researchers were aware of who was being treated with hydroxychloroquine. The researchers have reported that:
Fevers and coughs resolved more quickly in patients who received hydroxychloroquine compared with those who only received standard care.
None of the patients who received hydroxychloroquine developed severe illness, compared with 4 of the patients who did not receive the drug.
Chest CT results showed that 80.6% of patients in the hydroxychloroquine arm had improvement in their pneumonia, compared with 54.8% of the patients receiving standard care alone.
A total of 61.3% of patients in the hydroxychloroquine group were classed as having a significant improvement in their pneumonia.
Although this study was also small and larger clinical trials are still needed, it does not appear to suffer from the same limitations and flaws as Raoult’s and Zelenko’s evidence.
Results from further trials investigating hydroxychloroquine in the US and Brazil are expected over the coming weeks. In the meantime, the US Food and Drug Administration (FDA) has issued an Emergency Use Authorization (EUA) allowing hydroxychloroquine and chloroquine (Resochin) to be prescribed to hospitalized COVID-19 patients who can’t access the drugs through a clinical trial.
- World Health Organization (WHO). "Solidarity" clinical trial for COVID-19 treatments. Update on hydroxychloroquine. June 17, 2020. Available at: https://www.who.int/emergencies/diseases/novel-coronavirus-2019/global-research-on-novel-coronavirus-2019-ncov/solidarity-clinical-trial-for-covid-19-treatments. [Accessed June 24, 2020].
- National Institutes of Health (NIH). NIH halts clinical trial of hydroxycholoroquine. June 20, 2020. Available at: https://www.nih.gov/news-events/news-releases/nih-halts-clinical-trial-hydroxychloroquine. [Accessed June 24, 2020].
- Food and Drug Administration (FDA). Coronavirus (COVID-19) Update: FDA Revokes Emergency Use Authorization for Chloroquine and Hydroxychloroquine. June 15, 2020. Available at: https://www.fda.gov/news-events/press-announcements/coronavirus-covid-19-update-fda-revokes-emergency-use-authorization-chloroquine-and. [Accessed June 24, 2020].
- Stat News. WHO resumes hydroxychloroquine study for Covid-19, after reviewing safety concerns. June 3, 2020. Available at: https://www.statnews.com/2020/06/03/who-resuming-hydroxychloroquine-study-for-covid-19/. [Accessed June 5, 2020].
- Mehra MR, Ruschitzka F, Patel AN. Retraction—Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis. Lancet. Published Online June 4, 2020 https://doi.org/10.1016/ S0140-6736(20)31324-6.
- NPR. WHO Halts Hydroxychloroquine Trial Over Safety Concerns. May 25, 2020. Available at: https://www.npr.org/sections/coronavirus-live-updates/2020/05/25/861913688/who-halts-hydroxychloroquine-trial-over-safety-concerns. [Accessed May 27, 2020].
- Mehra MR, Desai SS, Ruschitzka F, Patel AN. Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis. Lancet May 22, 2020. DOI: https://doi.org/10.1016/S0140-6736(20)31180-6.
- Reuters. U.S. group bombards doctors with coronavirus petition to cut 'red tape'. March 28, 2020. Available at: https://www.reuters.com/article/us-health-coronavirus-text-messages/us-group-bombards-doctors-with-coronavirus-petition-to-cut-red-tape-idUSKBN21E367. [Accessed April 1, 2020].
- Sackett D, Rosenberg W, Gray J, et al. Evidence based medicine: what it is and what it isn't. BMJ 1996;312:71. doi: https://doi.org/10.1136/bmj.312.7023.71.
- Sibbald B. Understanding controlled trials: Why are randomised controlled trials important? BMJ 1998;316:201. doi: https://doi.org/10.1136/bmj.316.7126.201.
- Gautret P, Lagier J-C, Parola P, et al. Hydroxychloroquine and azithromycin as a treatment of COVID‐19: results of an open‐label non‐randomized clinical trial. Int J Antimicrob Agents. Available online March 20, 2020. https://doi.org/10.1016/j.ijantimicag.2020.105949. [Accessed April 1, 2020].
- Darren D, Gates S, Morris T. Statistical review of Hydroxychloroquine and azithromycin as a treatment of COVID-19: results of an open-label non-randomized clinical trial. March 23, 2020. Available at: https://zenodo.org/record/3725560#.XoUOR9MzbyJ. [Accessed April 1, 2020].
- For Better Science. Chloroquine genius Didier Raoult to save the world from COVID-19. March 26, 2020. Available at: https://forbetterscience.com/2020/03/26/chloroquine-genius-didier-raoult-to-save-the-world-from-covid-19/. [Accessed April 1, 2020].
- Gautret P, Lagier J-C, Parola P, et al. Clinical and microbiological effect of a combination of hydroxychloroquine and azithromycin in 80 COVID-19 patients with at least a six-day follow up: an observational study. Available at: https://www.mediterranee-infection.com/wp-content/uploads/2020/03/COVID-IHU-2-1.pdf. [Accessed April 1, 2020].
- Lowe, D. Science Translational Medicine. In the Pipeline. More on Chloroquine/Azithromycin. And On Dr. Raoult. March 29, 2020. Available at: https://blogs.sciencemag.org/pipeline/archives/2020/03/29/more-on-cloroquine-azithromycin-and-on-dr-raoult. [Accessed April 1, 2020].
- Forward. This doctor was already treating patients with Trump’s ‘gift from God’ drug - before FDA approval. March 24, 2020. Available at: https://forward.com/news/national/442285/coronavirus-hydroxychloroquine-trump-doctor/. [Accessed April 1, 2020].
- Zelenko V. Correspondence from Dr Vladimir Zelenko on Treatment of COVID-19 in New York. March 23, 2020. Available at: https://docs.google.com/document/d/1SesxgaPnpT6OfCYuaFSwXzDK4cDKMbivoALprcVFj48/mobilebasic. [Accessed April 1, 2020].
- TechStartups. Dr. Vladimir Zelenko provides important update on three drug regimen of Hydroxychloroquine Sulfate, Zinc and Azithromycin (Z-Pak) he used to effectively treat 699 coronavirus patients with 100% success [UPDATE 3]. March 31, 2020. https://techstartups.com/2020/03/31/dr-vladimir-zelenko-provides-important-update-three-drug-regimen-hydroxychloroquine-sulfate-zinc-azithromycin-z-pak-used-effectively-treat-699-coronavirus-patients-100-su/. [Accessed April 1, 2020].
- COLlive. Backed by Trump, Dr. Zelenko Responds to Critics Of His Virus Cure. March 25, 2020. Available at: https://collive.com/backed-by-trump-dr-zelenko-responds-to-critics-of-his-virus-cure/. [Accessed April 1, 2020].
- Chen Z, Hu J, Zhang Z, et al. Efficacy of hydroxychloroquine in patients with COVID-19: results of a randomized clinical trial. medRxiv. doi: https://doi.org/10.1101/2020.03.22.20040758. [Accessed April 1, 2020].
- Chen J, Liu D, Liu L, et al. A pilot study of hydroxychloroquine in treatment of patients with common coronavirus disease-19 (COVID-19). J Zhejiang Univ (Med Sci). 2020, 49(1) 0-0. DOI: 10.3785/j.issn.1008-9292.2020.03.03. [Accessed April 1, 2020].
- Reuters. Brazil doctor urges caution on coronavirus 'game changer' drug. March 26, 2020. Available at: https://www.reuters.com/article/us-health-coronavirus-brazil-drug-idUSKBN21C3E5. [Accessed April 1, 2020].
- NPR. Clinical Trials Set To Determine If Anti-Malaria Drug Effective Against COVID-19. March 31, 2020. Available at: https://www.npr.org/sections/coronavirus-live-updates/2020/03/31/824572119/clinical-trials-set-to-determine-if-anti-malaria-drug-effective-against-covid-19. [Accessed April 1, 2020].
- Live Science. Study will test if hydroxychloroquine can prevent COVID-19. Available at: https://www.livescience.com/hydroxychloroquine-prevent-covid-19-study.html. [Accessed April 1, 2020].
- U.S. Department of Health & Human Services (HSS). HHS accepts donations of medicine to Strategic National Stockpile as possible treatments for COVID-19 patients. March 29, 2020. Available at: https://www.hhs.gov/about/news/2020/03/29/hhs-accepts-donations-of-medicine-to-strategic-national-stockpile-as-possible-treatments-for-covid-19-patients.html. [Accessed April 1, 2020].