AUCTORES
Globalize your Research
Research Article | DOI: https://doi.org/10.31579/2693-4787/042
1 University of Texas Medical Branch, Department of Internal Medicine, Galveston, Texas
*Corresponding Author: Moe Ameri. University of Texas Medical Branch, Department of Internal Medicine, Galveston, Texas.
Citation: Moe Ameri, Pooja Bhakta, Marissa Lee, Benjamin Greiner. (2022). Unpublished Literature Use Among Oncology Systematic Reviews; A Bibliometric Analysis, J. Clinical Oncology Research and Reports, 3(1) DOI: 10.31579/2693-4787/042
Copyright: © 2022 Moe Ameri. This is an open-access article distributed under the terms of The Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Received: 19 August 2022 | Accepted: 31 August 2022 | Published: 17 January 2023
Keywords: oncology; systematic review; unpublished research; publication bias; clinical trial registry
Aim: Despite mechanisms in which to prevent publication bias, it continues to have significant impacts on systematic reviews. Searching clinical trial registries for unpublished literature while conducting systematic reviews can reduce the effects of publication bias. In this study, we aim to analyze the use of clinical trial registry searches in hematology oncology systematic reviews.
Materials and Methods: Systematic reviews and meta-analysis published between the dates of 10/26/11 - 10/26/21 from the top five leading hematology oncology journals were screened for clinical trial registry use. Of the reviews that did not perform clinical trial registry searches, we assessed twenty-five at random on ClinicalTrials.gov for any trials that were potentially missed by not performing a registry search.
Results: 302 studies were assessed, two were inaccessible, for a total of three hundred reviews analyzed. Of those, thirty-five (11.67%) performed clinical trial searches. Twenty-five randomly sampled, twenty-two (88%) had at least one potentially relevant trial that was missed, with a median of 1, range of 0-6 (IQR 1-3), and 2 systematic reviews excluded as outliers.
Conclusions: Omission of clinical trial registry searches by systematic reviewers may result in missing potentially relevant data. Our study identified a large proportion of hematology oncology systematic reviews which missed potentially relevant trials as a result of clinical trial registry search avoidance. We recommend utilizing clinical trial registry searches in order to reduce publication bias.
Systematic reviews are a process for synthesizing existing evidence in an attempt to provide the most up-to-date recommendations on a clinical question or healthcare topic.[1,2] These reviews can combine primary research on a given topic or multiple meta-analyses, a statistical method designed to derive an overall effect from several smaller studies.[2–4] Many systematic reviews are used as the basis for practice guidelines to assist physician decision making - a result of their ability to improve sample sizes and generalizability compared to individual studies.[5,6] Although systematic reviews have the ability to change clinical practice, limitations of these studies must be considered.
One limitation of systematic reviews is publication bias. Publication bias is the tendency to preferentially publish research with results that are clinically or statistically significant and not publish those that demonstrate negative effects. [2,6] Several mechanisms are commonly used to limit publication bias including funnel-plots, selection models, and Egger’s regression intercept. Another method to reduce publication bias is through clinical trial registry (CTR) searches for grey, or unpublished, literature when conducting a systematic review. [7]
The International Clinical Trials Registry Platform was designed to ensure that a complete view of research is accessible to all those involved in healthcare decision making in order to improve research transparency and strengthen the validity and value of the scientific evidence base.[8] As a result of this platform, the International Committee of Medical Journal Editors (ICMJE) began to require registration in a CTR to be considered for publication. [9] In 2007, the Food and Drug Amendments Act of 2007 that required all clinical trials to be recorded at ClinicalTrials.gov.[10] Despite these advances to prevent publication bias, many systematic reviews fail to perform CTR searches for unpublished literature; one study of anaesthesia related systematic reviews found that only 43% of systematic reviews evaluated for publication bias while 20.3% included a search of the unpublished literature.[11] The omission of CTR searches can result in publication bias and reduce the trustworthiness of systematic reviews.
In this bibliometric analysis, we aimed to determine the extent to which haematology oncology systematic reviewers performed CTR searches in an effort to reduce publication bias. The primary objective was to determine the frequency of which haematology oncology publications searched clinical trial registries. Our secondary objective was to determine the prevalence of potentially relevant trials which were missed by systematic reviews that did not perform CTR searches. Finally, we made recommendations to reduce publication bias in future systematic reviews.
This study did not meet requirements for human study participants research as explained by U.S. Department of Health and Human Services 45 CFR part 46 and therefore was not eligible for IRB approval. This study was conducted for meta-analysis reporting as described by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. [12]
The top 5 leading Hematology Oncology Journals were selected using the Google Scholar h5-index metric. This tool measures visibility and citation impact of publications associated with different journals. The Journals selected were American Journal of Clinical Oncology, Lancet Oncology, Annals of Oncology, JAMA Oncology, and Nature Reviews Clinical Oncology. Following this, a PubMed search was performed among the top 5 H-5 index oncology journals between 10/26/11 - 10/26/21 for all meta-analysis and systematic reviews using the following search query: ((((("American Journal of Clinical Oncology"[Journal]) OR "The Lancet Oncology"[Journal]) OR "Ann Oncol"[Journal]) OR "JAMA Oncology"[Journal]) OR "Nature Reviews Clinical Oncology"[Journal]) AND ((("meta-analysis"[Title/Abstract]) OR "meta analyses"[Title/Abstract]) OR "meta-analysis"[Title/Abstract]). This search query was created using a previously validated method for accurately searching through appropriate systematic reviews. [13]
The primary objective was achieved by screening each article for clinical trial registry searches in the full manuscript. Classification of systematic reviews were stated by authors and had to fulfil the definitions of each type of study. [14] Our inclusion criteria were all systematic reviews and meta-analysis accepted for publication prior to 10/26/2021. Exclusion criteria were studies that were not systematic reviews or meta-analysis. Each article, including the abstract and full manuscripts, were accessed using the home author’s library subscription. The primary outcome was to analyse the proportion of systemic reviewers of haematology oncology publications that searched clinical trial registries.
The second objective was to query systematic reviews that did not perform clinical trial registry searches for potentially relevant, missed trials. To do this, 25 systematic reviews that did not perform clinical trial registry searches were randomly selected using the Microsoft Excel simple random sampling technique. Search queries were created based on our best approximation of what the study would have used by reviewing manuscript key words, diseases, treatments, and other interventions. Using the search string, relevant trials were obtained from ClinicalTrials.gov. Trials deemed relevant were then assessed for publication status. References of the systematic reviews were then reviewed and, if the potentially relevant trial was referenced by the original systematic review, the trial was not counted against the study. Potentially relevant trials were only counted against the systematic review if they were published before the end date for which systematic review methods concluded.
Our query resulted in 302 studies which met inclusion criteria. Of this, 2 systematic reviews were inaccessible, resulting in a final sample of 300 studies. Thirty-five systematic reviews (35/300; 11.67%) performed CTR searches. Nature Reviews Clinical Oncology and Annals of Oncology had the lowest percentage of systematic reviews which performed CTR searches (0% and 8%, respectively) published in their journals (Table 1).
Table 1: Number of CTR searches by Year and Journal
JAMA Oncology had the greatest percentage of publications which performed CTR searches (18%). Based on our analysis, there was a positive trend in CTR searches by year with a slope intercept of y=0.518x + 0 (R2 0.401; Figure 1).
Figure 1: A line chart is shown and represents the number of CTR searches by year. Slope intercept; y=0.518x + 0; R2=0.401.
The most frequently searched CTR was ClinicalTrials.gov (34) followed by WHO International CTR (5), European Union CTR (2), International Standards of CTR (1), and Australia and New Zealand CTR (0). Among systematic reviews that did not perform CTR searches, 53 reported searching Cochrane Central Register of Controlled Trials, which is not a clinical trial registry as it only reports trial results rather than full trial details.
In our secondary analysis of 25 randomly selected systematic reviews that did not perform a CTR search, 22 of 25 (88%) had at least 1 potentially relevant trial that was missed. The median number of missed relevant trials was 1 with a range of 0-6 (IQR 1-3) with 2 systematic reviews excluded as outliers (Figure 2).
Figure 2: A box and whisker plot represents the minimum, median, maximum, 1st interquartile, and 3rd interquartile range of missed relevant trials. Dots represent outliers.
Despite the use of mechanisms which reduce the effects of publication bias, it continues to impact systematic review results. [2,6] In this study, we aim to analyse the use of clinical trial registry (CTR) searches in haematology oncology systematic reviews. Based on our findings, by not using CTR searches, many relevant unpublished trials can be missed, and this omission can lead to negative effects on the accuracy and trustworthiness of haematology oncology systematic reviews. This can be improved by utilizing clinical trial registry searches to reduce publication bias.
From our findings, approximately 88% of all haematology oncology systematic reviews did not conduct a CTR search. These findings are like other findings in anaesthesiology and critical care journals. The use of clinical trial registries in anaesthesiology systematic reviews found about only 12% of systematic reviews used CTR searches.[15] Similarly, another study in critical care systematic reviews found only 11% of systematic reviews used CTR searches.[16] Consequently, haematology oncology systematic reviews may have a large negative impact from publication bias.
The consequences of not performing CTR searches can be detrimental. For instance, a meta-analysis that included unpublished data on the use of quinine for nocturnal leg cramps showed altered results when unpublished data was included with published data.[17] This study exemplifies the effects of publication bias as almost all published studies had reported larger estimates of the efficacy of quinine in leg cramps than did unpublished studies.[17] Furthermore, a meta-analysis of published and unpublished data on the management of major depressive disorder concluded that published data overestimated the benefit of reboxetine versus placebo by up to 115% and reboxetine vs SSRIs by up to 23% and also underestimated harm.[18] Lastly, a revisited study comparing systematic review and meta-analysis of published and unpublished randomized trials on the efficacy and acceptability of agomelatine, found that none of the negative trials were published and conflicting results were found between published and unpublished studies.[19] This study found the standardized effect size to be more than three times higher in the published trials compared to the unpublished trials.[19] When studies were compared between published vs unpublished, a significant antidepressant effect of agomelatine was shown in the subgroup of published trials only. Therefore, clinical relevance of the antidepressant effect of agomelatine was not able to be evaluated as the meta-analysis only included published trials. [19]
Our secondary analysis found that 88% of systematic reviews that did not perform a CTR search had at least 1 potentially relevant, unpublished trial that was omitted and could have been recognized with a registry search. Given that most reviews had more than 1 potentially relevant trial omitted, we cannot rule out that the published results by many of these studies may have been confounded by publication bias. Using unpublished data can strengthen the conclusion of systematic reviewers. Additionally, and most importantly, unpublished data can shift the risk-benefit ratio against their favor and may even identify adverse events to therapies being studied.
Researchers should make every effort to limit the effects of publication biases as these biases can pose serious flaws to systematic reviews and the yield to their study. One such way to limit these biases is by including unpublished data by using a clinical trial registry search. However, an integrated approach should be taken to limit biases from every angle. There are several mechanisms in which to limit publication bias, such as funnel-plots, selection models, and Egger’s regression intercept. [7] Many of these techniques have unique methods for detecting bias. Therefore, from our study, we recommend systematic reviewers utilize clinical trial registry searches as well as other methods for reducing publication bias.
Taking at face value the core value of a systematic review is to provide a review of the evidence that is available about a particular topic, systemic reviews should be well defined, using a clear and particular questions to guide the narrative alongside, they should also use clear methods to identify, select and appraise the research, so they can extract and analyze data properly.[20] Therefore it is reasonable to think that systematic reviews can possibly benefit from including both published and unpublished literature to avoid a type of publication bias, called positive outcomes bias. Meaning generally positive outcomes are usually the ones published. Therefore, we believe that doing a CTR search, among other researching tools will help eliminate this bias, because it will allow researchers a more holistic approach to systemic review.
There were many limitations to this study. As previously discussed, many researchers intentionally omit CTR searches which may lead to a substantial decrease in the number of CTR searches found in this study. Our search query may have failed to be broad enough to include all haematology oncology systematic reviews regardless of our use of previously published methodology. [15,16] Lastly, some values may have also been incorrectly recorded prior to the analysis. However, this was reduced by our use of two reviewers (PB and MA) performing independent queries of all studies and cross-validating inaccuracies.
This study found that 11.7% of all haematology oncology systematic reviews conducted a clinical trial registry search. Furthermore, 88% of systematic reviews that did not perform a CTR search missed 1 or more relevant trials. By not using clinical trial registry searches, potentially relevant, unpublished trials may be missed. This omission could have negative impacts on the accuracy of haematology oncology systematic reviews. We recommend utilizing clinical trial registry searches to reduce publication bias.
Clearly Auctoresonline and particularly Psychology and Mental Health Care Journal is dedicated to improving health care services for individuals and populations. The editorial boards' ability to efficiently recognize and share the global importance of health literacy with a variety of stakeholders. Auctoresonline publishing platform can be used to facilitate of optimal client-based services and should be added to health care professionals' repertoire of evidence-based health care resources.
