Summary

The statistical part is an essential ingredient of any research study. The research methodology and application of statistical methods have developed rapidly over the years and have significantly improved the research activities in every field of study. In this paper, is presented an evaluation of statistical methods used in 49 scientific papers, published in biomedical sciences journals in Albania from 2012 to 2016. Based on this review, 31 papers (63.3% of 49) presented numerical results and aimed significance with no statistical analysis performed. Majority of studies which performed statistical analysis (83.3 % of 18 papers) failed to prove test assumptions. In 38.9% of cases there was no explicit statement of the tested Null-Hypotheses. Multivariable techniques to adjust for confounding factors were absent in all papers in which the multivariate analysis was necessary to arrive to conclusions. In many papers (61.1% of cases) the statistical tests used were not specified and defined correctly. Failure to state degrees of freedom was also very common. In these circumstances there is high evidence that the statistical reviewing of biomedical sciences journals has strong deficiencies and must be improved as much as possible.

Keywords: data analysis, documentation of statistics, biomedical research

Introduction

Statistical methods are an inseparable part of the biomedical research with a significant increase in the use of statistics which has been documented for a wide range of biomedical journals over the past decades.¹ However, there is wide consensus that standards are generally low, statistical errors are alarmingly common in published research, according to statisticians at least half of the published papers in biology and medicine contain serious statistical mistakes.^2-3

Unfortunately, even simple and basic statistical methods such as t-tests or chi-square tests are constantly misused in biomedical research, because test assumptions are not assessed before application.⁴

The misuse of statistics in biomedical research has been discussed repeatedly, and it has been indicated that it is dishonest and at times can have serious consequences.^4-5 The inappropriate use of statistical analysis may lead to inaccurate conclusions and false research conclusions. Therefore, valuable efforts have been made by many journal editors to improve the quality of statistics by strengthen the statistical peer reviewing of incoming manuscripts.^6-7 Apart from these efforts, there is low indication that standards have improved over time, for as much as recent studies refer a continuance of major problems.^8-9

In this paper we present a comprehensive evaluation of data analysis and documentation of statistical methods in biomedical research papers in Albania.

Methods

We reviewed 49 scientific papers which presented numerical results, published in Biomedical Journals in Albania from 2012 to 2016. We evaluated the process of data analysis and documentation of statistical methods. The review was based on earlier studies,¹⁰ which showed the most common statistical errors occurring in biomedical research.

Results

Based on the review, 31 papers (63.3% of 49) presented numerical results with no statistical analysis performed (Figure 1). All this studies claimed significance without data analysis or statistical test mentioned.

Figure 1 Statistical analysis in papers with numerical results.

Eighteen papers (36.7% of 49) performed statistical analysis. Many of them contained statistical errors related to data analysis and documentation of statistical methods performed. Some errors were not serious in nature, but many were serious enough to cast doubt on conclusions.

The most frequent deficiency related to data analysis was failure to prove test assumptions. Majority of studies (83.3 % of papers which performed statistical analysis) failed to prove test assumptions. In 38.9% of cases there was no explicit statement of the tested Null-Hypotheses. Multivariable techniques to adjust for confounding factors were absent in all papers in which the multivariable analysis was necessary to arrive to conclusions. In (Table 1) is a summary of several statistical errors related to data analysis and their frequency.

Documentation of applied statistical methods was generally poor and insufficient. In a large number of studies (61.1% of cases) the statistical tests used were not specified and defined correctly. Failure to state if the test was one or two sided was very common. In 66.7 % of papers failed to state if the test was one or two tailed, while in 44.4 % of cases there was no statement for using paired or unpaired t-test. Table 2 summaries important statistical errors related to documentation of statistical methods.

Discussion

When performing statistical data analysis or estimation techniques, it should be clear that each method is based on several underlying assumptions, which have to be fulfilled in order to ensure correct and significant results.

The type of the statistical test applied for a particular data should be clearly explained.¹¹ Every evasive statement, related to the application of different statistical tests should always be avoided.^5,11

Furthermore, when applying t-tests or chi-square tests, researchers have to be aware of choosing the accurate version of the test, since they have various forms.⁴ If expected counts in a cell are less than 5, than chi-square tests should not be used, as their result under this condition is no longer credible.¹¹

It is essential, that all statistical methods applied are described appropriately and with enough detail, to enable a literate reader, to recalculate all results, in case he has access to the data.⁸ In addition, a subsection where all techniques and methods used are explained correctly, is obligatory in every research paper.

For statistical tests, which has paired and unpaired versions (eg, t-test, Wilcoxon test), it is obligatory to specify which form of the test was performed and the degrees of freedom has to be declared. In any case, randomly used tests do not need to be explained in detail, while any new test applied should be summarized or referenced.^10,12

However, it is very difficult for a researcher or academician to study all statistical tests for his or her research. Consequently, still majority of researchers are unaware to which statistical tests they should perform to the data they have collected.¹³ One possible solution to improve statistics in papers published might be adding a statistical review stage to allow the statisticians to have a deeper look at the various statistical observations.

Conclusions

The statistical error level is high in papers published in biomedical research in Albania. Journal editors should seriously consider improvement of quality by enhancing the statistical reviewing of incoming manuscripts, as there is also evidence, that the statistical reviewing of biomedical journals has strong deficiencies. In these circumstances statistical reviewers should at least be given the opportunity to see the revised manuscripts before final publication.

References

1. Altman DG. Statistics in medical journals. Stat Med. 1982;1(1):59–71.
2. Gore SM, Jones IG, Rytter EC. Misuse of statistical methods: critical assessment of articles in BMJ from January to March 1976. Br Med J. 1977;1:85–87.
3. Gardner MJ, Bond J. An exploratory study of statistical assessment of papers published in the British Medical Journal. JAMA. 1990;263:1355–1357.
4. Olsen CH. Review of the use of statistics in infection and immunity. Infect Immun. 2003;71(12):6689–6692.
5. Welch GE 2nd, Gabbe SG. Review of statistics usage in the American Journal of Obstetrics and Gynecology. Am J Obstet Gynecol. 1996;175(5):1138–1141.
6. Goodman SN, Altman DG, George SL. Statistical reviewing policies of medical journals. J Gen Intern Med. 1998;13(11):753–756.
7. Murray GD. Statistical guidelines for the British Journal of Surgery. Br J Surg. 1991;78(7):782–784.
8. Bajwa SS. Basics, common errors and essentials of statistical tools and techniques in anesthesiology research. J Anaesthesiol Clin Pharmacol. 2015;31(4):547–553.
9. Marshall SW. Testing with confidence: the use (and misuse) of confidence intervals in biomedical research. J Sci Med Sport. 2004;7(2):135–137.
10. Strasak AM, Zaman Q, Pfeiffer KP, et al. Statistical errors in medical research – a review of common pitfalls. Swiss Med Wkly. 2007;137(3-4):44–49.
11. Goodman NW, Hughes AO. Statistical awareness of research workers in British anaesthesia. Br J Anaesth. 1992;68(3):321–324.
12. Altman DG. Statistics in medical journals: some recent trends. Stat Med. 2000;19(23):3275–3289.
13. Altman DG, Goodman SN, Schroter S. How statistical expertise is used in medical research. JAMA. 2002;287(21):2817–2820.