An evaluation of homogeneity tests in meta−analyses in pain using simulations of individual patient data

Abstract

In this paper we consider the validity and power of some commonly used statistics for assessing the degree of homogeneity between trials in a meta-analysis. We show, using simulated individual patient data typical of that occurring in randomized controlled trials in pain, that the most commonly used statistics do not give the expected levels of statistical significance (i.e. the proportion of trials giving a significant result is not equal to the proportion expected due to random chance) when used with truly homogeneous data. In addition, all such statistics are shown to have extremely low power to detect true heterogeneity even when that heterogeneity is very large. Since, in mos