Vitamin D meta-analyses often disagree due to selection of: trial, data extraction, statistics, etc. Dec 2014

A Case Study of Discordant Overlapping Meta-Analyses: Vitamin D Supplements and Fracture
Mark J. Bolland*, Andrew Grey
Department of Medicine, University of Auckland, Private Bag 92 019, Auckland 1142, New Zealand *m.bolland@auckland.ac.nz

Background: Overlapping meta-analyses on the same topic are now very common, and discordant results often occur. To explore why discordant results arise, we examined a common topic for overlapping meta-analyses- vitamin D supplements and fracture.

Methods and Findings: We identified 24 meta-analyses of vitamin D (with or without calcium) and fracture in a PubMed search in October 2013, and analysed a sample of 7 meta-analyses in the highest ranking general medicine journals. We used the AMSTAR tool to assess the quality of the meta-analyses, and compared their methodologies, analytic techniques and results. Applying the AMSTAR tool suggested the meta-analyses were generally of high quality. Despite this, there were important differences in trial selection, data extraction, and analytical methods that were only apparent after detailed assessment. 25 trials were included in at least one meta-analysis. Four meta-analyses included all eligible trials according to the stated inclusion and exclusion criteria, but the other 3 meta-analyses “missed” between 3 and 8 trials, and 2 meta-analyses included apparently ineligible trials. The relative risks used for individual trials differed between meta-analyses for total fracture in 10 of 15 trials, and for hip fracture in 6 of 12 trials, because of different outcome definitions and analytic approaches. The majority of differences (11/16) led to more favourable estimates of vitamin D efficacy compared to estimates derived from unadjusted intention-to-treat analyses using all randomised participants. The conclusions of the meta-analyses were discordant, ranging from strong statements that vitamin D prevents fractures to equally strong statements that vitamin D without calcium does not prevent fractures.

Conclusions: Substantial differences in trial selection, outcome definition and analytic methods between overlapping meta-analyses led to discordant estimates of the efficacy of vitamin D for fracture prevention. Strategies for conducting and reporting overlapping meta-analyses are required, to improve their accuracy and transparency.

 Download the PDF from VitaminDWiki

The following is most of the text and only a few of the many excellent tables from the study/PDF

Introduction

The number of meta-analyses published in recent years has dramatically increased [1,2]. Partly, this is because systematic reviews and meta-analyses are considered the highest level of evidence, but it is also relatively easy to undertake and publish a meta-analysis [3]. However, many meta-analyses are not novel, and either reproduce or extend earlier analyses on the same topic- i.e. are overlapping. In a random sample of meta-analyses that were published in 2010 and included randomised trials, 67% had at least one other overlapping meta-analysis [3]. Overlapping meta-analyses may report discordant results and conclusions, particularly as the number of such analyses increases. The consequences of this include contradictory recommendations for clinical practice, confusion amongst clinicians and their patients, and public disenchantment with clinical science.
Discordant meta-analyses have been reported previously for a variety of interventions [4-8], and recommendations for assessing such meta-analyses are available [9]. These recommendations focus on the methods and quality of the review, both of which have become much more standardised since the recommendations were proposed. The effect of vitamin D supplements on fracture is the subject of a large number of meta-analyses [10]. In 2012, an individual patient data meta-analysis was the 21st meta-analysis published on this topic, but identified only 14 relevant randomised controlled trials [11]. Two recent clinical guidelines on vitamin D [12,13], based on meta-analyses of the same clinical trials by independent groups, reached very different conclusions [14,15]. As a case study of overlapping meta-analyses, we conducted a detailed review of meta-analyses of vitamin D and fracture. We investigated differences between the meta-analyses by applying recommendations for comparing discordant overlapping meta-analyses. We focused on the quality and methodology of the meta-analyses with regard to trials included, trial data utilised, analytic approaches, and conclusions, and considered the implications these differences have for clinical practice, interpretation of existing meta-analyses and performance of future analyses.

Methods

Ethics statement
Ethical approval was not required for this work.
We searched PubMed in October 2013 for systematic reviews and metaanalyses of vitamin D with fracture as an outcome (S1 Appendix). We identified 24 meta-analyses, and analysed the most recent meta-analysis in each of the highest ranking general medical journals (Ann Intern Med, BMJ, Cochrane Database Syst Rev, JAMA, JAMA Intern Med, Lancet, NEJM). We chose this sample of meta-analyses because they are likely to have been conducted to the highest standard, as well as being the most closely scrutinised during peer review and post-publication. Thus, we analysed 5 trial-level and 2 patient-level metaanalyses on the effect of vitamin D with or without calcium on fracture [16-22].
Jadad and colleagues recommended assessing discordant systematic reviews in 6 domains- the clinical question asked, study selection and inclusion, data extraction, study quality, ability to combine studies, and statistical methods [9]. Following this approach, the quality of each meta-analysis was assessed using the AMSTAR tool [23], and the inclusion and exclusion criteria, endpoints, trials included, and data on hip and total fracture outcomes for each contributing study were extracted by one author (MB) and checked by a second (AG). Differences were resolved by consensus. Some trials reported data for non-vertebral fracture rather than total fracture. In this situation, we used data for non-vertebral fracture when total fracture data were not available. Only one meta-analysis described the reasons for exclusion of individual trials in detail [18]. For each of the other metaanalyses, we assessed whether trials that were not included in the meta-analysis were eligible for inclusion according to the published inclusion and exclusion criteria, and tried to determine why the trial was not included. Where data for the efficacy of vitamin D on fracture outcomes for a trial differed between metaanalyses, we tried to determine the reason. The recommended approach to analysis of a randomised controlled trial is an unadjusted intention-to-treat analysis using all randomised participants with data from the final study timepoint [24]. We considered the result from this approach to be the best estimate of treatment efficacy. An analysis restricted to those participants who completed the study was termed a ‘‘per-protocol analysis.’’ Finally, we compared the conclusions of the meta-analyses, with each author independently rating the strength of the conclusions toward the use of vitamin D supplementation to prevent fracture on a three point scale (positive/mixed/negative toward vitamin D supplementation) and on a scale from 1 to 5 (1 most negative, 5 most positive toward vitamin D supplementation). These ratings were based solely on the conclusions of the meta-analysis, and did not consider the data or analyses used in the meta-analyses.

Results

Table 1 shows the characteristics of the 7 meta-analyses and the trials included in each meta-analysis. The number of included trials in each meta-analysis ranged from 7 to 20. Of the 25 trials included in any of the meta-analyses [25-50], 6 were included in only 1 meta-analysis, 3 in 2 meta-analyses, 2 in 3 meta-analyses, 3 in 4 meta-analyses, 7 in 5 meta-analyses, and 4 in 6 meta-analyses. No trial was included in all meta-analyses.
X= study not included in meta-analysis. Reasons for non-inclusion: NDA- eligible for inclusion but no patient-level data available; agent- did not compare vitamin D plus calcium with placebo; size- study smaller than inclusion criteria allowed; age- age outside inclusion criteria; uncertain- unknown reason for exclusion; duration- duration of study less than inclusion criteria allowed; design- design did not meet inclusion criteria; controls- untreated control group; date- after search date; IM- intramuscular administration.
alimited or no fracture data in primary publication, but data obtained from lead author and published in at least 1 meta-analysis. The Bolton-Smith fracture trial data were not published in Avenell 2009 until after publication of the Bischoff-Ferrari 2009 meta-analysis. btrial does not appear to meet eligibility criteria for meta-analysis.
Abbreviations: Secondary- included in secondary analyses only. Vit D: vitamin D.

The number of trials that met criteria for inclusion but were not included in each meta-analysis ranged from 0 to 8 trials: 4 meta-analyses included all trials that met the eligibility criteria [16-18,20], with 2 meta-analyses ‘‘missing’’ 3 trials [19,22], and 1 missing 8 trials [21]. Of the 8 trials that were missing from at least 1 meta-analysis, 4 were missed in 1 meta-analysis, 2 in 2 meta-analyses, and 2 in 3 meta-analyses. Two meta-analyses [ 16,17] included 1 and 3 trials, respectively, that did not appear to meet the stated eligibility criteria (Table 1). In both cases, other trials were not included in the meta-analyses despite having similar design to the included trials that appeared ineligible.
Table 2 shows the quality assessment of each meta-analysis. Generally, the meta-analyses were of high quality, although all meta-analyses did not report some of the AMSTAR items, and some of the methods used in 3 meta-analyses [ 19,21,22] were of uncertain validity. Reporting of AMSTAR items was less common in the 2 patient-level meta-analyses [20,22].
Tables 3 and 4 show data from each trial used in each meta-analysis for hip fracture and total fracture, respectively. For hip fracture, the relative risk differed between meta-analyses for 6 of 12 trials for which data were reported in two or more meta-analyses, and for total fracture, the relative risk differed between metaanalyses in 10 of 15 trials. Tables 3 and 4 show the reasons for the differences in relative risks between the meta-analyses, which are summarised in Table 5. Many of the differences arose when results obtained using analyses other than the recommend approach (unadjusted intention-to-treat analysis of all participants with data from the final study timepoint) were used in a meta-analysis, with the most common example being the use of a per-protocol analysis. The majority of the differences (4 of 6 for hip fracture, 7 of 10 for total fracture) led to more favourable estimates of the efficacy of vitamin D on hip or total fracture being used for individual trials than if the recommended approach was applied. In general, the Cochrane review [18] was most likely to use the recommended approach, and the relative risks used for each study in that review are the most conservative.
The differences between meta-analyses in relative risks for individual trials were most prominent for the total or non-vertebral fracture endpoint. Non-vertebral fracture has commonly been reported in trials, but is often used interchangeably with total fracture. Some meta-analyses adopted this approach [17], whereas others carried out separate analyses for total fracture and for non-vertebral fracture [18]. In several meta-analyses, there were inconsistent approaches to handling data (Table 3). Participants with hip fracture were added to those with all non-vertebral fracture for 2 trials [25,31] in one meta-analysis [17], and for one of these two trials [31] in two meta-analyses [16,19], effectively counting hip fractures twice. In 2 meta-analyses [16,19], the primary endpoint was non- vertebral fracture but, when this endpoint was not reported, the authors used different estimates of non-vertebral fractures for different individual trials. Thus Table 4 shows that for different trials the authors used total fracture; total fracture

Data for hip fracture were not analysed in Tang 2007 [17], and individual trial data for hip fracture were not reported in DIPART 2010 [20], or Bischoff-Ferrari 2012 [22], No data on hip fractures were reported in any meta-analysis for Komulainen 1998 [29], Larsen 2004 [37], Flicker 2005 [38], Bolton-Smith 2007 [43], Prince 2008 [46], Pfeifer 2009 [47], Bischoff- Ferrari 2010 [48], Salovaara 2010 [49], and Sanders 2010 [50],
Study excluded- study met meta-analysis exclusion criteria. Study not included- study did not meet exclusion criteria but not included in meta-analysis. Data not included- data not included in meta-analysis but available from other meta-analyses or primary publication.
Abbreviations: Vit D- vitamin D, RR- relative risk, Cl confidence interval. NS- not stated. Vit D- vitamin D. CaD- co-administered calcium and vitamin D.
Bold text- indicates differences in relative risks for individual studies between meta-analyses. afactorial/multi-arm studies permitting multiple comparisons between randomised groups.

Reasons for differences in reported data between meta-analyses:

b data after 36 month from Chapuy 1994;
c per-protocol analysis (all randomised participants not included);
d data after 18 month from Chapuy 1992;
e original paper reports different numbers for placebo group, correct numbers confirmed by original authors in Avenell 2009;
f age-adjusted relative risk;
g data for total fracture not hip fracture;
h adjustments to the number of participants with outcomes and denominators were made using an intraclass correlation coefficient of 0.026 to account for cluster randomisation; 'subgroup of participants only.

Data are for total fracture unless otherwise indicated. Individual trial data for fracture were not reported in DIPART 2010 [20], or Bischoff-Ferrari 2012 [22], No data on total or non- vertebral fracture were reported in any meta-analysis for Bischoff 2003 [33], and Bischoff-Ferrari 2010 [48],
Study excluded- study met meta-analysis exclusion criteria. Study not included- study did not meet exclusion criteria but not included in meta-analysis. Data not included- data not included in meta-analysis but available from other meta-analyses or primary publication.
Abbreviations: Vit D- vitamin D, RR- relative risk, Cl confidence interval. NS- not stated. Vit D- vitamin D. CaD- co-administered calcium and vitamin D.
Bold text- indicates differences in relative risks for individual studies between meta-analyses. afactorial/multi-arm studies permitting multiple comparisons between randomised groups.

Reasons for differences in reported data between meta-analyses:

b non-vertebral fracture,
c data after 36 month from Chapuy 1994;
d per-protocol analysis (all randomised participants not included);
e data after 18 month from Chapuy 1992;
f hip fractures plus all non-vertebral fractures;
9 original paper reports different numbers for placebo group, correct numbers confirmed by original authors in Avenell 2009; hhip/wrist/forearm fracture used for non-vertebral fractures;
'unknown how calculated;
Excluded participants receiving intramuscular vitamin D;
includes participants receiving intramuscular vitamin D;
'osteoporotic fracture;
mfracture numbers calculated indirectly from Table 3 in meta-analysis;
"low trauma fractures;
°low trauma non-vertebral fractures;
Participants with vertebral fracture subtracted from participants with low trauma fractures;
‘‘adjustments to the number of participants with outcomes and denominators were made using an intraclass correlation coefficient of 0.026 to account for cluster randomisation;
"subgroup of participants only;
ship fractures used for non-vertebral fractures;
‘participants with vertebral fractures subtracted from participants with all fractures;
"relative risk only reported for two subgroups,
v total numbers of fractures (not numbers of participants with fracture).

Table 5 Reasons for differences in results between meta-analyses,

and effects on estimate of efficacy of vitamin D on fracture

minus spine fracture; low trauma fracture minus spine fracture; hip, wrist, or forearm fracture; hip fracture; and counted hip fractures twice. For the 2 trials where only subsets of fractures were used (hip fracture, or hip/wrist/forearm fracture), data on total fractures were available. One meta-analysis utilised one fracture endpoint for each study determined hierarchically in descending order from total fracture, hip fracture, and non-vertebral fracture [21]. For the resulting meta-analysis, total fracture was used for 10 trials, hip fracture for 4 trials, and non-vertebral fracture for 2 trials. Hip fracture is only a small subset of total fracture, and for all 4 trials where hip fracture was used, data on the broader endpoint of non-vertebral fracture were used in other meta-analyses.

Table 6 shows the conclusions from the meta-analysis.

Some of the conclusions differ substantially. For example, in 3 meta-analyses Bischoff-Ferrari and colleagues conclude that higher doses but not lower doses of vitamin D prevent fractures [16,19,22], whereas 3 other meta-analyses concluded that vitamin D, used without calcium supplements, does not prevent fractures, regardless of the dose [18,20,21]. Table 6 shows that our assessment of the strength of the conclusions in favour of vitamin D supplements ranged from mixed to strongly positive, with a median score of $4 for 6 of the 7 meta-analyses. The metaanalysis that most closely adhered to the recommended approach to analysis of a randomised controlled trial and fulfilled the most number of items in the AMSTAR tool had the least positive conclusion [18].

Discussion

Among overlapping meta-analyses of vitamin D and fracture, there were substantial differences in the trials included, the data used from each trial, the analytical approach adopted, and the conclusions drawn, despite the metaanalyses being of high quality and published in the highest ranking medical journals. Only 4 meta-analyses included all eligible trials, with the number of ‘‘missed’’ trials ranging from 3 to 8 in the other 3 meta-analyses. Two metaanalyses included trials that did not appear to meet eligibility criteria, while excluding other trials of similar design. The relative risks used for individual trials varied between meta-analyses, with differences being more common for total fracture (67% of trials) than for hip fracture (50% of trials). The differences in relative risks led to more favourable estimates of the efficacy of vitamin D compared to analyses using recommended analytic approaches on 11/16 (69%) occasions. The conclusions of the meta-analyses were discordant, ranging from strong statements that vitamin D prevents fractures to equally strong statements that vitamin D used without calcium does not prevent fractures. All meta-analyses were favourable toward prescribing of vitamin D for fracture prevention, although in some meta-analyses the recommendations were restricted to certain subgroups, or to co-administration of vitamin D with calcium supplements.

The reasons for the differences between the meta-analyses for the trials included and the data used (Table 5) were often not readily apparent. An explanation as to why trials were not included was provided in only one meta-analysis. Fracture data for 3 of the trials included in the Cochrane review were unpublished and obtained for that review [18]. It is not clear whether the authors of other metaanalyses sought these unpublished data, or why, once published in the Cochrane review, they were not included in later meta-analyses. On a similar note, the Cochrane review authors clarified ambiguous reporting of treatment group numbers in the primary publication for one study with the lead author of the study [31] (Table 3/4), whereas in the other meta-analyses, incorrect denominators for both treatment groups were used.
The reasons for the differences in relative risks between meta-analyses can only be deduced by detailed, careful examination of the meta-analyses and the primary publications. One trial-level meta-analysis did not report the number of participants with fracture or the number of participants in each treatment group for individual trials [17] and neither patient-level analysis [20,22] reported these data or relative risks for individual trials. The absence of this information limits verification of the accuracy of the data and analyses undertaken. For patient-level analyses where data is censored at an earlier timepoint [20], this information is very important because the results at the earlier time point may differ from those of the overall trial. For example, in a patient-level meta-analysis of vitamin D and mortality [51], data for the Women’s Health Initiative trial [41] was censored at 3 years, restricting the number of deaths to about 25% of those occurring in the trial, and providing a more favourable effect estimate than that for the entire follow-up. Several meta-analyses used data from per-protocol analyses for individual studies. The recommended analysis for a randomised controlled trial is an unadjusted intention-to-treat analysis including all available data from all randomised participants [24]. We think the same principle applies in metaanalyses of randomised controlled trials. When only per-protocol data are reported for trials, the Cochrane handbook recommends performing sensitivity analyses to explore differences between intention-to-treat approaches (that assume participants lost to follow-up did not have an event), with results using per-protocol data [52]. None of the meta-analyses performed such sensitivity analyses. None of the meta-analyses provided sufficient details to permit a reader to understand if data from individual trials could be incorporated in the metaanalysis in different ways (such as using all fractures versus using only low-trauma fractures), and none compared their handling of the data with previous metaanalyses.
The methodological differences between meta-analyses influenced the conclusions drawn from them. Each of the 3 meta-analyses that considered trials of vitamin D with calcium supplements separately to trials of vitamin D [18,20,21] concluded that vitamin D alone does not prevent fractures, regardless of dose. However, the 3 meta-analyses by Bischoff-Ferrari and colleagues that assessed vitamin D with or without calcium concluded that higher doses of vitamin D prevent fractures [16,19,22]. We have several reservations about the conclusions of these 3 meta-analyses. As highlighted in Tables 3 and 4, these meta-analyses used more favourable effect estimates for vitamin D for individual trials than estimates obtained using unadjusted intention-to-treat analysis of all participants with data from the final study timepoint. Most trials categorised as high dose vitamin D studied co-administered calcium and vitamin D, but the benefits were attributed to vitamin D. Three large trials of high dose vitamin D [40,45,50] were excluded or only included in secondary analyses because of their study design, but all had relative risks for fracture with vitamin D greater than 1, essentially excluding clinically significant benefits on fracture prevention for vitamin D (Tables 1,3-4). Finally, 2 of these meta-analyses made questionable assumptions about received vitamin D doses and focused on treatment adherence analyses [19,22], methodology that has been criticised [53]. In our view, the Cochrane review [18] is the most detailed and comprehensive, receives the highest rating using the AMSTAR tool for quality assessment, includes the broadest range of trials, and utilises the recommended intention-to-treat approach with the most conservative effect estimates. We think the meta-analyses in the Cochrane review are the most reliable with the greatest external validity- i.e. the results are most generalisable to the wider population.
We followed the approach recommended in 1997 for assessment of discordance amongst systematic reviews [9]. The widespread use of checklists for reporting meta-analyses, such as the PRISMA checklist [54], means that later meta-analyses have become more standardised and of higher quality. This is reflected in Table 2 which shows the high level of reporting of AMSTAR items used to assess metaanalysis quality. However, despite the apparent high quality of these metaanalyses, there were important differences in 3 of the domains that give rise to discordant meta-analyses [9]:

• study selection and inclusion,
• data extraction, and
• statistical methods.

The differences are not readily apparent unless each metaanalysis is scrutinised in considerable detail. Thus, it is very likely that the casual reader, and even an expert reviewer, will not notice the differences or understand why the results of overlapping meta-analyses differ. Many of the methodological differences are based on subjective decisions made by the authors. Since different researchers make different judgements on these methodological issues, their decisions and the reasoning behind the decisions should be reported. In Table 7, we propose additions to guidelines for the reporting of overlapping meta-analyses to facilitate their interpretation. They may also decrease redundant overlapping meta-analyses [3], by requiring authors to clearly identify previous publications and make apparent what the new meta-analyses adds to existing knowledge. An important limitation of our analysis is that it is limited to a single topic and a sample of meta-analyses published in high-impact journals, but it seems likely that the weaknesses in methodology and reporting we found will be present in other overlapping meta-analyses.
There are important clinical consequences arising from discordant conclusions from overlapping meta-analyses. They engender confusion among clinicians and patients, and foster public disenchantment with biomedical research, exemplified by the statement often used in the media that ‘‘the experts can’t make up their minds’’.

Table 7. Suggestions for improved reporting of overlapping meta-analyses.

Abstract	State article is an overlapping meta-analysis
	State goal of current meta-analysis
Introduction	Report number of previous meta-analyses on topic
	Summarise conclusions of previous meta-analyses
	State goal of current-meta-analysis
Methods	Reference all previous meta-analyses on topic
	List all relevant studies identified in literature search not included in current meta-analysis, and reasons for exclusion
	State which studies included in previous meta-analyses are excluded from current meta-analysis, and reasons for exclusion
	State which studies included in current meta-analysis have not been included in previous meta-analyses, and reasons for inclusion
	State where data for individual studies in current meta-analysis differ from those used in previous meta-analyses, and reasons for differences
	For patient-level analyses, if data have been censored at an early time point, state whether estimates of effect size differ from estimates at final time point
Results	Data for numbers of events/participants reported (including for patient-level analyses)
Discussion	Discuss conclusions of previous meta-analyses
	Discuss what current meta-analysis adds to existing body of literature
	If conclusions of current meta-analysis differ from previous meta-analyses, state reasons for differences
Registration	Mandatory registration of meta-analysis protocol, including statistical analysis plan

Another specific possibility is that patients taking vitamin D supplements in the hope of preventing fractures might be falsely reassured that they are improving their skeletal health by the reporting of positive meta-analyses with methodological weaknesses or limited generalisability.
In summary, this detailed review reveals substantial differences between overlapping meta-analyses of vitamin D and fracture published in the highest ranking general medical journals, despite all meta-analyses generally being assessed as high quality using the AMSTAR tool. The reasons for these differences were often not readily apparent, but the differences led to more favourable estimates of the efficacy of vitamin D compared to estimates obtained using recommend analytic approaches. From this specific example, it is possible to propose additional guidelines for reporting meta-analyses, in order to create greater accuracy and transparency, especially amongst overlapping meta-analyses that report discordant results.

Supporting Information
S1 Appendix. List of 24 meta-analyses or systematic reviews of vitamin D and fracture identified in October 2013 search of PubMed using the terms vitamin D; fracture or osteoporosis; systematic review or meta-analysis. doi:10.1371/journal.pone.0115934.s001 (DOCX)
Author Contributions
Conceived and designed the experiments: MB AG. Analyzed the data: MB AG. Contributed reagents/materials/analysis tools: MB AG. Wrote the paper: MB AG.

References

1. Bolland MJ, Grey A, Reid IR (2007) The randomised controlled trial to meta-analysis ratio: original data versus systematic reviews in the medical literature. N Z Med J 120: U2804.
2. Bastian H, Glasziou P, Chalmers I (2010) Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med 7: e1000326.
3. Siontis KC, Hernandez-Boussard T, Ioannidis JP (2013) Overlapping meta-analyses on the same topic: survey of published studies. BMJ 347: f4501.
4. Cook DJ, Reeve BK, Guyatt GH, Heyland DK, Griffith LE, et al. (1996) Stress ulcer prophylaxis in critically ill patients. Resolving discordant meta-analyses. JAMA 275: 308-314.
5. Campbell J, Bellamy N, Gee T (2007) Differences between systematic reviews/meta-analyses of hyaluronic acid/hyaluronan/hylan in osteoarthritis of the knee. Osteoarthritis Cartilage 15: 1424-1436.
6. Poolman RW, Abouali JA, Conter HJ, Bhandari M (2007) Overlapping systematic reviews of anterior cruciate ligament reconstruction comparing hamstring autograft with bone-patellar tendon-bone autograft: why are they different? J Bone Joint Surg Am 89: 1542-1552.
7. Hernandez AV, Walker E, Ioannidis JP, Kattan MW (2008) Challenges in meta-analysis of randomized clinical trials for rare harmful cardiovascular events: the case of rosiglitazone. Am Heart J 156: 23-30.
8. Thorlund K, Druyts E, Avina-Zubieta JA, Wu P, Mills EJ (2013) Why the findings of published multiple treatment comparison meta-analyses of biologic treatments for rheumatoid arthritis are different: an overview of recurrent methodological shortcomings. Ann Rheum Dis 72: 1524-1535.
9. Jadad AR, Cook DJ, Browman GP (1997) A guide to interpreting discordant systematic reviews. CMAJ 156: 1411-1416.
10. Bolland MJ, Grey A, Reid IR (2009) Time for a moratorium on vitamin D meta-analyses? BMJ 339: b4394.
11. Grey A, Bolland MJ, Reid IR (2012) Vitamin D dose requirements for fracture prevention. N Engl J Med 367: 1367-1368; author reply 1369-1370.
12. IOM (Institute of Medicine) (2011) Dietary Reference Intakes for Calcium and Vitamin D. Washington, DC: The National Academies Press.
13. Holick MF, Binkley NC, Bischoff-Ferrari HA, Gordon CM, Hanley DA, et al. (2011) Evaluation, treatment, and prevention of vitamin D deficiency: an Endocrine Society clinical practice guideline. J Clin Endocrinol Metab 96: 1911-1930.
14. Rosen CJ, Abrams SA, Aloia JF, Brannon PM, Clinton SK, et al. (2012) IOM committee members respond to Endocrine Society vitamin D guideline. J Clin Endocrinol Metab 97: 1146-1152.
15. Holick MF, Binkley NC, Bischoff-Ferrari HA, Gordon CM, Hanley DA, et al. (2012) Guidelines for preventing and treating vitamin D deficiency and insufficiency revisited. J Clin Endocrinol Metab 97: 1153-1158.
16. Bischoff-Ferrari HA, Willett WC, Wong JB, Giovannucci E, Dietrich T, et al. (2005) Fracture prevention with vitamin D supplementation: a meta-analysis of randomized controlled trials. JAMA 293: 2257-2264.
17. Tang BMP, Eslick GD, Nowson C, Smith C, Bensoussan A (2007) Use of calcium or calcium in combination with vitamin D supplementation to prevent fractures and bone loss in people aged 50 years and older: a meta-analysis. Lancet 370: 657-666.
18. Avenell A, Gillespie WJ, Gillespie LD, O’Connell D (2009) Vitamin D and vitamin D analogues for preventing fractures associated with involutional and post-menopausal osteoporosis. Cochrane Database Syst Rev: CD000227.
19. Bischoff-Ferrari HA, Willett WC, Wong JB, Stuck AE, Staehelin HB, et al. (2009) Prevention of nonvertebral fractures with oral vitamin D and dose dependency: a meta-analysis of randomized controlled trials. Arch Intern Med 169: 551-561.
20. DIPART (Vitamin D Individual Patient Analysis of Randomized Trials) Group (2010) Patient level pooled analysis of 68 500 patients from seven major vitamin D fracture trials in US and Europe. BMJ 340: b5463.
21. Chung M, Lee J, Terasawa T, Lau J, Trikalinos TA (2011) Vitamin D With or Without Calcium Supplementation for Prevention of Cancer and Fractures: An Updated Meta-analysis for the U.S. Preventive Services Task Force. Ann Intern Med 155: 827-838.
22. Bischoff-Ferrari HA, Willett WC, Orav EJ, Lips P, Meunier PJ, et al. (2012) A pooled analysis of vitamin D dose requirements for fracture prevention. N Engl J Med 367: 40-49.
23. Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, et al. (2007) Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol 7: 10.
24. ICH E9 (1998) Note for Guidance on Statistical Principles for Clinical Trials. CPMP/ICH/363/96.
25. Chapuy MC, Arlot ME, Duboeuf F, Brun J, Crouzet B, et al. (1992) Vitamin D3 and calcium to prevent hip fractures in the elderly women. N Engl J Med 327: 1637-1642.
26. Chapuy MC, Arlot ME, Delmas PD, Meunier PJ (1994) Effect of calcium and cholecalciferol treatment for three years on hip fractures in elderly women. BMJ 308: 1081-1082.
27. Lips P, Graafmans WC, Ooms ME, Bezemer PD, Bouter LM (1996) Vitamin D supplementation and fracture incidence in elderly persons. A randomized, placebo-controlled clinical trial. Ann Intern Med 124: 400-406.
28. Dawson-Hughes B, Harris SS, Krall EA, Dallal GE (1997) Effect of calcium and vitamin D supplementation on bone density in men and women 65 years of age or older. N Engl J Med 337: 670676.
29. Komulainen MH, Kroger H, Tuppurainen MT, Heikkinen AM, Alhava E, et al. (1998) HRTand Vit D in prevention of non-vertebral fractures in postmenopausal women; a 5 year randomized trial. Maturitas 31: 45-54.
30. Pfeifer M, Begerow B, Minne HW, Abrams C, Nachtigall D, et al. (2000) Effects of a short-term vitamin D and calcium supplementation on body sway and secondary hyperparathyroidism in elderly women.
31. J Bone Miner Res 15: 1113-1118.
32. Chapuy MC, Pamphile R, Paris E, Kempf C, Schlichting M, et al. (2002) Combined calcium and vitamin D3 supplementation in elderly women: confirmation of reversal of secondary hyperparathyroidism and hip fracture risk: the Decalyos II study. Osteoporos Int 13: 257-264.
33. Meyer HE, Smedshaug GB, Kvaavik E, Falch JA, Tverdal A, et al. (2002) Can vitamin D supplementation reduce the risk of fracture in the elderly? A randomized controlled trial. J Bone Miner Res 17: 709-715.
34. Bischoff HA, Stahelin HB, Dick W, Akos R, Knecht M, et al. (2003) Effects of vitamin D and calcium supplementation on falls: a randomized controlled trial. J Bone Miner Res 18: 343-351.
35. Trivedi DP, Doll R, Khaw KT (2003) Effect of four monthly oral vitamin D3 (cholecalciferol) supplementation on fractures and mortality in men and women living in the community: randomised double blind controlled trial. BMJ 326: 469.
36. Avenell A, Grant AM, McGee M, McPherson G, Campbell MK, et al. (2004) The effects of an open design on trial participant recruitment, compliance and retention-a randomized controlled trial comparison with a blinded, placebo-controlled design. Clin Trials 1: 490-498.
37. Harwood RH, Sahota O, Gaynor K, Masud T, Hosking DJ (2004) A randomised, controlled comparison of different calcium and vitamin D supplementation regimens in elderly women after hip fracture: The Nottingham Neck of Femur (NONOF) Study. Age Ageing 33: 45-51.
38. Larsen ER, Mosekilde L, Foldspang A (2004) Vitamin D and calcium supplementation prevents osteoporotic fractures in elderly community dwelling residents: a pragmatic population-based 3-year intervention study. J Bone Miner Res 19: 370-378.
39. Flicker L, MacInnis RJ, Stein MS, Scherer SC, Mead KE, et al. (2005) Should older people in residential care receive vitamin D to prevent falls? Results of a randomized trial. J Am Geriatr Soc 53: 1881-1888.
40. Grant AM, Avenell A, Campbell MK, McDonald AM, MacLennan GS, et al. (2005) Oral vitamin D3 and calcium for secondary prevention of low-trauma fractures in elderly people (Randomised Evaluation of Calcium Or vitamin D, RECORD): a randomised placebo-controlled trial. Lancet 365: 1621-1628.
41. Law M, Withers H, Morris J, Anderson F (2006) Vitamin D supplementation and the prevention of fractures and falls: results of a randomised trial in elderly people in residential accommodation. Age Ageing 35: 482-486.
42. Jackson RD, LaCroix AZ, Gass M, Wallace RB, Robbins J, et al. (2006) Calcium plus vitamin D supplementation and the risk of fractures. N Engl J Med 354: 669-683.
43. Porthouse J, Cockayne S, King C, Saxon L, Steele E, et al. (2005) Randomised controlled trial of calcium and supplementation with cholecalciferol (vitamin D3) for prevention of fractures in primary care. BMJ 330: 1003.
44. Bolton-Smith C, McMurdo ME, Paterson CR, Mole PA, Harvey JM, et al. (2007) Two-year randomized controlled trial of vitamin K1 (phylloquinone) and vitamin D3 plus calcium on the bone health of older women. J Bone Miner Res 22: 509-519.
45. Lyons RA, Johansen A, Brophy S, Newcombe RG, Phillips CJ, et al. (2007) Preventing fractures among older people living in institutional care: a pragmatic randomised double blind placebo controlled trial of vitamin D supplementation. Osteoporos Int 18: 811-818.
46. Smith H, Anderson F, Raphael H, Maslin P, Crozier S, et al. (2007) Effect of annual intramuscular vitamin D on fracture risk in elderly men and women-a population-based, randomized, double-blind, placebo-controlled trial. Rheumatology (Oxford) 46: 1852-1857.
47. Prince RL, Austin N, Devine A, Dick IM, Bruce D, et al. (2008) Effects of ergocalciferol added to calcium on the risk of falls in elderly high-risk women. Arch Intern Med 168: 103-108.
48. Pfeifer M, Begerow B, Minne HW, Suppan K, Fahrleitner-Pammer A, et al. (2009) Effects of a longterm vitamin D and calcium supplementation on falls and parameters of muscle function in communitydwelling older individuals. Osteoporos Int 20: 315-322.
49. Bischoff-Ferrari HA, Dawson-Hughes B, Platz A, Orav EJ, Stahelin HB, et al. (2010) Effect of high- dosage cholecalciferol and extended physiotherapy on complications after hip fracture: a randomized controlled trial. Arch Intern Med 170: 813-820.
50. Salovaara K, Tuppurainen M, Karkkainen M, Rikkonen T, Sandini L, et al. (2010) Effect of vitamin D(3) and calcium on fracture risk in 65- to 71-year-old women: a population-based 3-year randomized, controlled trial-the OSTPRE-FPS. J Bone Miner Res 25: 1487-1495.
51. Sanders KM, Stuart AL, Williamson EJ, Simpson JA, Kotowicz MA, et al. (2010) Annual high-dose oral vitamin D and falls and fractures in older women: a randomized controlled trial. JAMA 303: 18151822.
52. Rejnmark L, Avenell A, Masud T, Anderson F, Meyer HE, et al. (2012) Vitamin D with calcium reduces mortality: patient level pooled analysis of 70,528 patients from eight major vitamin D trials. J Clin Endocrinol Metab 97: 2670-2681.
53. Higgins JPT, Green S (editors) (2011) Cochrane Handbook for Systematic Reviews of Interventions, version 5.1.0 [updated March 2011]: The Cochrane Collaboration. Available www.cochrane-handbook. org.
54. Abrahamsen B, Avenell A, Bolland M, Rejnmark L, MacLennan GS, et al. (2013) A pooled analysis of Vitamin D dose requirements for fracture prevention. IBMS BoneKEy 10: 10.1038/bonekey.2012.1256.
55. Moher D, Liberati A, Tetzlaff J, Altman DG (2009) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ 339: b2535.