Methodological analysis of study designs
For scientific and ethical reasons, researchers must give careful consideration to the numbers of patients used in clinical trials. Researchers have now analyzed the procedures for sample size determination given in original protocols submitted to Ethics Committees. Their results have just appeared.
If patient-oriented clinical research is to provide the basis for modern healthcare, it must be well planned, fully transparent and relevant to real needs. Steps must be taken to ensure that clinical projects are logically structured and permit one to draw clear and informative conclusions from the results. Otherwise, clinical research is either irrelevant or misleading.
“Whether or not a given study can provide enough information to answer the question it intends to address largely depends on the number of patients recruited,” says Professor Ulrich Mansmann of the Institute for Medical Informatics, Biometry and Epidemiology (IBE) at LMU Munich. With the collaboration of the British Health Research Authority, he and his team recently carried out a comprehensive methodological review based on the analysis of 446 research protocols for randomized clinical trials in phases II, III and IV.
Evaluation of confidential protocols
For the first time, researchers were able – subject to strict rules to preserve confidentiality – to review and rigorously evaluate a representative set of unpublished research protocols that had been submitted to Ethics Committees in the UK during the year 2009. Tim Clark und Dr. Ursula Berger, who are members of Mansmann’s group, focused particularly on how each protocol dealt with the issue of how many patients needed to be enrolled in theplanned trial.
“If the sample size chosen is too small, the results obtained will be inconclusive. If it is larger than necessary, it entails needless exposure of subjects to a therapy whose risks are unclear, which is ethically problematic,” Mansmann explains. Thus, on both ethical and scientific grounds, both of these pitfalls should be avoided.
Statistical tools are available that enable sample sizes to be related to the probable informational value of a study, in terms of reliability of the results, effect of the intervention, variability within the study population and logistical aspects of the trial design. How the proposed sample sizes were actually justified was therefore the central point at issue in the new analysis.Whether or not the assumptions underlying sample size determinations were specified, and how they were validated, also served as indicators of the quality of study designs.
Sample sizes tend to be too large
Overall, the review found that the majority of the 446 protocols assessed did not contain enough information to allow the researchers accurately to recalculate the sample sizes proposed. The team also observed that studies sponsored by commercial firms listed the assumptions on which sample size determinations were based more often than projects submitted by non-commercial entities did. Conversely, studies from academic sources were more likely to supply justifications for the assumptions made. Overall, the new work reports a tendency to use larger sample sizes than would have been necessary on the basis of the assumptions given in the protocols examined.
The review of research protocols will make it possible to developinstruments that permit Ethics Committees to evaluate more objectively the methodological quality of clinical research protocols. "And improvements in the review process for clinical trials should also make it easier for applicants to come up with methodologically better study designs,” Mansmann concludes. (British Medical Journal 2013)