A huge number of clinical trials are poorly designed and as such the results they produce can be irrelevant to real end users.
Now researchers at the University of Aberdeen through collaboration with over 80 international researchers, clinicians and policymakers and leaders in this field have developed a tool which helps trial designers to create more useful trials.
The PRECIS-2 tool is an improved version of a tool devised by members of the same team and already being used by trial designers in the UK and is already being tested by the National Institute of Health in the US.
“Around 25,000 randomised trials are published every year around the world,” explains Professor Shaun Treweek from the University of Aberdeen’s Health Services Research Unit. “They are seen as the ‘gold standard’ test for proving a new drug or treatment is better than the existing one.
“The problem is that these trials are often fundamentally flawed from the outset because the parameters of the test are often too narrow. In order to make a ‘cleaner’ experiment, trial designers may exclude people with underlying health problems, or the elderly, or carry out all the tests in hospitals, rather than GP practices even though the treatment would be used mainly by GPs, for example.
“If the trial doesn’t include the same kind of people who will be receiving the treatment, or those who will be administrating it, then the results are likely to be largely irrelevant in practice. For example, a team in New Zealand looked at 17 major asthma trials in an international treatment guideline and found that of the people in their community receiving treatment for asthma – only around six per cent were eligible for any one of these trials.
“This doesn’t mean the drugs are harmful but it means something that may look extremely effective in a tightly controlled trial might not be nearly as effective in the real world, which is obviously a potential waste of money, time and resources.”
Professor Treweek says the shortcomings of many randomised trials have been recognised by the medical community for 50 years, but the same mistakes are repeated to this day because many of those around the world who design trials have not been properly trained in research methods.
Now, Professor Treweek and Kirsty Loudon at the University of Aberdeen and colleagues in Dundee and Canada have devised a new easy-to-use tool, which helps trial designers at every stage of the design process to ensure they are making design choices that match the information needs of the people they are designing the trial for, which generally means patients and health professionals.
“The PRECIS-2 tool helps trial teams work through key questions in a very explicit way,” says Professor Treweek. “At the end – the data you input produces an image. If your decisions are consistent, it looks like a ‘wheel’. If it’s a ‘bumpy’ wheel it means your decisions have been inconsistent. If it’s a wide wheel then it means you’ve designed a trial that is likely to be highly relevant to real world practice. A narrow wheel means they’ve designed a tightly controlled experiment which may be useful in the future but is unlikely to resonate with people in the real world. It’s a great way of identifying potential flaws in the design of a trial before any real money, time and effort has been spent on the trial itself.
“PRECIS-2 is an update of a tool we devised in 2009 which was highly cited and has just been included in a list of ‘useful papers’ by the UK National Institute for Health Research (NIHR) Randomised Trials Coordination (http://www.bmj.com/content/350/bmj.h2147/related) site. It is also being used in the US by the National Institute for Health in some pilot projects to assess grant proposals to see if the trials being proposed are likely to be useful to patients, health professionals and policymakers.
“The end goal is to see the use of this tool taught as standard ‘good practice’ for trial designers so it becomes a routine part of the process. Funders are starting to take notice of this and we hope many will consider using it to ensure that their money is being spent on trials that have relevant results and real impact.”