When do you stop an HIV prevention trials for futility? A primer for HIV prevention advocates

A PDF of this document is available here.

Large HIV prevention clinical trials are critical to find out if new experimental products will work to prevent HIV. These trials often involve large numbers of volunteers and are very expensive. Therefore, it is also important to monitor them carefully on an ongoing basis to ensure that they are being conducted ethically, and to check if there are interim findings that would suggest they be stopped early. This fact sheet looks at why trials are stopped early for “futility.” What does this mean, when is such a recommendation made, and how does it affect other ongoing trials?

About clinical trial review committees
Clinical trials are closely monitored and regulated by a variety of entities, including independent bodies that review the trial protocol and data on an ongoing basis to ensure that the trial is ethical and should continue. These often include a Data Safety Monitoring Board (DSMB) or an Independent Data Monitoring Committee (IDMC) made up of researchers, statisticians, ethicists, and community representatives, all of whom are completely independent of the trial being conducted. They make recommendations that may have an important impact on the clinical trials they monitor. It is important to recognize that such recommendations are a sign that the system of checks and balances that protects participants and researchers involved in trials is working. Regular review and prompt recommendations on the part of DSMBs are an essential part of this system. Please see our DSMB sheet at www.avac.org/dsmb for more information.

What does futility mean to clinical trials?
In everyday, non-scientific usage, the term futility describes an effort that is pointless or that serves no useful purpose. In terms of clinical trials, futility means that a trial will not be able to answer the question or questions it set out to explore. A finding of futility is one possible outcome of interim data review by an IDMC or DSMB. In the context of HIV prevention research, a DSMB can recommend a trial stop for “futility” when interim data analysis shows that the trial is unlikely to be able to answer the question(s) posed by the trial, even if safety data and ethical conduct is not being questioned. There are many reasons this could happen. These include:

• Enrollment rates and/or participant retention is not adequate enough to generate enough data in a timely manner to adequately evaluate the experimental product;
• Changes in the site or the study community make it impossible to conduct the study as it was originally designed; or
• Interim data review shows that there are similar numbers of HIV infections in the active and control arm and the estimated additional infections in the trial would not be able to determine the “statistically” true, if any, effectiveness of the experimental product.

In some cases, a DSMB may recommend a change or revision of an original study protocol, to allow for additional recruitment or enrollment, rather than recommending that the trial be stopped completely. Each trial is different and each possible recommendation that a DSMB/IDMC makes is based on the complete picture of the trial and on several factors – such as safety findings and profile, other trials in the field, trial conduct, level of enrollment. Each DSMB trial recommendation needs to be seen in the context of the trial, the community and the field to be better understood.

Understanding one type of futility finding
This example is designed to help explain the rationale for a futility finding when a DMSB or IDMC finds roughly equivalent numbers of infections in the active and placebo arms of a trial.

If you imagine a marathon race, we could ask if one runner could prove s/he is clearly faster than the rest. Let’s say we describe “clearly faster” as one runner able to finish at least 30 minutes ahead of the rest. In some ways a research study can be like a race – where we ask will the active drug be “clearly better” than placebo, where we assess “clearly better” by an effectiveness measure, let’s say 30% better than the placebo.

• At the beginning of the race and the trial, there’s still plenty of time for one runner (or one product) to prove s/he is clearly much faster than the rest. There is not enough data to know the answer and there is a chance to answer the question within this study or race.
• When the runners have gone through about 1/3 of the race, it would still be difficult to determine who will clearly be the fastest runner. Even if they were running close to each other, it would still be still possible for one of them to finish 30 minutes faster than the others as one of them could race ahead. If we look at a trial at this point, the IDMC may note that there was still a good chance for the study to answer the question of which product is clearly better. The IDMC would recommend the study continue as planned, if no worrisome safety issues or other serious problems.
• Close to the end of the race, if the racers are running close together there is very little chance that one runner can still prove he is clearly 30 minutes faster than the rest. Even if one of the runners sprinted ahead and went as fast as humanly possible, by the time the race was nearly done there was no chance for one to prove he could finish 30 minutes faster than another. This is where the comparison ends – a race will be allowed to finish, but not a trial.

Stopping early for futility does not mean there was a problem with how the study was conducted or a problem with the study products. The DSMB or IDMC looks carefully at conduct of trial and would have closed the trial earlier if they were concerned in the way the trial was being executed. Final analyses may give us more information for understanding why a trial ends early for futility.

What about other trials looking at similar products?
Another race with the same runners in a different setting might turn out differently. Many factors can affect who wins a race and whether the race can tell us who is the fastest runner overall.

Acknowledgement: We are grateful to Lisa Noguchi of the Microbicide Trials Network (MTN) for her presentation on futility in April 2011 that was the basis for this primer.

Please note that this document is a work in progress. Please feel free to send any questions or comments to avac@avac.org.