In this blog we will discuss a particular type of clinical trial design, the non-inferiority trial. We will explain the difference between non-inferiority trials and more typical clinical trial designs to help build understanding of the non-inferiority limit.
Standard clinical trials usually aim to show that a new treatment is better than an existing one. Often this is done by measuring some outcome (say survival time – read our survival blog here) for two groups of patients, each of which has been given one of the treatments. At the end of the trial, the difference in the outcomes between the two treatments is calculated; if this is greater than zero, and the difference is statistically significant, we can conclude that the new treatment is better than the existing one. (We are assuming throughout that a high value for the outcome is better than a low one.) This is called a superiority approach, since the aim is to show that one treatment is superior to the other.
However, superiority trials are not always feasible. If the difference between the treatments is believed to be very small, the sample size required to show superiority will be huge. In some situations, the new treatment may not be superior to the existing treatment at all – if so it will be impossible to show it is superior, however large the sample size.
Key Takeaways
- Non-Inferiority trials aim to demonstrate that a new treatment is not worse than an existing one by more than a pre-defined margin, making them useful when treatments are expected to have similar effects but the new option offers other benefits (e.g. lower cost or better safety).
- The non-inferiority margin defines the maximum acceptable difference in outcomes for the new treatment to still be considered “not inferior”; trial results are interpreted by comparing the confidence interval of the outcome difference to this margin.
- Setting the margin is critical and must be done in advance using clinical judgement and regulatory guidance, as it directly influences the trial’s conclusions and is subject to regulatory scrutiny by bodies such as the FDA and EMA.
The non-inferiority trial is a useful alternative in such situations. As the name suggests, the aim is to show that the new treatment is not inferior to the existing one – that is, it is either equally effective or better. If this can be established, the new treatment can be considered as a replacement for the existing treatment, especially if it has other advantages (e.g. cost, safety) which make it preferable.
To decide whether the new treatment is indeed “equally effective or better”, the difference between the treatments is compared to a non-inferiority margin. We will explain what this is in the following example.
Non-inferiority margin – an example
Let’s consider an example where the outcome is survival time, and suppose the two treatments are in fact equally effective – they both have a median survival time of 12 months. If the two treatments were compared in a study with a small sample size, the results might be something like those shown in the figure below:
In this example, the estimates of the median survival time are:
- Existing treatment: 11.2 months.
- New treatment: 11.8 months.
- Estimate of the difference in survival times: 11.8 – 11.2 = 0.6 months.
- 95% confidence interval (CI) for the difference: -5.0 to 6.2 months.
Would this establish that the new treatment is not inferior to the existing one? No – based on these results, the new treatment might reduce survival time by as much as 5 months compared to the existing treatment, which is clearly much worse.
On the other hand, with a study the results could be much more precise, as shown in this figure:
In this example, the estimates of the median survival time are:
- For existing treatment: 12.1 months.
- For new treatment: 12.0 months.
- Estimate of the difference in survival times: 12.0 – 12.1 = -0.1 months.
- 95% confidence interval (CI) for the difference: -0.3 to 0.1 months.
Even though we may assume that this difference is small enough to conclude non-inferiority, this is where the non-inferiority margin comes in. The non-inferiority margin is the maximum difference between the treatments for which the outcomes can be considered to be “equal”. In the current example, a difference of one month, say, might be the maximum that can be considered equal, so the non-inferiority margin would be one month. The aim of the study would then be to exclude the possibility that the survival time for patients who received the new treatment was more than one month lower than that of those who received the old treatment. Written more formally, the study aims to show that Survival timenew treatment – Survival timeold treatment ≥ -1.
Once the non-inferiority margin has been set, the CI for the difference is compared to the margin to decide the outcome of the trial. If the CI is entirely above the margin, the new treatment is non-inferior to the existing one.
How this works in various scenarios is shown in the figure below.
- Scenario A: The CI is completely above the non-inferiority margin, and also above zero – this means that both superiority and non-inferiority have been demonstrated.
- Scenario B: The CI is completely above the non-inferiority margin, but not above zero; non-inferiority has been demonstrated, but not superiority.
- Scenario C: Again, the CI is completely above the non-inferiority margin, so non-inferiority has been demonstrated. This is true even though the CI is entirely below zero – what matters is that it is above the non-inferiority margin.
- Scenario D: The CI extends below the non-inferiority margin, so non-inferiority has not been demonstrated.
- Scenario E: The CI is entirely below the non-inferiority margin, so again non-inferiority has not been demonstrated. In fact in this case the new treatment is clearly inferior to the existing one.
Setting the non-inferiority margin
Clearly the outcome of a non-inferiority trial depends critically on the non-inferiority margin. In the example above, if the non-inferiority margin had been, say, 2 months, non-inferiority would also have been demonstrated in scenario D. Given this importance, it is important to set the non-inferiority margin in advance and to do so in an objective way. Indeed, the ICH guidance (ref 1) requires that the non-inferiority margin is specified in the clinical trial protocol.
Setting a non-inferiority margin objectively can be done by thinking carefully about what it means for outcomes to be considered “equal”. This will usually require input from clinicians on what the smallest clinically meaningful difference in outcomes is – for any difference smaller than this, the treatments can be considered “equal”.
Both the FDA (ref 2) and the European Medicines Agency (ref 3) have published guidelines describing in detail methods for setting non-inferiority margins but the guidelines aren’t always straightforward so we will focus more on how to set these limits in our next blog.
References
1
ICH Harmonised Tripartite Guideline. Statistical Principles for Clinical Trials E9. September 1998.
2
Non-Inferiority Clinical Trials to Establish Effectiveness. Guidance for Industry. U.S. Department of Health and Human Services Food and Drug Administration, November 2016.
https://www.fda.gov/downloads/Drugs/Guidances/UCM202140.pdf
3
Guideline on the choice of the non-inferiority margin. Committee for medicinal products for human use, European Medicines Agency, 2005.
Comments are closed.