Understanding Cut-Point Analysis for ADA Assays

What are the most important properties of a new drug? Ultimately, we want to ensure that any new therapy treats what it is designed to treat—that it’s effective—and that the patient suffers as few negative consequences as possible while doing it—that it’s safe. These two criteria often boil down to what the drug does to the body—is the drug toxic, does it target the correct protein, etc—but what can be overlooked is the effect the body has on a drug.

Protein-based therapeutics, in particular, can face a staunch challenge from the immune system. The immune response to a therapy can have dramatic consequences for its performance, such as preventing it having its desired effect, as well as resulting in serious adverse effects for the patient. Testing how much of an immune response a new therapy elicits—its immunogenicity—is, therefore, an important step on the road to understanding drug safety and efficacy.

Let’s take this back a step. What does the immune system do to an invader? One of the main avenues of attack is through antibodies, which are proteins specifically designed to attach to foreign objects. All biological material—bacteria, viruses, transplanted organs, etc—have proteins on their surface known as antigens. Your cells have antigens too—that’s how your immune system (usually) knows to leave them alone—but if something enters your body with unexpected antigens, your immune system swings into action.

For example, if, say, the SARS-CoV-2 virus enters your body, it is identified as being foreign because of its antigens. Your immune system then produces antibodies which are shaped specifically to attach to the antigens on the surface of the virus. This stops the virus invading your cells, which, in turn, prevents it from multiplying and means you don’t contract the disease it causes: Covid-19.

The same thing happens when a protein-based therapeutic enters your body. Your immune system doesn’t know whether the stranger at the castle gates is a deadly virus or a miracle cure. It treats the therapeutic as an unknown antigen—and therefore a threat—producing antibodies which stick to the protein. These are Anti-Drug Antibodies (ADAs).

This can be a big problem. When ADAs stick to the therapeutic protein, they can change its shape or block the site where it interacts with its target. This reduces the effectiveness of the therapeutic, as the intended interaction with the target protein is made significantly less likely to occur or even impossible. Such an immune response to a therapeutic drug can also cause fever, inflammation, and tiredness—among other adverse effects—just the same as the immune response to a virus.

And it gets worse. After defeating an invader, the immune system adds its antigen to its catalogue of previous foes so it can react quicker the next time that enemy is encountered. Sometimes this a good thing: it’s why vaccines work and why we gain immunity to some diseases after getting them. For therapeutics, however, its bad news. If the drug is given repeatedly, it means that the immune response will be quicker and stronger each time it is administered. The effectiveness of the therapeutic will be reduced and the patient may face worsening adverse effects.

ADAs, therefore, are bad for both the drug and the patient. They reduce the effectiveness of therapeutics, and can result in unpleasant and occasionally serious side effects. So it is important to have a good understanding of the immunogenicity of a therapeutic in development. The potential for immunogenicity also informs the cost/benefit analysis which goes into the licensing and business decisions of any new therapeutic.

So, how do we test a drug’s immunogenicity? In a typical ADA detection assay performed during a preclinical assessment, blood samples are taken from animals, which are then tested (often in a “ligand binding assay”) to determine whether the drug elicits an immune response.  Of course, use of animals has ethical problems, and the animal immune response may not be the same as a human immune response – but let’s leave these issues aside for now.

As with any experiment, we have to decide what we’re going to count as a positive test sample for ADAs.

This is a classic problem of noisy data: a common bane of bioassay. By chance, there may be test samples which are actually negative which happen to look a lot like a weak positive result, and test samples which should be classified as positive which look like negative results. Remember that a positive ADA test is bad – we don’t want our therapeutic to generate ADAs!

Of course, we want to minimise the frequency of these false positive and false negative ADA results. Many false positives could lead to a perfectly good therapeutic failing to make it to market, or being prescribed alongside an unnecessary cocktail of immunosuppressants and anti-inflammatories. More concerningly, false negatives could lead to unexpected adverse reactions in patients given the new drug, as they could make the immune response seem weaker than it actually is. But how do you decide?

The signal strength above which a result is considered positive, and below which a result is considered negative is called the cut-point. Returning to our Covid-19 example, such a decision is inherent in the ubiquitous lateral flow tests. In their design, someone decided the minimum amount—the cut-point—of SARS-CoV-2 virus which should be present in a sample in order for that dreaded second pink line to appear.

Whilst a single cut-point could be used, it is actually better to use 2 cut-points in the case of an ADA assay. The objective here is to identify the samples which contain ADAs: we want to screen out samples until all we’re left with are truly positive samples, as far as is possible.

The first phase is a screening test cut-point. It is designed to all the samples where it is “beyond reasonable doubt” that the sample is negative. Some could-be positive or could-be negative samples will not be screened out at this point.

The second, confirmatory, test is generally more sensitive. The aim here is to try to determine if any of the could-be positive-but-might-be negative samples can actually be safely classed as negative. You could just skip the first phase and put every sample through the second test, but in general it is more expensive and time consuming. Using the screening test first is thus a sensible approach.

Unfortunately, the process to actually calculate what the cut-points should be is far from trivial, especially with noisy biological data. It’s not as simple as just picking a number: an involved statistical process including information about assay sensitivity, precision, and more must take place. Sometimes the cut-points require corrections for individual analysts or pieces of equipment, since such factors could have small effects on the assay which would alter the crucial balance between false and true negatives.

This traditional cumbersome statistical approach has now been improved. Devanarayan et al (2017) have developed a more streamlined approach to cut-point analyses that not only reduces complexity for the statistician performing the analysis but, perhaps more importantly, produces results which are far easier for the end user to implement. Variability caused by different analysts or equipment is taken into account at the start of the cut-point calculation, meaning time-of-use corrections to the cut-point due to these factors are no longer necessary. This dramatically reduces the statistical complexity of the assay: data can be compared against a single number to determine the result of the test. Compared to pulling out a table of cut-point corrections for every measurement, this certainly provides a significant time saving. The ADA team at Quantics use this new methodology.

ADA assays form a crucial part of understanding a new therapeutic’s safety, efficacy and thus the potential benefit to patients. A good knowledge of the immunogenicity of a therapeutic could be the difference between a product which fails clinical trials and one which forms part of a safe, well-defined treatment plan for patients.

You can find more information about our cut-point analysis services here. If you have any questions about what Quantics can do for your ADA assay—or for any other bioassay—please don’t hesitate to get in contact.