In Vitro vs In Vivo Potency Assays: Modern Vaccine Batch Release

Vaccines are among the most important technological developments in human history. From the earliest attempts to prevent smallpox by variolation to modern mRNA vaccines built without a virus needing to have entered the development lab, the vaccine has become a vital string in the bow of public health officials. As with all medicines, batches of vaccines must be thoroughly tested to ensure their quality, with safety and efficacy priorities. Historically, this batch release testing has been done using in vivo testing on laboratory animals. As analytical techniques have advanced, however, in vitro testing is swiftly becoming a preferred option. Here, we will examine some of the challenges of in vivo batch release testing, and discuss why in vitro techniques can be superior.

In the simplest possible terms, the potency of a drug product is the biological effect produced by a specific quantity of that product. If two different samples contain the same concentration of their active ingredient, then the one which elicits the greater response in an assay is of a higher potency. It is crucial that the potency of any drug product is controlled as this allows for consistent dosing for end users. This is most certainly the case for vaccines, for which the potency is considered a Critical Quality Attribute (CQA). CQAs are properties of a product which must remain within specified limits to ensure safety and efficacy, and should be tested for each batch of product before its release to public use.

For many small molecule drug products, the potency depends simply on the concentration of the active ingredient: the higher the concentration, the more potent the batch. Vaccines, however, rely on biological active ingredients – deactivated or attenuated viruses, for example – whose potency depends on several interacting factors alongside their concentration to give an overall effect.

More sophisticated testing methods are, therefore, required to ensure the potency of vaccine products remain within specifications for batch release. Specifically, a method of testing which mimics biological processes which occur when the vaccine enters the body is often necessary as this ensures the batch elicits a protective response as intended.

There are two main categories of such testing methods: in vivo (Latin, meaning “in the living”) assays, which involve the use of animal models, and in vitro (“in glass”) assays, which are techniques which test the properties of the vaccine outside of a living organism.

Let’s imagine we have been given a batch of vaccine, and we are assigned the task to assess whether it falls within specifications for potency. What are some ways we might go about performing this test?

A good first move is to make sure we are measuring our potency relative to that of a known reference standard. We have covered the details of why using relative potency is a good idea in a previous blog, but the short version is this: biological experiments are highly dependent on experimental conditions. This means that they are often highly variable, and it’s difficult to ascertain the effect of any one factor on the measured results. By comparing a measurement taken for a test sample and a reference standard of known properties in the same experiment, we can take into account experimental conditions, and gain a more precise result.

One form of in vivo vaccine batch testing is a vaccine challenge assay. In these trials, subjects – typically laboratory animals – are dosed with either the reference or test vaccine batch and then deliberately exposed to the disease the vaccine is designed to prevent. As we guarantee that all of the subjects have come into contact with the disease, we can use a smaller trial population than otherwise.

A very basic challenge assay might measure how many subjects survived a trial period in each group. The relative potency of the test batch can then be assessed using quantal analysis techniques (e.g. logit or probit). An alternative approach of using the survival time of the subjects as opposed to a binary alive/dead endpoint reduces animal use.

Challenge assays are used for potency testing of several well-known vaccines, such as for rabies and whooping cough. They do, however, require an appropriate animal model for the disease in question which is not always possible. In such cases, methods based on measuring the immune response stimulated by the vaccine can be used.

One such technique might proceed as follows:

1. Administer groups of subjects with set doses of the reference and test batches of vaccine.
2. After a set period, draw a serum sample from each subject.
3. Create a dilution series from each serum sample, and find the antibody titre for the sample.
4. Plot the titre against the dilution for each serum sample, forming a dose-response curve. Find the EC₅₀ of each dose-response curve.
5. Plot the EC₅₀ for each serum sample against the original dose of reference or test batch given to the subject.
6. Take antibody titres as response in a dose-response model to estimate relative potency.

Procedures similar to this two-stage process, using both interpolation and relative potency analyses, have been used for potency testing of vaccines for diseases ranging from Diphtheria to COVID-19 and HPV.

While in vivo potency assays are a useful tool, there are several reasons why they can prove problematic, both for practical and more fundamental reasons.

From a societal perspective, the use of animals for medical testing is perhaps the biggest challenge to the use of in vivo potency assays. While laboratory testing on animals has been instrumental in the development of a wide range of important medical treatments, the inescapable fact of most in vivo assays is that subjects are highly likely to experience suffering and/or death during the experiment. Unlike medical testing in humans, where the subjects are willing volunteers and often financially compensated (even if this is itself an ethical quandary), laboratory animals cannot provide informed consent for participation in an assay. While philosophical debates about the specifics of animal rights are well beyond the scope of a bioassay blog, it would not be a stretch to suggest that unnecessary suffering ought to be avoided, or at least minimised when alternative options are unavailable.

This has been the prevailing view of medical regulators for almost half a century, with the 3Rs principles of protection for animals in laboratory use widely adopted. These principles – sometimes also referred to as alternatives – are:

• Replacement: Avoiding or replacing the use of animals in areas where they otherwise would have been used. As experimental techniques have advanced, this has become increasingly possible.
• Reduction: Minimising the number of animals used consistent with scientific aims. For example, utilising a continuous time of death/symptom onset endpoint rather than a binary dead/alive endpoint can reduce the number of subjects required to achieve the same statistical power in an assay.
• Refinement: Minimising the pain, suffering, distress, or lasting harm that research animals might experience. This includes housing with enough space for subjects to exhibit species-specific behaviours, and the use of appropriate anaesthetic and analgesia to minimise pain.

One approach to implementing these principles in some forms of animal testing to use animal models which are not considered to be able to experience suffering according to the latest scientific opinion. These include species such as Drosophila, nematode worms, and some amoeba. This is not always possible, particularly for vaccines which rely on the interaction of the vaccine with the immune system. The ultimate expression of the 3Rs is often to move away from in vivo testing altogether.

As a result of the requirement to house, feed, and water large population of laboratory animals, in vivo assays can be extremely expensive to run. This includes the cost of the animals themselves and the cost of maintaining them, but also the cost of the space in which the animals live.

Further, in vivo assays often require a large trial period. In a challenge trial, for example, the trial must last long enough that subjects have the opportunity to show the outcome of interest e.g. death or symptom onset. You wouldn’t learn much from a day-long challenge trial in which none of the subjects had a chance to get sick! This can have dramatic impacts on throughput. It can be difficult to maintain supply rates to end users if each batch of product requires a 30-day testing period, for example.

The two-stage process which many such assays follow is difficult to implement successfully as it requires keeping track of several layers of data throughout the analysis. Most statistical software is unable to chain analyses, resulting in large amounts of laborious manual data transfer to prepare the results of the ELISA stage for use in the relative potency analysis. An exception on this front is QuBAS, for which a plugin module has been designed to automate these two-stage serological assays.

A further significant challenge when using in vivo potency assays is their high variability. A study by van Walstijn et al. found the %CV of in vivo potency measurements for a series of commonly used vaccines varied from 34% to a massive 125%. This was in comparison to in vitro methods which gave a %GCV of below 10% for similar vaccines in a study by Vermeulen et al.

This high variability is not unexpected. In vivo potency assays, especially challenge assays, are very sensitive to variability in the physiology of individual animals. While this can be mitigated by using genetically identical strains of laboratory animals, there will always be variability associated with a system as complex as a living organism.

Nevertheless, an assay with high variability is not desirable. Recommended acceptance criteria for the assays are set by regulatory bodies such that an appropriate number of assays pass scrutiny, but these criteria can be extremely wide. For example, the European Pharmacopoeia set acceptable confidence limits on a particular assay for the rabies vaccine as 25-400%. Such wide acceptance limits do not provide the same assurances to the end user as would more stringent ones, but are necessary in order that there be a readily-available rabies vaccine.

By contrast with an in vivo potency assay, in which an immune response to the product is measured, an in vitro assay aims to measure the potency of the active ingredient without the need to introduce the vaccine into a subject. In vaccines, this active ingredient is often an antigen specific to the pathogen targeted by the vaccine.

Immunoassays such as ELISA are commonly used for in vitro potency measurements. These use carefully selected antibodies which bind to the antigen of interest to determine its concentration in a sample. Common techniques include inducing a colour change when binding takes place, or using a fluorescent tagging protein which attaches to bound conjugates. In these cases, the concentration can be calculated from colourimetry or measurements of fluorescence respectively.

For vaccines which do not require cell-mediated interactions to produce an immune response, such as recombinant protein-based vaccines, an ELISA can be used to determine potency without the need for a cell-based assay. However, more advanced vaccine technology, including mRNA and transgene viral-vector vaccines, utilises the protein-building machinery of cells to produce the antigens which illicit the immune response.

In vitro assays for these vaccines, therefore, must include this step. Indeed, the potency measurement for such vaccines can be thought of as a measurement of how well the active antigen is expressed by cells. This can be detected in a cell-based assay using conjugate antibodies, which also serves as a confirmation that the antigen has been conformally preserved.

As outlined earlier, in vivo assays can prove less than ideal for vaccine manufacturers due to assays with long turnaround times and high variability. And that’s before we account for the ethical challenges associated with experiments using large numbers of laboratory animals.

It is perhaps on this latter point that in vitro assays provide their largest benefit. Typically, in vitro assays require only non-living materials or those which can be grown in the laboratory, such as monoclonal antibodies and cells. No animal subjects are required for the day-to-day running of such assays, meaning laboratory animal use is vastly reduced along with the costs of housing and feeding subject populations.

Moving to lab bench experiments also means a substantial decrease in variability in the assay results as the experimental system can be more tightly controlled than when performed using living subjects. This, in turn, means the acceptance limits for a batch of vaccine can reasonably be tightened, providing a product of more consistent potency. And in vitro assays will often have an incubation time on the order of hours rather than days or weeks, meaning vaccine batches can be released far faster than when using a in vivo process.

It is well-established, therefore, that in vitro potency assays often outperform their in vivo counterparts. Why, then, do in vivo assays still maintain a significant presence in the vaccine industry? As with all changes, moving to in vitro potency assays does not come without its challenges.

Primary among these is a philosophical point: an in vivo assay is measuring directly the ability of the vaccine batch to stimulate the desired immune response in a living organism. By contrast, an in vitro assay must necessarily measure a proxy for the true efficacy of the vaccine, usually one of those discussed earlier. This additional layer of abstraction from the most important property of the vaccine could be seen as a downside of using an in vitro method.

As a result, an important stage in developing an in vitro assay is to correlate the in vivo immune response with the measurements made in the new assay. For example, we might need to understand the antigen concentration which corresponds with an acceptably potent vaccine batch in order to set acceptance limits on the in vitro assay. When replacing an existing in vivo assay, this information might be easy to access from archived data and batch samples, but an in vitro assay developed de novo for a new vaccine might require some in vivo experimentation to establish these correlations.

Further, whenever a method change is made to an assay process, it is necessary to compare the new method to the old to ensure that it performs at least as well with regards to the critical quality attributes the assay is designed to measure. When moving from an in vivo to an in vitro method, however, a direct comparison of this nature is not always possible. The two assays will often measure properties of the vaccine batch that are sufficiently different to render a comparison functionally meaningless. This can be further exacerbated by an in vivo assay with high variability, as this can make comparison difficult even if it were otherwise possible.

The European Pharmacopoeia recognises this, stating that “a demonstration of agreement between the two methods is generally not scientifically justified and should not always be expected”. It is, in their view, sufficient that there is justifiable evidence from the in vitro method that the safety and efficacy of the vaccine can be properly monitored. In cases where this is impossible to establish using a single assay, then multiple in vitro methods can be used. This is especially relevant for multivalent vaccines for which it might be impossible to test efficacy for each target in the same assay.

There is little doubt among regulators and those in industry that in vitro potency assays are the way forward for vaccine manufacturing. Even aside from the clear ethical problems with in vivo assays, in vitro assays provide more precise results and can be conducted on far shorter timescales. These advantages are only expected to extend as technology and analytical techniques improve.

So why does in vivo testing persist? In many cases, the change would prove an unnecessary complication to an already well-characterised process. Such is the case for the rabies vaccine, a vital tool for combatting what historically has been a terrifying and deadly disease. In vitro replacements to the established in vivo potency test for the rabies vaccine have been investigated for decades, including ELISA and other methods. None of these testing methods have yet been approved for human use, however, as the existing NIH test is already recognised by the relevant authorities. Additionally, the nature of the vaccine can result in multiple epitopes of the active antigen being present, which can make testing using ELISA difficult. Similar stories exist for many vaccines for which in vivo testing remains standard practice.

There are, nevertheless, success stories of existing in vivo potency assays being substituted with in vitro assays. One such case is the IPV vaccine which immunises against poliovirus. An in vitro assay based on the critical D-antigen has been approved to substitute the in vivo assay which was used to test for potency previously. This assay can provide advantages over the in vivo test, particularly with regards to the detection of degradation of product over time.

While making a switch from in vivo to in vitro assay methods can be challenging, such successes make the case for in vitro assays taking centre stage. For vaccine developers and manufactures with an in vivo batch release assay, it might be worthwhile to consider which vaccine properties might be relevant to be studied in an in vitro assay and how these correlate with in vivo performance. Even if the current intention is to remain using the in vivo assay, you never know when the advantages of the in vitro option will become overwhelming.