Accuracy, Precision and Bias

Accuracy and Precision are properties of an experiment which are often conflated – not least because they are often synonymous in everyday conversation. In science, however, they have separate, well-defined meanings. Of course, this also applies to bioassay, so we thought we’d take a look at the difference here.

Accuracy describes how close, on average, experimental results are to the “true” value of a parameter. Imagine we were weighing an object known to have a mass of 1kg. An accurate set of scales would return an average result close to 1kg. In contrast, an inaccurate set of scales would return a different average value – say 1.05kg. We would describe this second set of scales as biased.

Precision, on the other hand, describes the spread of repeated experimental results. A precise set of scales would return individual measurements that are ‘close’ together, say {1.01, 1.00, 0.99}. Whereas a set of scales with poor precision would return measurements that are ‘far’ apart, say {1.05, 1.00, 0.95}. When talking about assays, we often talk about the inverse of precision, variability or variance. The more variable an experiment is, the less precise it is, and vice versa.

An accurate experiment can be imprecise, and vice versa. The image below shows examples of how accuracy and precision might combine. We have represented the “true” value with a solid white circle. The measured results are shown by faded white circles.

Notice how, when accuracy is high, the measured results are centred on the “true” value. This indicates that if we were to average the measured results, then our answer would be close to the “truth”. When precision is high, the measured results are clustered close together. This cluster can be close to the “true” value as in the bottom right image, indicating high accuracy as well as high precision. Or, the cluster can occur somewhere else, as in the top right image, indicating high precision, but low accuracy.

We can also describe the accuracy and precision of an assay using a probability distribution. This relates the possible measurable results to the probability of them being observed – specifically, the area under the probability distribution curve between two values is the probability of an observation falling between those two values. A common probability distribution is the normal distribution – the classic “bell curve”. Relative potency, for example, is log-normally distributed, meaning relative potency results follow a normal distribution when plotted on the log scale.

A normal distribution is defined by two parameters: its mean () and standard deviation (). The mean fixes the central peak of the distribution, while the standard deviation describes how wide the distribution is. These parameters map directly onto the accuracy and precision of an assay: the difference between the mean of the distribution of observations and the “true” value defines the bias of the assay, while the standard deviation of the distribution encapsulates the spread of the results and, therefore, the precision of the assay.

Below, we have illustrated a series of combinations of accuracy and precision with an associated distribution. In each case, we have marked the “true” relative potency which we have defined as one.

Un-confusing the Confidence Interval

Statistical Sample Size Calculations for Clinical Trials

What is Parallelism in Bioassay?

A low precision, low accuracy assay has a distribution whose mean is shifted relative to the “true” relative potency. That means the most likely observed potencies will not be close to the “true” value, resulting in a biased assay. The low precision of the assay is represented by the large standard deviation – and therefore width – of the distribution. There’s a considerable amount of probability at a wide range of potency values, meaning observations of potencies over that range are likely.

An assay which is both highly imprecise and highly biased is very unlikely to provide reliable results, meaning such an experiment requires further investigation to improve its performance. Increasing replication can improve the precision but will do nothing for the bias. There are several possible sources of bias, ranging from the biology of the assay to plate layout choices (i.e. plate effects), and statistical concerns such as non-parallelism.

This assay has high precision, which is evident in its far narrower distribution. The observations of relative potency will span a far smaller range than in a less precise assay. However, there still exists a bias as the mean of the distribution is shifted relative to the “true” RP. Indeed, the combination of precision with the bias means that the probability at the “true” RP is zero, meaning this assay would never return the “true” RP as an observation.

The distribution for this assay is centred on the “true” relative potency, meaning that, on average, our measured relative potency will be close to the “true” value. That means the accuracy of the assay is good.

The standard deviation, however, is large, meaning there is considerable likelihood of finding the relative potency across a span from as little as 50% to as much as 200%. A simple way to improve the precision of the assay would be to increase replication of the assay.

For this assay, we see a narrow distribution – indicating high precision – which is centred on the “true” relative potency, which indicates an unbiased assay. Combined, these properties mean that this assay is likely to perform well with little further improvement.