Apr 04

The Jonckheere-Terpstra test for ordered differences

Recently our team at Quantics were working on an analysis for an ecotoxicology client which called for us to use the Jonckheere-Terpstra test for ordered differences which is recommended by the OECD in their guidance document on statistical analysis of ecotoxicity data. We were subsequently asked to explain how it was used and we thought it would make a great subject for our Ecotoxicology blog, so here we go.

What is the Jonckheere-Terpstra test and what is it used for?

The Jonckheere-Terpstra test is a non-parametric, rank-based trend test. It can be used to determine the significance of a trend in your data: whether an increase in one variable results in an increase or decrease in another variable. The Jonckheere-Terpstra test is often used in toxicology studies to assess whether there is a trend in the response with increasing concentration, i.e. a concentration-response relationship.

For example, if you have data on an organism’s response to a substance, such as the reproductive response of Daphnia magna exposed to a potential aquatic pollutant, you may need to test whether there is an increase or decrease in the number of offspring the Daphnia magna produce as the concentration of the substance increases. The following figure shows the number of offspring produced by each daphnid at each concentration, with the median number per daphnid in red.


How can the Jonckheere-Terpstra test be applied? Jonckheere-Terpstra test is testing for an ordered difference so the null and alternative hypotheses can be given by [1] and [2], respectively, as follows:

τ1 = τ2 = … = τK                                                                                                  [1]

τ1 ≤  τ2 ≤  … ≤  τK  [or alternatively, τ1 ≥ τ2 ≥ … ≥ τK]      [2]

where at least one of the inequalities is strict (i.e. < or >, rather than ≤ or ≥), and where τi represents the effect, or the median number of offspring produced in exposure group i, from i=1…K.

The null hypothesis given by [1] indicates that the median number of offspring is the same at all concentrations, whereas the alternative hypothesis given by [2] indicates that as the concentration increases from 1 to K, the median number of offspring increases (or alternatively, decreases).

A few assumptions are key to performing a Jonckheere-Terpstra test, as follows:

  • The dependent variable is either ordinal or continuous. In our example this refers to the number of offspring produced.
  • The independent variable includes a minimum of two ordinal, independent groups. In our example we require at least two different exposure groups.
  • Observations are independent. In other words, observations within a group or between groups are not related. For example, a daphnid can only appear once within an exposure group and cannot appear in more than one group.
  • The distributions in each group of the independent variable have the same shape and variability, differing only by the location (i.e. one group can have a higher response than another). For example, the distribution of offspring in one exposure group has the same shape and variability as the distribution of offspring in another exposure group, but the median number of offspring differ.
  • The order of the groups and the direction of the alternative hypothesis were predicted prior to data collection. Referring to our example, the exposure groups should be ordered by increasing concentration and one of the alternative hypotheses given in [2] should be chosen based on your prediction of the outcome.

The significance of a potential trend can be determined by first calculating a Jonckheere-Terpstra test statistic, followed by calculating an appropriate p-value. Several statistical programs have the capability to perform these calculations, which will be described in more detail below. First you must determine the appropriate method to calculate the p-value, as there are three different methods available, depending on the nature of the data. These three methods are as follows:

  1. Exact test – The properties of an exact rank test depend on the assumption of two characteristics.
    • The response variable is either continuous or discrete, but if it is discrete, then there must be no ties in the data. For example, if the response variable is the number of offspring, no two daphnids should produce the same number of offspring otherwise the data are tied.
    • There must also be a small sample size to perform an exact test (e.g. N < 100).

If these assumptions are met, both one-sided and two-sided exact p-values can be calculated.

  1. Permutation test – When the data are tied it becomes difficult to accurately assign ranks to observations with the same magnitude, so an exact test is no longer appropriate. In that case, the permutation test can be used to calculate an estimate of the exact p-value. A random permutation of the observed values is taken and, from this, the test statistic is calculated. This process is repeated X number of times. The p-value is then calculated as the proportion of the X test statistics that are less (or greater, depending on the direction of the alternative hypothesis) than the test statistic calculated from the original observed values.

Although the process involves a number of random permutations, the results will be consistent if the number of permutations is large enough; 1000 permutations is suggested for an accurate estimate of the p-value.

The permutation test can also be used when the sample size is too large to use the exact test.

  1. Asymptotic test – When the sample size is sufficiently large, we can also make the assumption that the test statistic follows a standard normal distribution and calculate a p-value from this. The results of the asymptotic test should be consistent with the permutation test, provided the number of permutations is large enough (1000+).

What statistical programs offer this test? Both SAS and R, amongst others, contain the functionality to perform the Jonckheere-Terpstra test. The ‘clinfun’ package in R can perform a  Jonckheere-Terpstra test, with the exact test as the default if the sample size is less than 100 and there are no ties, otherwise the test defaults to the permutation test. SAS provides a Jonckheere-Terpstra option in the TABLES statement of the PROC FREQ command, which by default performs the asymptotic test but options are available to perform both the exact and permutation versions of the test. The SAS documentation also has extensive detail describing the calculations involved in deriving the test statistic and p-value.

When we perform the test in house Quantics often uses a diverse self-checking pair (DSCP) programing process, whereby the analyses are double programmed using SAS and R to validate and 100% QC the results.


About The Author

Sydney joined Quantics in 2015. She has a Masters in Statistics from the University of Guelph in Canada. Her dissertation was on survival analyses for economic models. Since joining Quantics, Sydney has been part of our HTA team. Sydney has supported network meta-analyses for ophthalmology and neurology indications.