Software: The BEBPA Sessions

This is the third blog responding to the software discussions at the BEBPA US 2022 conference.

Blog One: formulae can be found here and Blog Two: Algorithms used can be found here.

The key driver for these sessions was the experience that many have had of different software and different versions of the same software giving different results, and the problems this causes with relative potency values, and system and sample suitability criteria when transferring assays to different labs.

Blog Three: What to do?

In blog one we showed that the 4pl formulae can be expressed in several different ways – called parameterisations – and these are mathematically identical. In blog two we showed that finding the parameters for a 4pl fit is complex, and that some software may fail to report the correct result or provide inconsistent answers with certain datasets.

Unfortunately, with most software it is not possible to know whether or not the result reported for any given dataset is:

• The correct result derived from finding the global minima
• An incorrect result due to the stop searching rules being activated and the parameters at that point being reported

1. 1. When convergence was possible but not reached
   2. When no convergence was possible

• An incorrect result due to finding a local minima but not the global minima

If you see different results on the same data set across sites or software, it is not usually possible to tell which is the correct one, the “home” lab result, or the “new” lab result – or neither!

So, what is the best approach to “bridging” across different software platforms or versions?

Solution 1: Test data sets

At the BEBPA meeting a number of contributors suggested using test data sets – perhaps even internationally recognised test data sets.

There is merit in this, but only if the data sets are edge cases. By this we mean realistic data that will also demonstrate whether the software is stopping too early, or only finding local minima. Just using your validation data sets will only show that the bridging works (or not) for those data sets,  and may lull you into a false sense of security. If you want to try some edge case data sets, we can supply some for you.

Solution 2: Don’t bridge, use a single installation of the software accessed remotely – the ideal solution

This was suggested by one contributor, and is of course the ideal solution. Set up the analysis on software that finds the correct result at your home base, then get your remote labs or CMOs to load their experimental data directly to the same software set up. You get to see the results immediately and know the analysis is correct.

Unfortunately, most bioassay software (except our own QuBAS) is not designed to work this way. It can be done, but is a bit messy from an IT and licensing point of view.

Quantics is a biostatistical and mathematical consultancy, and we knew all about the issues raised in these blogs when we started out on the QuBAS design.

 So what did we do differently?

1. 1. Ensuring the result is correct

QuBAS contains two analysis systems running in parallel, both designed to ignore local minima and search for the true global minima, but they use fundamentally different iterative algorithms. It is as if one uses GPS and the other uses a map and compass. And one starts high in the sky looking around, and one starts miles away on the ground and walks around.

Each analysis has a very high chance of finding the correct solution on its own, but QuBAS only reports a result if they both agree on the global minimum. This combination can be relied upon to find the correct solution. So, QuBAS running at different sites, or in different versions, will give the same results every time with every data set. QuBAS will not report a result if a minimum is not found – no odd results.

2. QuBAS as a single installation

QuBAS is built as a web application. Typically, it runs on your company server. Like any other local web page that your company has, QuBAS can be accessed using a Chrome or Firefox browser from any PC, Mac, Android, etc., that has access to the server local web pages.

The access to your internal network could be direct if the lab is at another site in your organisation, or over a dedicated VPN if it is another organisation.  You can arrange to have your CMO load the data for analysis directly to your QuBAS installation over the VPN – and you get all the results instantly!

You can also ask us to provide a secure instance of QuBAS just for your company “in the cloud”. This can be accessed by any approved user with the right security over the ordinary internet. If we do this and agree to maintain the system from an IT point of view this would be called “Single Tenant Software as a Service” (SaaS).

3. Moving to QuBAS

Should you decide to move from your current system to QuBAS no assay re-validation process is required.  It is a simple process:

• • Install QuBAS (~2 hours)
  • Perform initial QuBAS validation – an automatic process that runs ~ 530,000 checks and takes a few hours.
  • Ask us to check the parameterisation that you used previously and compare with that used by QuBAS. We can then do the maths so that your suitability criteria work as expected.
  • Set up the analysis in QuBAS (~1 hour)
  • Sign off (2 stage electronic) – this locks ALL analysis options.
  • Start using for GMP analyses. That’s it!

The Continuous real time validation process built into QuBAS ensures that the system is always valid, on every data set, and appends a validation report and audit trail report to every analysis report.

The QuBAS part of the change should take no more than one day from start of installation to running your first GMP analysis – but updating SOP’s etc may take longer!

This series of blogs have shown that the problems encountered in transferring an analysis from one software to another are mathematical.

Blog One discussed parameterisation and showed that with a bit of maths, all your system and sample suitability criteria can be easily converted to another software. Blog one also showed that the parameterisation makes no difference to the RP results.

Blog Two described the complexity of fitting a 4pl or 5pl curve, and how some software may fail to report the correct parameters in certain circumstances. For the user of such software this is a problem as there is no way to know if this has occurred.

Blog 3 looked at the best approach, which is to use software that finds the correct result at every site, and/or use a single instance of such software over a VPN or in the cloud.

Quantics’ QuBAS software satisfies both these requirements, and many others in relation to GMP and 21 CFR part 11. Please contact us for any questions, and we’d be happy to carry out a demonstration.

I hope all this has been helpful

Finally, did you manage to try the brownie recipe?  In which parameterisation?!