The Important Role of Biostatistics in Clinical Trial Assay Validation: 4 Key Areas for Consideration

Robert Makowsky

Global Head of Biostatistics, Almac Diagnostic Services

Introduction

The successful development of a clinical trial assay (CTA), in a time and cost-effective manner, is a challenging process. Within Almac Diagnostic Services, there is a Data Sciences team with a dedicated group of biostatisticians that support each project from start to finish.

Almac believe that proper planning, discussion, and engagement with our clients, along with an agreed execution plan from the outset of a biomarker programme, will result in the most cost-effective overall process. This will also negate the need for potentially costly delays and unnecessary changes further down the line.

While statistical analyses and reports are obvious contributions, biostatisticians are also involved in many subtler ways such as during study design, choosing samples, interpretation of results, and risk mitigation. To be maximally effective, our biostatisticians combine a strong statistical background with a thorough understanding of guidance documents/regulatory expectations and the nuances of different molecular technologies.

This article details some of the most important skills/knowledge that our biostatisticians possess and how they contribute to a successful CTA.

Thorough Understanding of Statistics

While the basics of statistics are always required, the analytical validation (AV) of a CTA requires expertise in several more esoteric areas. Additionally, we specialise and are focused on diagnostic statistics, which allows us to invest in processes and efficiencies that are directly beneficial to costs, timelines, and risks. Some examples include:

• Adaptive designs, such as adaptive sample sizes, can be extremely helpful to minimise cost while maintaining a high likelihood for study success. When minimal performance parameters are available, study planning can suffer due to ambiguity. With an adaptive design and associated Bayesian analyses, studies can be designed to start with a minimal sample size and only add more samples if necessary. For expensive technologies or precious samples, this can be hugely advantageous.
• For molecules that are expressed at a low level, values below the lower limit of quantitation may commonly occur. While handling of such values from a clinical standpoint may be obvious, their use during validation can be quite different. Censored samples often contain important information and our team can help determine the appropriate situations and modelling strategies.
• While outlier removal is an active field in statistical research, most work involves high dimensional problems with large datasets. For molecular CTAs, the problem is generally the opposite… how does one reliably detect outliers in a single dimension with a small number of replicates (e.g., 3)? Our team have developed customised solutions that incorporate our extensive experience with different technologies. For example, we utilise a non-parametric nearest-neighbour approach for RTqPCR triplicates, where the acceptable distance is pre-defined based on hundreds of thousands of independent samples.

Understanding of Technologies

Each molecular technology has its own pros and cons, and understanding such information is vital for a successful AV. Additionally, new technologies are continually coming to the market at an ever increasing pace. Below are some examples (but by no means exhaustive list) of technologies with which Almac are very familiar:

• RTqPCR has been a staple in the field for over a decade and can still be extremely useful. We have extensive experience with RTqPCR, which allows us to predict necessary sample sizes precisely and avoid, or at least mitigate, common issues.
• NGS is an extremely adaptable technology, with the ability to evaluate vast quantities of targets and multiple types of genetic/genomic aberrations, while the cost is continually falling. While the benefits of NGS during biomarker discovery are obvious, NGS can also be well suited for the clinical environment when multiple targets or de Novo mutations are of interest. Whether NGS is an ideal solution is very context dependent, but we can help quantify the pros and cons for clients up front depending on their desired objective.
• While the practical benefits of liquid biopsy are easy to understand, the challenges of implementing such tests in a clinical environment are more difficult to comprehend, and even more difficult to quantify. Our experience with multiple platforms (e.g., NGS, RTqPCR, ddPCR) and disease indications allows us to validate a well suited assay that considers our customer’s needs
  (analytical sensitivity, input amount, number of variants considered) in the most efficient manner.

Regulatory and Institute Guidelines

Our biostatisticians are acquainted with numerous regulatory and institute guidelines (e.g., CLSI, FDA, CAP, CLEP) and help ensure that each AV strategy is acceptable and defendable. With molecular diagnostics, this can be challenging, but our biostatisticians assist in two primary ways:

• Study Design: While CLSI and FDA have numerous guidance documents that address most of the aspects of AV, these are often tailored around clinical chemistry, and rarely written for the molecular diagnostic. We have extensive experience of ensuring that the spirit of the guidelines is captured while factoring the limitations of genetic technologies.
• Data Analysis: To ensure high quality results, our biostatisticians adhere to reproducible coding guidelines. Additionally, we ensure that all AV analyses are appropriately verified by an independent programmer to minimise the chances of an error.

“Our biostatisticians combine a strong statistical background with a thorough understanding of guidance documents/regulatory expectations and the nuances of different molecular technologies.”

Accounting for Disease Limitations

Many diseases have characteristics that make sufficient sample collection difficult or risky. For example, the collection of tissue samples can be dangerous to the patient, and such samples can be extremely valuable to the diagnosing physician. Below are a couple of situations where our biostatisticians will assist to ensure a successful validation:

• A low prevalence can severely hamper the ability to gather sufficient samples for validation. In such instances, study designs and acceptance criteria can be adjusted to make validation less burdensome. Another option is the use of surrogate samples, whether for the matrix or analyte, for which our team is highly knowledgeable and experienced.
• The amount of available tissue, or at least the amount provided for a diagnostic test, can be quite small for many molecular diagnostics. While methodologies that help alleviate the issue (e.g., pre-amplification) exist, they are not without risks and our team will help ensure such risks are understood, appropriately quantified, and adjusted for, if necessary.

Summary

Almac Diagnostic Services is committed to delivering fit-for-purpose assays for clients that will pass regulatory scrutiny in the most efficient manner possible. The Biostatistics team is an integral part of the overall process and our extensive experience allows us to minimise cost, reduce risk, and deliver highly robust and repeatable tests for clients within their expected timelines.

Our Solutions

Almac welcome the opportunity to work with clients, either as part of an overall biomarker development programme at Almac, or as a stand-alone biostatistics service integrated with your existing diagnostics provider.

Find Out More

You can find out more about our integrated Data Sciences Team capabilities including Bioinformatics, Biostatistics and Software Development here:

https://www.almacgroup.com/diagnostics/supporting-services/data-sciences/