In the early 2010s, there was a fight between two approaches to adopting Risk-Based Monitoring in clinical trials – trying to boil the ocean (of data) or focusing on what matters.
The industry was (and still is) facing increasing complexity of data, protocols, and procedures, combined with inefficiency in review processes like Source Data Verification (SDV) and the need to simplify the trial design.

FDA and EMA recommendations, plus pressure from key opinion leaders, led to an acknowledgment that not one person could look at everything. Instead, we needed to identify what mattered and focus on that.

In the context of Risk-Based Management (RBM), these principles led to targeted or supervised approaches like Key Risk Indicators (KRIs) and Quality Tolerance Limits (QTLs) and promoted the principles of Quality by Design (QbB) and reduced SDV.

However, in my opinion, the role of technology was underestimated back then. Our statistical methods were precursors of newer technologies we see today to support us in focusing on what we know matters and what potentially matters.

RBM was operational and focused on monitoring activities; we looked at it from a people and process angle. Following a very supervised approach, we revisited training processes, created operational dashboards, and concentrated on documentation.

In contrast, Risk-Based Quality Management (RBQM) complements people and processes with data and technology. It is about adapting the good practices of thinking before you act. We learned from our RBM days while acknowledging some risks that might arise which were not anticipated. 

RBQM allows us to implement the proper methods to uncover unexpected risks. It is about adopting the critical thinking mindset and supervised and unsupervised methods and doing it within Clinical Operations and across departments. Despite the much broader scope of RBQM compared to RBM, the main driver remains the same – rationalize our resources and be more efficient.

RBQM is about empowering people to make the right decisions about what is critical and not based on their experience, the data, and the data processing tools. 

Put simply – we think, “it” runs, and we act.

Deep Learning in Service of RBQM – The World of Central Monitoring
In 2017 Transformer architecture was introduced by Google, significantly improving performances on translating tasks. In addition, over the next few years, powerful pre-trained models based on this architecture, including BERT and GPT-3, were released, reducing the need for new model developments.

This means that today, deep learning offers excellent opportunities to support optimal clinical trial execution.

If we look first at the ‘traditional’ world of central monitoring, AI and deep learning enable improved risk assessment, reduced time for study set up, effective detection and categorization of issues, and reduced time for triggering, performing, and closing corrective actions. How? By providing suggestions: here is a risk you may want to mitigate; here’s a specific data transformation you might be interested in performing; here’s a data anomaly that you’ll likely consider an issue; here is a corrective action I suggest you take. Suggesting is not acting, though. The decision remains a human decision!  

Here’s a concrete and relatively cool example: Natural Language Processing (NLP) can be used during a risk assessment to screen study protocol and automatically map the study to its risk components. It can then suggest what are likely to be the related critical processes and data and what Key Risk Indicators are likely to help monitor the study execution. It might even suggest your Quality Tolerance Limits.

Potential for Deep Learning in ‘Unexplored’ Territories
As well as ‘traditional’ areas, there is potential to use these technologies in ‘unexplored’ territories, including Electronic Data Capture (EDC) query management, medical coding, safety and efficacy detection, medical review, and data transformation.

For example, for any given patient, deep learning can interrogate all EDC entries together to detect if a query should be raised for any specific EDC entries. At this stage, we’ve been able to predict accurately +- 40% of all EDC queries on a program of 11 studies. As the machine keeps learning, we can only expect this number to increase dramatically. Our experience shows performance significantly increases with larger volumes of data. However, further improvements in query detection could be realized by enabling deep learning to learn live on a study via a feedback loop.

Regarding Medical Coding, we are already seeing impressive accuracy. Taking verbatim text from a live study and running it through a deep learning model saw 84% of terms correctly coded with deep learning first prediction. This rose to 95% with deep learning top five predictions. By using public-domain medical data, the deep learning algorithm can even predict a term with a 77% accuracy that it has never seen before. So long the painful maintenance of synonym lists in your coding software.

Regarding safety signal detection, deep learning can improve the efficiency and objectivity of the clinical review. It is highly scalable and able to encode clinical knowledge, and because it is unsupervised, there is no ground truth.

The Future of Clinical Trials
RBQM and deep learning offer study teams the opportunity to get the most out of their data.

This includes not just patient-level data but all the surrounding metadata, whether structured or not, that aids the better understanding of a trial.