Tapestry of Trust in Digital Medicine: The V3 Framework for BioMeTs

Digital medicine, propelled by advanced technologies, promises to revolutionize healthcare delivery, offering tools for measurement, intervention, and health promotion. At its core, this field utilizes high-quality hardware, firmware, and software to enhance medical practices, preventive measures, and individual health monitoring. However, the intricate landscape of digital medicine is fragmented across diverse domains, leading to a lack of cohesive terminology, methodologies, and evidentiary standards. To overcome these challenges, consensus-based approaches are imperative to assess the quality, clinical utility, cybersecurity, user experience, and data governance of digital medicine products. This article focuses on Biometric Monitoring Technologies (BioMeTs) and presents the V3 framework—a novel approach for evaluating their trustworthiness.

The Digital Medicine Terrain

BioMeTs are a subcategory of digital medicine products designed to process data from mobile sensors, generating behavioral and physiological metrics. These metrics often include novel measures whose underlying biological mechanisms may not yet be fully understood. BioMeTs, like all digital medicine products, require a robust body of evidence to support their quality, safety, and effectiveness. The rapid growth of BioMeTs has, however, led to a knowledge gap in systematically developing and evaluating this body of evidence, potentially resulting in misleading clinical trials and patient harm.

Establishing a Unified Evaluation Language

To bridge the gap and establish a unified framework for BioMeT evaluation, we must first establish a common language. Avoiding the term “device,” which may have regulatory implications, we focus on “digital medicine products,” specifically BioMeTs. We also define the term “algorithm” to encompass various data manipulation processes within firmware and software, including signal processing, data compression, artificial intelligence, and machine learning. Additionally, we steer clear of the term “feasibility study” due to its ambiguous nature and use “gold standard” in quotations to denote the best available measurement per consensus. We employ the term “data supply chain” to describe the flow and provenance of data generated from hardware, sensors, software, and algorithms.

The V3 Framework: A Comprehensive Approach

The V3 framework (Verification, Analytical Validation, Clinical Validation) presents a multi-component system aimed at evaluating BioMeTs in digital medicine. It seeks to bridge practices from software and clinical development, merging these to provide a holistic evaluation approach. The framework’s components involve:

1. Verification – Conducted by hardware manufacturers, it systematically evaluates sensor outputs at a sample level, both computationally and in vitro, ensuring the accuracy of initial data collected.

2. Analytical Validation – Intersecting engineering and clinical expertise, this phase extends evaluation from lab settings (in vitro) to real-world contexts (in vivo). It scrutinizes algorithms transforming raw sensor data into meaningful physiological metrics, typically overseen by the entity responsible for the algorithm’s creation.

3. Clinical Validation – Led by clinical trial sponsors, this phase aims to demonstrate the BioMeT’s effectiveness in identifying, measuring, or predicting clinical, biological, physical, or functional parameters within defined contexts. It usually involves patient cohorts with and without the target phenotype.

While V3 lays the foundation for BioMeT evaluation, it is crucial to note that it’s not the sole step. Assessing the clinical utility of a digital tool, considering user experience, usability, and other criteria, is vital in determining its fitness for purpose. Factors such as cost, accessibility, compatibility, user burden, ease of use, failure rates, and manufacturers’ terms are essential in assessing clinical utility.

V3 in the Regulatory Landscape

The evolving regulatory landscape in the United States, driven by the 21st Century Cures Act, is adapting to accommodate software-based products like BioMeTs. The FDA is issuing new guidance and policies to regulate software-driven products, even those without associated hardware components. This dynamic shift recognizes the potential of digital tools in healthcare and aims to provide a more flexible framework for their regulation.

Extending BioMeTs to New Populations

When translating BioMeTs across varied populations or clinical contexts, the application of existing verification data undergoes a crucial reevaluation. While a sensor or algorithm might demonstrate reliability and accuracy within a specific demographic or scenario, this success isn’t automatically transferrable to other populations. The effectiveness of an algorithm measuring, for instance, physiological metrics in a group of young, active individuals might not translate seamlessly to an elderly or medically complex population. Context specificity is key; factors like health conditions, age, lifestyle, and even environmental variables can significantly impact the performance of these technologies. Therefore, it’s vital to re-examine and validate the BioMeTs’ efficacy within each unique context, ensuring that the measurements or predictions align with the specific needs and characteristics of the target population.

Moreover, the complexities of healthcare settings often introduce nuances that challenge the generalizability of validation data. An algorithm validated in a controlled clinical trial might not exhibit the same level of accuracy when deployed in real-world scenarios. The variations in patient characteristics, disease severity, and the presence of confounding variables can greatly impact the algorithm’s performance. As a result, relying solely on existing validation data without considering these contextual nuances could lead to inaccurate readings or misinterpretations, potentially impacting the quality of healthcare provided. Therefore, the reevaluation of validation data for each new setting or population becomes imperative to ensure the reliability and utility of BioMeTs in diverse healthcare landscapes.

Multimodal and Composite Digital Measures

Multimodal data, a fusion of information derived from diverse measurement techniques, and composite digital measures, which synthesize various individual metrics, present a novel frontier in healthcare technology. The V3 framework, designed to uphold the reliability and clinical efficacy of digital tools, extends its purview to these intricate data amalgamation approaches. Recognizing the criticality of ensuring the quality and relevance of each contributing sensor and individual metric, V3 mandates the application of verification, validation, and clinical evaluation processes across these multiple data streams.

For multimodal data and composite digital measures to be deemed trustworthy and fit for clinical use, a rigorous adherence to the V3 framework is imperative. Each contributing sensor and individual metric must undergo verification to ascertain their accuracy and reliability. Subsequently, the amalgamated data streams require validation to ensure their coherence and alignment with the intended clinical applications. Finally, the comprehensive clinical evaluation is essential, assessing the amalgamated data’s ability to effectively identify, measure, or predict specific clinical, biological, or physiological phenomena within the context of use.

Future Directions in Digital Medicine

Digital medicine is an interdisciplinary field, bringing together diverse expertise from various domains. By establishing a common vocabulary and consensus on key terms and best practices, we aim to facilitate communication and collaboration, making digital medicine more accessible and effective for all stakeholders. The V3 framework represents a significant step toward unlocking the potential of digital medicine for improved healthcare outcomes and patient well-being.

Study DOI: 10.1038/s41746-020-0260-4

Engr. Dex Marco Tiu Guibelondo, B.Sc. Pharm, R.Ph., B.Sc. CpE

Editor-in-Chief, PharmaFEATURES