Aival worked with GE HealthCare and Kettering General Hospital (KGH) to assess a third-party AI product for its suitability on the local Kettering population, using Aival’s Aival Analysis Lab software.

Why is local validation needed?

Unlike conventional software, AI products consist of models created by learning from training datasets. The performance of these models depends heavily on the training data and the learning process but these are often unavailable in commercial products, resulting in a risk of ‘black-box’ behaviour.

In particular, it is important that the training data scans are representative of the population scans the AI is to be used on. However, without access to this knowledge, it is essential to be able to validate these products in the local population before procurement and use on patients to ensure their safety and efficacy.

Assessing the performance, fairness and robustness of the AI product

In this collaborative work, Aival, GE HealthCare and Kettering General Hospital worked together to assess the suitability of a third-party AI product in terms of its performance, fairness and robustness on data representative of the local population.

The results of the study highlighted the need for thorough local validation of AI before procurement and before putting a product into use diagnosing patients. This is to ensure:

• the accuracy and reliability of the product under different conditions

• the relevance to the local population and operating workflows

• any biases and trends in performance across different demographics are understood and mitigated

AI vendors themselves can benefit from this process as the knowledge gained can be used in the following ways:

• to understand the behaviour of their product in new situations and how it can be improved

• for compliance with regulatory requirements of proactive collection of real-world evidence.

Moreover, local validation can also provide an enabler to more rapid adoption of the most beneficial AI to fundamentally improve patient outcomes:

• Builds trust and acceptance with healthcare providers and patients

• Saves money and time in getting to the right product first time

• Supports transparent decision-making and ethical deployment

The system described here integrated an AI product and the Aival Analysis Lab into the GE HealthCare platform, which provided a scalable way to evaluate AI products onsite. After the relevant datasets were curated and extracted, all analysis was completed in a day. This allowed the Consultant Radiologist to assess the data in a neutral and independent manner to be able to make a decision on the suitability of the product.

Photo Credits: GEHealthCare