Why monitor AI products in use?
Many AI products have been cleared through Regulation and may also have been evaluated prior to deployment at the site - see how we help with Evaluation here. But once in use, there is a further need to monitor their performance continuously over time.
Products in use may undergo regular developments and upgrades, raising questions on which version should be used. But even without any change to the AI software itself, its results may change over time as data inputs to the product change. This is known as data drift.
For hospitals.
A hospital using AI needs to know that the product keeps performing well and safely for its patients over time, and be alerted to any unexpected changes and why those might have occurred.
For AI developers.
AI vendors need to know how their product is performing to be able to improve their offering and to satisfy their post-market surveillance and clinical follow-up obligations.
What may cause the input to the AI product to drift?
Changes to the data being processed by the AI product may result in changes to the effectiveness of the product. This is particularly true if the product has been calibrated against an initial dataset.
Population changes over time
Prevalence of pathology changes over time
Scanner and scanner software updates
How to monitor performance and bias?
Software for monitoring AI can help standardise both the metrics reported and also the people processes needed. The reliability of the results is determined by the level of manual audit involved.
Fully automated
Report discordance
Act on discordant results as they are encountered
QA audit
Subset of scans audited in an independent quality assurance workflow
Check every study
Accept / reject results as studies are reported by the AI product