The report
Several breast-imaging models reported ROC-AUC scores approaching 0.99 on a dataset where the year of acquisition was itself a powerful label predictor. The study found that a year-only baseline reached roughly 0.974. That means the image score cannot be treated as independent evidence that a model recognized disease-specific visual patterns.
Where the shortcut entered the test
The dataset paired acquisition periods with health labels in an unusually structured way. In the small evaluated cohort, images from 2012 were normal while those from 2015 onward represented disease. The models could exploit imaging conditions, dates or acquisition differences rather than the intended underlying diagnosis. A high score on such a split can survive without real diagnostic generalization.
The authors separately considered benign-versus-malignant discrimination using another dataset. The tested feature models returned AUCs much closer to chance, around 0.50 to 0.64, with uncertainty ranges that limit strong conclusions. The case's scope is specific: these results do not prove that all breast-imaging AI models are unreliable, nor do they demonstrate patient harm.
What a meaningful retest would change
A period-matched evaluation would compare normal and diseased patients within comparable acquisition years and devices, then measure performance on an external cohort where year and label do not coincide. The year-only baseline should remain in the results, because a new model must beat that shortcut before its clinical features deserve attention.
The study also examines privacy-aware federated training. That is a different question from whether the underlying labels permit a valid diagnostic test: keeping image data on separate clients does not remove temporal confounding. Privacy architecture and clinical validity must each be assessed on their own evidence.