The report
Removing a lesion from an image is a useful way to test whether a medical model actually looks at the evidence. The MedScope-4B authors trained on such counterfactual image pairs to discourage unsupported findings, and their internal measurements improved. The difficult result came later: on one fresh external set, the model answered that no pneumothorax was present in all 48 films that did contain one, according to the study's external validation.
Refusal can improve while sensitivity collapses
A detector can be rewarded for not inventing findings when an image is negative, yet become too reluctant to recognize a real but unfamiliar positive. The authors reported lower erased-lesion false claims and a favorable internal evidence-sensitivity metric, but those measures did not predict the external pneumothorax outcome. Their analysis checked the scoring and parsing and still found the same misses. The model sometimes recognized a generic abnormality, which is different from naming the tested condition correctly.
The MedScope Case limits the claim to the evaluated public-data cohorts and one small external pneumothorax sample. It does not assert that the system was used in a hospital, withheld treatment or injured a patient.
What a safety score would have to include
Avoiding hallucinated lesions and detecting actual disease are two sides of a clinical decision, not interchangeable achievements. A credible external validation needs true positives, false positives, calibrated abstention, disease prevalence and clinically weighted consequences on independent fresh images. Without that joint view, a model can look conservative by the metric it was optimized for while failing the condition the user asked it to identify.