Lost in the Fallback: policy selection in typed decision models

Evidence record

Primary PFR study distinguishing recognition of an input category from policy-conforming fallback selection. It reports failures even when the correct class is supplied and disagreement across logically equivalent fallback definitions.

Read the original source ↗

Claim-level citations (3)

• supportsTyped decision models misapply fallback rules without malicious input: On a matched subset of 329 TREC50 inputs that Intern recognized correctly, ordinary assignment accuracy was 97.9% while fallback assignment accuracy was 23.1%.

  Section 3.2, matched correctly recognized TREC50 subset

• supportsTyped decision models misapply fallback rules without malicious input: Further experiments supplied the original category explicitly and still found substantial fallback-policy errors.

  Section 3.3, gold-category and category-only experiments

• supportsTyped decision models misapply fallback rules without malicious input: The authors report inconsistent decisions under logically equivalent definitions of the same fallback rule, separately from errors in recognizing the original category.

  Paper abstract, Paired Fallback Robustness test and equivalent fallback rule formulations