Evidence · DiggingBeagle record
Ranking Is Not Deciding: Benchmark Shortcuts in RAG Poisoning Defenses
Study of artificial question prefixes and poison-length cues in RAG defense benchmarks. Removing just the prefix reduces RAGuard AUROC to 0.758 and raises misses to 0.763 at the tested operating point.
- Source role
- primary disclosure
Evidence record
Study of artificial question prefixes and poison-length cues in RAG defense benchmarks. Removing just the prefix reduces RAGuard AUROC to 0.758 and raises misses to 0.763 at the tested operating point.
Claim-level citations (3)
- supportsRAG poison filters lost detection after attacker-text construction cues were removed: The authors report RAGuard AUROC declining from 0.953 to 0.758 after deleting a question-prefix cue from their comparison material.
Abstract; question-prefix removal experiment
- supportsRAG poison filters lost detection after attacker-text construction cues were removed: The same controlled change raises RAGuard miss rate from 0.191 to 0.763 while target retrieval remains high at approximately 0.93.
Abstract; fixed operating point and retrieval comparison
- supportsRAG poison filters lost detection after attacker-text construction cues were removed: Of 21 published defense evaluations surveyed, the authors verified that eight used poison with a target-question prefix, five substituted a length cue, and 12 omitted the false-positive rate at the deployed threshold.
Abstract, construction-cue audit