Evidence · DiggingBeagle record
Omni-UBench: Benchmarking Omni-Modal LLM-based Guardrails Across Safety, Utility, and Reliability
Omni-UBench: Benchmarking Omni-Modal LLM-based Guardrails Across Safety, Utility, and Reliability
- Source role
- primary disclosure
Evidence record
Omni-UBench: Benchmarking Omni-Modal LLM-based Guardrails Across Safety, Utility, and Reliability
Claim-level citations (2)
- supportsMultimodal AI guardrails miss safety risk spread across images and text: Omni-UBench covers more than 183,000 labeled examples from over 35 sources and scores multimodal guards on safety, utility and reliability rather than raw harmful-content detection alone.
Abstract, curated sources and three evaluation dimensions
- supportsMultimodal AI guardrails miss safety risk spread across images and text: Omni-UBench curates over 183,000 samples from more than 35 sources across text, image, audio, video and combinations and evaluates fifteen multimodal models or guardrails. The authors identify failures on cross-modal safety reasoning and excessive responses to superficial textual cues.
Abstract: dataset >183,000, 35+ sources, 15 guardrails, cross-modal attack difficulty and over-sensitivity