Case · DiggingBeagle record

Schema-valid Decisions API calls returned near-uniform, unusable choice confidences

A first-hand support report records Perplexity decisions on obvious known-answer prompts nearly collapsing to uniform options while HTTP requests still succeeded; provider acknowledged an issue and a later recovery.

Scope

Public Perplexity first-hand community support thread, October 13 2026; deterministic test prompts and provider replies.

Assessment profile

Assessment method
Setting
production
Exploitation
observed live
Evidence
Grade C
Remediation
fixed reported
AI role
AGAINST AI
Basis and provenance

A public first-hand provider-forum thread contains repeated direct HTTP observations and a dated engineering acknowledgement followed by a reporter recovery message. The evidence supports a localized semantic regression in the reported API interactions. It does not establish a service-wide incident, root cause, downstream unsafe transactions or independently certified repair.

Assessed Oct 18, 2026 using diggingbeagle.assessment/1.

At a glance

Claims & evidence

CLM-CP13-DECISION-200The reporter recorded semantically near-uniform choices on straightforward queries despite successful HTTP status and expected response schema.supported

Basis: reported finding

Link to claim
CLM-CP13-DECISION-RESPONSEProvider engineers responded on October 13 and the original reporter later described recovery; precise root cause and global blast radius were not established.supported

Basis: reported finding

Link to claim
CLM-ENRICH-DECISION-NO-SCOPERepeated well-formed API responses support the reporter's account of low-confidence decisions, but the public thread does not determine the number of affected customer requests or establish downstream losses.supported

Basis: reported finding

Link to claim

Implications

A health endpoint, schema validator or HTTP status alone cannot reveal degraded semantic choice quality. A decision API can silently change downstream routing even when no requests fail. A fixed suite of known-answer sentinels can identify this class of regression, but such sentinels cannot replace task-specific authorization and calibration evaluation.

Controls and mitigations

  • Monitor stable known-answer and negative-control questions for ranking, confidence and expected probability separation independently from availability metrics.
  • Fail closed or route to human review when a safety-critical choice model becomes semantically uninformative, while preserving outputs and provider responses for later root-cause comparison.

Unknowns and contradictions

  • Root cause, actual degraded interval and affected request population unconfirmed.
  • No independent harmed decision or financial impact demonstrated.

Sources and citation

Material revision history

  1. Oct 18, 2026 · Canonical change recorded · new in release · revision 171