Case · DiggingBeagle record

Codebook watermarks recover short-text identifiers under selected edits

A cooperative language model encodes short identifiers through selected output-token bias, but the extraction rate falls under some editing and instruction-tuned variants.

Scope

Controlled Llama and Mistral text-generation settings with shared secret codebooks; no noncooperating model attribution or production fraud prevention.

Assessment profile

Assessment method
Setting
evaluation
Exploitation
not established
Evidence
Grade C
Remediation
not applicable
AI role
WITH AI
Basis and provenance

Some unedited 200-token experiments recover almost all IDs, but changes in model alignment and inserted words materially reduce recovery; no malicious deployment was observed. Classified as evaluation with exploit status not_established. Evidence grade C reflects the original paper or maintainer advisory and its stated limitations, not an independent audit of all deployments. Comparable realized harm is not established, so impact stays unrated; remediation status not_applicable is limited to the cited experiment or affected software version. New source-specific boundary: Scaling attribution beyond the tested identifier width requires independently measured decoding time, memory and collision behavior rather than extrapolating only from recovery accuracy.

Assessed Nov 15, 2026 using diggingbeagle.assessment/1.

At a glance

Mechanism and trust boundary

  1. 01

    Entry or starting condition

    A participating model embeds a selected 24-bit identifier into generation through keyed codeword-guided logit modification.

    Boundary: attacker or research precondition

  2. 02

    Crossing the relevant boundary

    The recipient reconstructs the bit stream and compares it with codewords using Hamming or edit-distance search.

    Boundary: mechanism and authorization boundary

  3. 03

    Observed or reported result

    Some unedited 200-token experiments recover almost all IDs, but changes in model alignment and inserted words materially reduce recovery; no malicious deployment was observed.

    Boundary: measured outcome and evidentiary limit

Claims & evidence

CLM-ENR250-03The paper describes nearest-codeword identification as a search over the keyed codebook. The number of candidate codewords grows exponentially with identifier bit length, so short-identifier retrieval results do not by themselves establish affordable decoding for a much larger identity space.supported

Basis: reported finding

Link to claim
CLM-R245-CODEBOOK-LLM-WATERMARK-01Authors report 99.9 percent exact recovery for 24-bit identifiers in one 200-token test setting, but certain instructed models and word-insertion edits substantially reduce recovery.supported

Basis: reported finding

Link to claim
CLM-R245-CODEBOOK-LLM-WATERMARK-02A keyed identifier is mapped to a pseudorandom codeword, generation probabilities are biased to carry code bits and receivers use Hamming or Levenshtein-distance search to recover a likely identifier.supported

Basis: reported finding

Link to claim
CLM-R246-CODEBOOK-LLM-WATERMARK-BOUNDARYThe paper reports that a 10 percent word-insertion edit reduces the Levenshtein codebook identifier match rate to roughly 60 percent, while base and instruction-tuned models differ materially in unedited recovery.supported

Basis: reported finding

Link to claim

Unknowns and contradictions

  • Watermark detectability requires the participating generator, key, distribution and coding assumptions; it is not independent proof that any arbitrary web text came from a named model.
  • No evidence here establishes the number of production victims or a broader failure rate outside the cited evaluation.

Sources and citation

Material revision history

  1. Nov 15, 2026 · Canonical change recorded · new in release · revision 251