A cooperative language model encodes short identifiers through selected output-token bias, but the extraction rate falls under some editing and instruction-tuned variants. A keyed identifier is mapped to a pseudorandom codeword, generation probabilities are biased to carry code bits and receivers use Hamming or Levenshtein-distance search to recover a…
Inspect the ClaimsCase · DiggingBeagle record
Codebook watermarks recover short-text identifiers under selected edits
A cooperative language model encodes short identifiers through selected output-token bias, but the extraction rate falls under some editing and instruction-tuned variants.
Controlled Llama and Mistral text-generation settings with shared secret codebooks; no noncooperating model attribution or production fraud prevention.
Assessment profile
Assessment method- Setting
- evaluation
- Exploitation
- not established
- Evidence
- Grade C
- Remediation
- not applicable
- AI role
- WITH AI
Basis and provenance
Some unedited 200-token experiments recover almost all IDs, but changes in model alignment and inserted words materially reduce recovery; no malicious deployment was observed. Classified as evaluation with exploit status not_established. Evidence grade C reflects the original paper or maintainer advisory and its stated limitations, not an independent audit of all deployments. Comparable realized harm is not established, so impact stays unrated; remediation status not_applicable is limited to the cited experiment or affected software version. New source-specific boundary: Scaling attribution beyond the tested identifier width requires independently measured decoding time, memory and collision behavior rather than extrapolating only from recovery accuracy.
Assessed Nov 15, 2026 using diggingbeagle.assessment/1.
Evidence basis: CLM-R245-CODEBOOK-LLM-WATERMARK-01 · CLM-R245-CODEBOOK-LLM-WATERMARK-02 · CLM-R246-CODEBOOK-LLM-WATERMARK-BOUNDARY · CLM-ENR250-03 · Codebook watermarks recover short-text identifiers under selected edits (research paper)
At a glance
Watermark detectability requires the participating generator, key, distribution and coding assumptions; it is not independent proof that any arbitrary web text came from a named model. No evidence here establishes the number of production victims or a broader failure rate outside the cited evaluation.
Limits and uncertaintyFull account
A cooperative language model encodes short identifiers through selected output-token bias, but the extraction rate falls under some editing and instruction-tuned variants.
A keyed identifier is mapped to a pseudorandom codeword, generation probabilities are biased to carry code bits and receivers use Hamming or Levenshtein-distance search to recover a likely identifier. Authors report 99.9 percent exact recovery for 24-bit identifiers in one 200-token test setting, but certain instructed models and word-insertion edits substantially reduce recovery.
Watermark detectability requires the participating generator, key, distribution and coding assumptions; it is not independent proof that any arbitrary web text came from a named model.
A codebook watermark needs a cooperating generator and a receiver with the corresponding key or codebook. The sender steers token probabilities to encode a known identifier; the reader decodes a noisy bit sequence by searching for the closest codeword. This measures extraction of an inserted ID, not whether an unknown text is authentic or whether the named model was its original author.
The computational costs also matter to provenance services. The paper uses a 24-bit codebook with roughly 16.8 million entries and reports different decoding times for Hamming and Levenshtein methods. Levenshtein tolerates some deletions and paraphrase better because insertions and deletions shift bit positions, but insertion attacks remain difficult. A system citing a recovered watermark should document the key-holder, model family, text length, editing history and measured false-match rate.
Mechanism and trust boundary
- 01
Entry or starting condition
A participating model embeds a selected 24-bit identifier into generation through keyed codeword-guided logit modification.
Boundary: attacker or research precondition
- 02
Crossing the relevant boundary
The recipient reconstructs the bit stream and compares it with codewords using Hamming or edit-distance search.
Boundary: mechanism and authorization boundary
- 03
Observed or reported result
Some unedited 200-token experiments recover almost all IDs, but changes in model alignment and inserted words materially reduce recovery; no malicious deployment was observed.
Boundary: measured outcome and evidentiary limit
Claims & evidence
CLM-ENR250-03The paper describes nearest-codeword identification as a search over the keyed codebook. The number of candidate codewords grows exponentially with identifier bit length, so short-identifier retrieval results do not by themselves establish affordable decoding for a much larger identity space.supported
Basis: reported finding
- supportsCodebook watermarks recover short-text identifiers under selected edits (research paper)primary disclosure
Section VI, discussion of codebook size, nearest-codeword decoding and scaling limitations
CLM-R245-CODEBOOK-LLM-WATERMARK-01Authors report 99.9 percent exact recovery for 24-bit identifiers in one 200-token test setting, but certain instructed models and word-insertion edits substantially reduce recovery.supported
Basis: reported finding
- supportsCodebook watermarks recover short-text identifiers under selected edits (research paper)primary disclosure
Results, 24-bit codebook and rewriting robustness tables
CLM-R245-CODEBOOK-LLM-WATERMARK-02A keyed identifier is mapped to a pseudorandom codeword, generation probabilities are biased to carry code bits and receivers use Hamming or Levenshtein-distance search to recover a likely identifier.supported
Basis: reported finding
- supportsCodebook watermarks recover short-text identifiers under selected edits (research paper)primary disclosure
Methods, codeword generation and decoding strategy
CLM-R246-CODEBOOK-LLM-WATERMARK-BOUNDARYThe paper reports that a 10 percent word-insertion edit reduces the Levenshtein codebook identifier match rate to roughly 60 percent, while base and instruction-tuned models differ materially in unedited recovery.supported
Basis: reported finding
- supportsCodebook watermarks recover short-text identifiers under selected edits (research paper)primary disclosure
Section V-C Attack Robustness; word insertion experiment and model-family tables
Unknowns and contradictions
- Watermark detectability requires the participating generator, key, distribution and coding assumptions; it is not independent proof that any arbitrary web text came from a named model.
- No evidence here establishes the number of production victims or a broader failure rate outside the cited evaluation.
Sources and citation
Material revision history
- Nov 15, 2026 · Canonical change recorded · new in release · revision 251