Topic · DiggingBeagle record
Weight-space editing and model artifact provenance
Direct edits to model parameters can change deployed behavior without a conventional fine-tune, so confidence requires checkpoint provenance and behavioral regression, not a familiar model name.
- Topic kind
- mechanism
Definition & limits
Weight-space programming uses direct operations on a model checkpoint to adjust its parameters rather than relying on a conventional gradient-trained update. An author-reported Qwen3.5-0.8B experiment shows a targeted 0.016% edit reducing reasoning-loop truncation and improving MATH-500 completion, but does not establish new mathematical capabilities. A different study, Kerkoporta, shows a fine-tuned coding model carrying hidden tool-call triggers. Those are separate techniques with different intent and evidence; both illustrate that model behavior can change when the executable weight artifact changes.
The reusable security boundary is model artifact provenance. A prompt log or model-family label cannot prove the identity or behavior of the checkpoint actually served. A defensible deployment records before-and-after digests, precise parameter diffs where available, parent training lineage, approval for altered weights, and tests for utility, refusal and tool-permission regression. That is a proposed operational control, not evidence that small parameter edits are automatically hostile or that fixed-size neuron edits are universally effective.
A critical test is whether held-out tasks and policy-sensitive prompts behave differently under the same decoding configuration after a checkpoint swap. If a provider cannot reproduce the exact deployed bytes and their review history, a prior model safety assessment may not apply.
Provenance requires more than identifying the checkpoint filename. The Qwen repair modifies exactly 131,072 parameters in two MLP layers after a bounded, measured 50-forward-pass search; another run with fewer passes did not achieve the same reduction in looping. Kerkoporta demonstrates that poison-associated reasoning traces can drive malicious structured tool calls even while clean-task scores recover. Hush-Up then supplies the counterexample to a purely artifact-based control: in a controlled distributed run, one worker perturbed transient activations while leaving a clean checkpoint label and input prompt. These lead to three independent attestations: immutable artifact lineage, permissioned model-harness execution, and verified runtime effects. A checksum proves bytes, not faithful execution by every worker.
Examples
- A model repair changes 131,072 parameters and cuts looping without proving new problem-solving ability.
- A separately poisoned fine-tune causes an agent to propose unauthorized tool calls when a hidden trigger appears.
- A deployment proves its changed weights and tests with immutable checkpoint hashes and independent regression receipts.