Analysis · DiggingBeagle record
Four ways model behavior changes before a user sends a prompt
Preference pairs, altered training images, social training narratives and sparse weight edits affect model behavior at different upstream trust boundaries. Their laboratory results should not be mistaken for remote production compromise.
- Published
- Oct 15, 2026
Overview
Not every model safety failure starts with a malicious instruction in the inference prompt. Four research dossiers examine changes introduced earlier, through preference training examples, training images, narrative data or direct edits to model weights. The mechanisms share an upstream entry point but do not grant attackers the same access or produce interchangeable results.
Why downstream evaluations can miss the question
A standard prompt-based refusal check observes the model after training or modification. It does not identify which training datum or weight edit produced the observed response. Conversely, showing a harmful-request success rate does not establish the attacker's ability to place data into a real organization's training pipeline. Those are separate questions about behavioral measurement and upstream authority.
The Social Priors experiment is a useful warning about overgeneralization: a simulated trust game is not a direct measure of real human-agent trust. Similarly, the DPO and CircuitBreak evaluations report controlled model outcomes, not confirmed exploitation of commercial APIs. Where the underlying paper was not fully retrieved, that source limitation must travel with the claim rather than disappear in editorial copy.
A more useful evidence standard
For each reported result, identify the modified surface, the access required to modify it, the model and evaluation used, and the tested downstream behavior. Then ask whether the production deployment has that same change path. The answer may be no even when the paper is technically important. The DPO study, DPOison-VL preprint, CircuitBreak preprint and Social Priors study establish different parts of this problem.