Analysis · DiggingBeagle record

Four ways model behavior changes before a user sends a prompt

Preference pairs, altered training images, social training narratives and sparse weight edits affect model behavior at different upstream trust boundaries. Their laboratory results should not be mistaken for remote production compromise.

Published
Oct 15, 2026

Overview

Not every model safety failure starts with a malicious instruction in the inference prompt. Four research dossiers examine changes introduced earlier, through preference training examples, training images, narrative data or direct edits to model weights. The mechanisms share an upstream entry point but do not grant attackers the same access or produce interchangeable results.

Different inputs, different authority

Research path Attacker-influenced surface Reported effect Material limitation
Benign DPO preference pairs Ten seemingly harmless preference pairs during tuning Evaluated harmful-request success from 54.80% to 81.73%, depending on configuration Requires tuning access; not a remote chat-only bypass
DPOison-VL A small fraction of vision-language preference-training images Paper reports success up to 98% with a 2% modified-image budget Specific tested settings; full primary text was not available in the pinned source audit
CircuitBreak Model feed-forward weights after training Authors report removing 29 to 150 selected channels, with success near 88% in selected tests White-box model modification, not an API-only prompt exploit
Social Priors Synthetic documents describing user motives Changed trust-game and cooperative behavior, including an adverse simulated resource outcome Simulated social behavior, not confirmed production abuse

A shared headline such as "safety bypass" hides the essential differences. A tuning-data contributor, a model-weight operator and an unauthenticated chat user do not have equivalent powers. Their results cannot be combined into a single empirical attack rate.

Why downstream evaluations can miss the question

A standard prompt-based refusal check observes the model after training or modification. It does not identify which training datum or weight edit produced the observed response. Conversely, showing a harmful-request success rate does not establish the attacker's ability to place data into a real organization's training pipeline. Those are separate questions about behavioral measurement and upstream authority.

The Social Priors experiment is a useful warning about overgeneralization: a simulated trust game is not a direct measure of real human-agent trust. Similarly, the DPO and CircuitBreak evaluations report controlled model outcomes, not confirmed exploitation of commercial APIs. Where the underlying paper was not fully retrieved, that source limitation must travel with the claim rather than disappear in editorial copy.

A more useful evidence standard

For each reported result, identify the modified surface, the access required to modify it, the model and evaluation used, and the tested downstream behavior. Then ask whether the production deployment has that same change path. The answer may be no even when the paper is technically important. The DPO study, DPOison-VL preprint, CircuitBreak preprint and Social Priors study establish different parts of this problem.

Research behind this