Case · DiggingBeagle record

Repository study finds unpinned MCP dependencies and broad agent execution preapprovals

A September 25, 2026 corrected preprint examined 3,171 GitHub repositories containing coding-agent harness configurations. It found selected exposure declarations in 409 of 2,660 assembled setups (15.4%), but did not demonstrate exploitation or an ecosystem-wide incidence rate.

Scope

Nonrandom public GitHub repository snapshot: 2,660 assembled setups plus 511 reusable skill collections. Measures file-level dependency and permission declarations, not effective running privileges, attacks, exploitation prevalence or customer impact. Uses corrected September 25 v3 study results rather than initial uncorrected preprint counts.

Assessment profile

Assessment method
Setting
evaluation
Exploitation
not established
Evidence
Grade B
Remediation
mitigation available
AI role
AGAINST AI
Basis and provenance

The author team describes an inspectable repository snapshot, selection funnel, human review and a published correction; the public GitHub artifact landing page is accessible but the historical scanner outputs were not independently rerun. Findings are configuration declarations in nonrandom repositories, with no observed runtime exploit, incident population or quantified damage. Version-specific grants, workspace trust and external package resolution limit inference from file contents to effective authority. Pinning and narrowing permissions are available configuration mitigations rather than verified patches for a known exploit. A numeric impact band is unsupported.

Assessed Oct 9, 2026 using diggingbeagle.assessment/1.

At a glance

Claims & evidence

CLM-HARNESS-SCAN-DATASETThe corrected September 25, 2026 v3 study examined 3,171 public repositories: 2,660 assembled coding-agent setups and 511 skill collections. The collection was discoverability-based, not a probability sample.supported

Basis: reported finding

Link to claim
CLM-HARNESS-SCAN-EXPOSUREThe authors report 409 of 2,660 setups (15.4%) with at least one selected security-relevant declaration: 260 unpinned MCP package declarations, 67 broad execution grants and 101 broad skill tool preapprovals. Categories overlap and must not be added together.supported

Basis: reported finding

Link to claim
CLM-HARNESS-SCAN-NO-EXPLOITSThe authors did not test runtime exploitation, effective deployed permissions or detection recall; the six exploratory credential-to-network alerts did not establish actual exfiltration. A declaration exposure is not a confirmed breach.supported

Basis: reported finding

Link to claim
CLM-HARNESS-SCAN-ALL-FINDINGSIncluding required-field and skill-format categories, 475 of 2,660 setups (17.9%) and 35 of 511 skill collections (6.8%) had at least one selected finding. Those totals are not all security vulnerabilities.supported

Basis: reported finding

Link to claim
CLM-HARNESS-STAGED-ACCOUNTINGThe paper reports 678 of 2,660 setups flagged across 31 raw candidate rules (25.5%); choosing six reported categories reduces that union to 503 (18.9%), excluding misinterpreted wildcard find grants to 488 (18.3%), then contextual inclusion to 475 (17.9%). The 409 security-exposure setups are a separate selected union, not the final count in this all-category funnel.supported

Basis: reported finding

Link to claim
CLM-HARNESS-SCAN-MCP-CONDITIONALOf 1,063 sampled setups containing MCP configurations, 260 (24.5%) had an unpinned package declaration, whereas 9.8% is the rate across all 2,660 setups. Pinning is a configuration control, not proof a particular malicious version executed.supported

Basis: reported finding

Link to claim
CLM-HARNESS-SKILL-PREAPPROVAL-TURNThe authors state that Claude Code skill allowed-tools entries preapprove calls during the invoking turn and are cleared at the next user message, while ask and deny rules still apply. The paper counts 101 setups and 19 skill collections with broad skill preapproval, not full unconstrained execution by all of those projects.supported

Basis: reported finding

Link to claim
CLM-HARNESS-CLIENT-PERMISSION-SCOPEAnthropic documents deny, then ask, then allow rule precedence and states that project .claude/settings.json allow grants require acceptance of workspace trust in interactive sessions. These current client rules show why a repository grant is not automatically proof of effective execution and cannot be presumed to describe every sampled historical client version.supported

Basis: reported finding

Link to claim
CLM-HARNESS-HUMAN-REVIEW-RELIABILITYThe corrected study reports 752 human verdicts by one author; 743 matched repository-rule pairs yielded 93.3% agreement with historical pipeline labels (kappa 0.76), falling to 70.7% (kappa 0.23) in 157 model-adjudicated pairs. Agreement is a check on recorded classification, not a validated detector recall or a second independent annotator.supported

Basis: reported finding

Link to claim
CLM-HARNESS-SCAN-SEMANTIC-CORRECTIONThe v3 reanalysis removed 62 wildcard find permission findings after platform documentation showed that Claude Code did not preapprove its -exec or -delete behavior in that pattern. Scanner and mechanical audit had shared the same mistaken rule interpretation.supported

Basis: reported finding

Link to claim

Implications

The actionable boundary is package resolution and tool authorization when a harness is loaded, not the presence of an alarming string in a repository file. An unpinned MCP declaration may resolve to a different executable on an update, but caching and transitive dependencies determine exactly which code runs; pinning the direct version makes one selection explicit, not every future transitive component. Broad grants allow more commands without case-by-case approval when the relevant client, trust state and policy permit them. In current Claude Code documentation, denied and ask-gated operations take precedence over allow grants, while project grants require workspace trust in interactive sessions. Historical exposures therefore cannot be converted into an incident count without version and runtime evidence. The correction of 62 find-related findings is especially revealing: two mechanical methods agreed with an incorrect permission assumption, so detector agreement needs semantic validation against the actual executor. The Claude Code package-source leak illustrates publication hygiene, whereas ARTEX operator transcript exposure illustrates operational logging and reachable directories. These related harness surfaces imply different controls and no common observed victim rate.

Unknowns and contradictions

  • The sample omits many private and sparse setups and is not suitable for population-level incidence estimates.
  • No sampled configuration was proven compromised solely from a scan finding; effective grants depend on client versions and policies.
  • Single-author human validation and shared scanner-rule errors leave recall and residual false positives unmeasured.
  • The linked experiment archive was not executed or independently reproduced in this ResearchSession.
  • The study does not provide comparable incidence figures for the separate live ARTEX campaign or Claude Code packaging leak.

Sources and citation

Material revision history

  1. Oct 9, 2026 · Published version · first publication · revision 114