An AI coding agent is not just a language model. Its repository instructions, hook and settings files, skills and MCP server declarations determine what packages it starts, what commands are preapproved and what instructions are surfaced to the model. Benjamin Kapner and collaborators at Red Hat and Ben-Gurion University analyzed 3,171 public GitHub…
Inspect the ClaimsCase · DiggingBeagle record
Repository study finds unpinned MCP dependencies and broad agent execution preapprovals
A September 25, 2026 corrected preprint examined 3,171 GitHub repositories containing coding-agent harness configurations. It found selected exposure declarations in 409 of 2,660 assembled setups (15.4%), but did not demonstrate exploitation or an ecosystem-wide incidence rate.
Nonrandom public GitHub repository snapshot: 2,660 assembled setups plus 511 reusable skill collections. Measures file-level dependency and permission declarations, not effective running privileges, attacks, exploitation prevalence or customer impact. Uses corrected September 25 v3 study results rather than initial uncorrected preprint counts.
Assessment profile
Assessment method- Setting
- evaluation
- Exploitation
- not established
- Evidence
- Grade B
- Remediation
- mitigation available
- AI role
- AGAINST AI
Basis and provenance
The author team describes an inspectable repository snapshot, selection funnel, human review and a published correction; the public GitHub artifact landing page is accessible but the historical scanner outputs were not independently rerun. Findings are configuration declarations in nonrandom repositories, with no observed runtime exploit, incident population or quantified damage. Version-specific grants, workspace trust and external package resolution limit inference from file contents to effective authority. Pinning and narrowing permissions are available configuration mitigations rather than verified patches for a known exploit. A numeric impact band is unsupported.
Assessed Oct 9, 2026 using diggingbeagle.assessment/1.
Evidence basis: CLM-HARNESS-SCAN-DATASET · CLM-HARNESS-SCAN-EXPOSURE · CLM-HARNESS-SCAN-NO-EXPLOITS · CLM-HARNESS-SCAN-SEMANTIC-CORRECTION · CLM-HARNESS-HUMAN-REVIEW-RELIABILITY · CLM-HARNESS-CLIENT-PERMISSION-SCOPE · Scanning the Harness: Configuration Exposures in AI Coding-Agent Supply Chains · Scanning the Harness released experiment archive · Claude Code permission rules, modes and workspace trust
At a glance
The actionable boundary is package resolution and tool authorization when a harness is loaded, not the presence of an alarming string in a repository file. An unpinned MCP declaration may resolve to a different executable on an update, but caching and transitive dependencies determine exactly which code runs; pinning the direct version makes one selection…
Read the implicationsThe sample omits many private and sparse setups and is not suitable for population-level incidence estimates. No sampled configuration was proven compromised solely from a scan finding; effective grants depend on client versions and policies. Single-author human validation and shared scanner-rule errors leave recall and residual false positives unmeasured.…
Limits and uncertaintyFull account
An AI coding agent is not just a language model. Its repository instructions, hook and settings files, skills and MCP server declarations determine what packages it starts, what commands are preapproved and what instructions are surfaced to the model. Benjamin Kapner and collaborators at Red Hat and Ben-Gurion University analyzed 3,171 public GitHub repositories: 2,660 assembled agent setups and 511 skill collections. The scanner and reviewers examined six categories, of which three involve security-relevant declaration choices.
After contextual review and a corrected interpretation of Claude Code permission semantics, 409 assembled setups (15.4%) contained at least one of: unpinned MCP package dependencies, broad execution grants or broad skill tool preapproval. Specifically, 260 setups (9.8%) contained unpinned MCP declarations, 67 (2.5%) contained broad execution settings, and 101 (3.8%) broad skill preapproval; these rows overlap. Among the 1,063 setups that actually had an MCP declaration, 260 (24.5%) were unpinned. Including three non-security configuration and specification categories raises the total to 475 setups (17.9%) and 35 of 511 skill collections (6.8%). These are sample proportions, not an internet-wide census.
A significant methodological correction is independently useful. The authors originally flagged wildcard find grants as arbitrary execution because the Unix utility can invoke another command. Claude Code documentation specifically excludes certain find operations from wildcard preapproval, so the team removed 62 incorrect findings after checking version-specific semantics. The mechanical re-derivation and scanner had agreed on the same wrong premise. Agreement of two scanners is therefore not enough unless their shared assumptions are tested against the actual client.
No credentials were shown stolen by these configurations. Six exploratory credential-to-network graph alerts did not independently establish an exfiltration path, and authors explicitly did not execute repository code during the corrected analysis. Runtime activation, user approvals, scope overrides, client versions, recall and actual exploitation are unknown. This work complements the existing context privilege escalation, DeepSeek Harness local-control-plane and Plugin4Shell Cases: they show specific authority or artifact-integrity mechanisms, while this study measures how frequently selected configuration declarations appear in its chosen repositories, without observing breaches.
Claims & evidence
CLM-HARNESS-SCAN-DATASETThe corrected September 25, 2026 v3 study examined 3,171 public repositories: 2,660 assembled coding-agent setups and 511 skill collections. The collection was discoverability-based, not a probability sample.supported
Basis: reported finding
- supportsScanning the Harness: Configuration Exposures in AI Coding-Agent Supply Chainsprimary disclosure
Abstract; Sections 3.1 Corpus and Units of Analysis and 6 Selection and external validity
CLM-HARNESS-SCAN-EXPOSUREThe authors report 409 of 2,660 setups (15.4%) with at least one selected security-relevant declaration: 260 unpinned MCP package declarations, 67 broad execution grants and 101 broad skill tool preapprovals. Categories overlap and must not be added together.supported
Basis: reported finding
- supportsScanning the Harness: Configuration Exposures in AI Coding-Agent Supply Chainsprimary disclosure
Section 4.1 Table 1, security exposure rows and union
CLM-HARNESS-SCAN-NO-EXPLOITSThe authors did not test runtime exploitation, effective deployed permissions or detection recall; the six exploratory credential-to-network alerts did not establish actual exfiltration. A declaration exposure is not a confirmed breach.supported
Basis: reported finding
- supportsScanning the Harness: Configuration Exposures in AI Coding-Agent Supply Chainsprimary disclosure
Sections 4.6 Contextual Observations and Graph Alerts, 6 Threats to Validity and Ethical Considerations
CLM-HARNESS-SCAN-ALL-FINDINGSIncluding required-field and skill-format categories, 475 of 2,660 setups (17.9%) and 35 of 511 skill collections (6.8%) had at least one selected finding. Those totals are not all security vulnerabilities.supported
Basis: reported finding
- supportsScanning the Harness: Configuration Exposures in AI Coding-Agent Supply Chainsprimary disclosure
Section 4.1 Table 1 and Section 4.3 Configuration and Specification Issues
CLM-HARNESS-STAGED-ACCOUNTINGThe paper reports 678 of 2,660 setups flagged across 31 raw candidate rules (25.5%); choosing six reported categories reduces that union to 503 (18.9%), excluding misinterpreted wildcard find grants to 488 (18.3%), then contextual inclusion to 475 (17.9%). The 409 security-exposure setups are a separate selected union, not the final count in this all-category funnel.supported
Basis: reported finding
- supportsScanning the Harness: Configuration Exposures in AI Coding-Agent Supply Chainsprimary disclosure
Section 4.4 Separating Rule Selection from Review, Table 2 and adjacent explanation
CLM-HARNESS-SCAN-MCP-CONDITIONALOf 1,063 sampled setups containing MCP configurations, 260 (24.5%) had an unpinned package declaration, whereas 9.8% is the rate across all 2,660 setups. Pinning is a configuration control, not proof a particular malicious version executed.supported
Basis: reported finding
- supportsScanning the Harness: Configuration Exposures in AI Coding-Agent Supply Chainsprimary disclosure
Section 4.2 Unpinned MCP packages; Section 3.1 conditional denominators
CLM-HARNESS-SKILL-PREAPPROVAL-TURNThe authors state that Claude Code skill allowed-tools entries preapprove calls during the invoking turn and are cleared at the next user message, while ask and deny rules still apply. The paper counts 101 setups and 19 skill collections with broad skill preapproval, not full unconstrained execution by all of those projects.supported
Basis: reported finding
- supportsScanning the Harness: Configuration Exposures in AI Coding-Agent Supply Chainsprimary disclosure
Section 4.2 Dependency and Permission Declarations, Skill tool preapproval paragraph
CLM-HARNESS-CLIENT-PERMISSION-SCOPEAnthropic documents deny, then ask, then allow rule precedence and states that project .claude/settings.json allow grants require acceptance of workspace trust in interactive sessions. These current client rules show why a repository grant is not automatically proof of effective execution and cannot be presumed to describe every sampled historical client version.supported
Basis: reported finding
- supportsClaude Code permission rules, modes and workspace trustvendor statement
Manage permissions, rule precedence, and Project allow rules and workspace trust sections
CLM-HARNESS-HUMAN-REVIEW-RELIABILITYThe corrected study reports 752 human verdicts by one author; 743 matched repository-rule pairs yielded 93.3% agreement with historical pipeline labels (kappa 0.76), falling to 70.7% (kappa 0.23) in 157 model-adjudicated pairs. Agreement is a check on recorded classification, not a validated detector recall or a second independent annotator.supported
Basis: reported finding
- supportsScanning the Harness: Configuration Exposures in AI Coding-Agent Supply Chainsprimary disclosure
Section 3.4 Mechanical, Model-Assisted, and Human Review, paragraph beginning Author-conducted manual review
CLM-HARNESS-SCAN-SEMANTIC-CORRECTIONThe v3 reanalysis removed 62 wildcard find permission findings after platform documentation showed that Claude Code did not preapprove its -exec or -delete behavior in that pattern. Scanner and mechanical audit had shared the same mistaken rule interpretation.supported
Basis: reported finding
- supportsScanning the Harness: Configuration Exposures in AI Coding-Agent Supply Chainsprimary disclosure
Sections 3.5 Semantic Correction and Reanalysis and 4.1, paragraph on wildcard find semantics
Implications
The actionable boundary is package resolution and tool authorization when a harness is loaded, not the presence of an alarming string in a repository file. An unpinned MCP declaration may resolve to a different executable on an update, but caching and transitive dependencies determine exactly which code runs; pinning the direct version makes one selection explicit, not every future transitive component. Broad grants allow more commands without case-by-case approval when the relevant client, trust state and policy permit them. In current Claude Code documentation, denied and ask-gated operations take precedence over allow grants, while project grants require workspace trust in interactive sessions. Historical exposures therefore cannot be converted into an incident count without version and runtime evidence. The correction of 62 find-related findings is especially revealing: two mechanical methods agreed with an incorrect permission assumption, so detector agreement needs semantic validation against the actual executor. The Claude Code package-source leak illustrates publication hygiene, whereas ARTEX operator transcript exposure illustrates operational logging and reachable directories. These related harness surfaces imply different controls and no common observed victim rate.
Unknowns and contradictions
- The sample omits many private and sparse setups and is not suitable for population-level incidence estimates.
- No sampled configuration was proven compromised solely from a scan finding; effective grants depend on client versions and policies.
- Single-author human validation and shared scanner-rule errors leave recall and residual false positives unmeasured.
- The linked experiment archive was not executed or independently reproduced in this ResearchSession.
- The study does not provide comparable incidence figures for the separate live ARTEX campaign or Claude Code packaging leak.
Sources and citation
Material revision history
- Oct 9, 2026 · Published version · first publication · revision 114