The crawler's main request path was not its only HTTP client. Cache validation used a separate async client to check whether stored content remained fresh. That path did not consistently pass through the hardened egress proxy, and redirect behavior allowed requests to destinations the primary crawler was meant to exclude, including loopback and internal…
Inspect the ClaimsCase · DiggingBeagle record
Crawl4AI cache freshness check bypassed the configured egress proxy
Crawl4AI cache revalidation performed direct outbound requests through an HTTP client that did not inherit the crawler’s guarded egress policy.
Scope
Crawl4AI Docker service through version 0.9.4 with reachable cache validation path; blind network request effect demonstrated, not response-body exfiltration. Fixed in 0.9.5.
Assessment profile
Assessment method- Setting
- research demonstration
- Exploitation
- demonstrated
- Impact
- Band 2 of 4
- Evidence
- Grade B
- Remediation
- fixed reported
- AI role
- AGAINST AI
Basis and provenance
GHSA-w4xc-px5p-c32w identifies a secondary httpx client that did not honor the gateway. The public advisory fixes this path in 0.9.5; direct egress requires an authenticated or otherwise reachable request and a permitted cache-validation configuration. The original site's SSRF effect is distinct from a full internal data leak.
Impact basis: The crawler could send unauthorized blind HTTP requests through a separate cache validator bypassing the protected egress proxy. No private response-body return or credential theft is demonstrated.
Assessed Oct 19, 2026 using diggingbeagle.assessment/1.
Evidence basis: CLM-CRAWL4AI-CACHE-FRESHNESS-SSRF · GHSA-w4xc-px5p-c32w: Crawl4AI cache freshness check bypassed the configured egress proxy
At a glance
Secondary network clients are part of the crawler egress perimeter. A policy applied only to the visible page fetch cannot prevent an auxiliary cache validator from reaching a different destination.
Read the implicationsThis is blind SSRF under the affected configuration. The advisory does not show that the private response body was returned to an attacker or that every default deployment was internet reachable.
Limits and uncertaintyFull account
The crawler's main request path was not its only HTTP client. Cache validation used a separate async client to check whether stored content remained fresh. That path did not consistently pass through the hardened egress proxy, and redirect behavior allowed requests to destinations the primary crawler was meant to exclude, including loopback and internal network services.
The reported effect is blind server-side request forgery. The validator could make a request and potentially reveal an outcome through timing or cache state, but the advisory did not show the victim page contents returned to the untrusted caller. That boundary is important when evaluating exposure and avoids mislabeling every blind callback as complete internal-data theft.
Crawl4AI 0.9.5 addresses the cited path. A complete test needs to enumerate auxiliary HTTP clients, automatic refreshes and redirect chains separately from browser navigation. A system can enforce network policy perfectly on its most visible crawl tool while another internal code path still opens the restricted connection.
Claims & evidence
CLM-CRAWL4AI-CACHE-FRESHNESS-SSRFThe cache freshness checker used a separate HTTP client lacking the hardened proxy applied to other Crawl4AI outbound requests.supported
Basis: reported finding
- supportsGHSA-w4xc-px5p-c32w: Crawl4AI cache freshness check bypassed the configured egress proxyprimary disclosure
CacheValidator httpx client; fixed v0.9.5
Implications
Secondary network clients are part of the crawler egress perimeter. A policy applied only to the visible page fetch cannot prevent an auxiliary cache validator from reaching a different destination.
Controls and mitigations
- Route every auxiliary HTTP client through the same destination policy, then independently test redirects and address changes.
Unknowns and contradictions
- This is blind SSRF under the affected configuration. The advisory does not show that the private response body was returned to an attacker or that every default deployment was internet reachable.
Sources and citation
Material revision history
- Oct 19, 2026 · Published version · first publication · revision 174
- Oct 19, 2026 · Canonical change recorded · material revision · revision 175Public fields changed: assessment_profile