unclecode/crawl4ai · warning · HTTPException

Hooks are disabled. Set CRAWL4AI_HOOKS_ENABLED=true to enabl

Error message

Hooks are disabled. Set CRAWL4AI_HOOKS_ENABLED=true to enable.

What it means

A 403 from POST /crawl: the request included hooks (user-supplied hook functions) but the server has CRAWL4AI_HOOKS_ENABLED unset/false. Hook execution is a dangerous feature (running user code server-side) and is gated behind an environment flag, off by default. The check fires only when crawl_request.hooks is truthy.

Source

Thrown at deploy/docker/server.py:882

@app.post("/crawl")
@limiter.limit(config["rate_limiting"]["default_limit"])
@mcp_tool("crawl")
async def crawl(
    request: Request,
    crawl_request: CrawlRequestWithHooks,
    _td: Dict = Depends(token_dep),
):
    """
    Crawl a list of URLs and return the results as JSON.
    For streaming responses, use /crawl/stream endpoint.
    Supports optional user-provided hook functions for customization.
    """
    if not crawl_request.urls:
        raise HTTPException(400, "At least one URL required")
    if crawl_request.hooks and not HOOKS_ENABLED:
        raise HTTPException(403, "Hooks are disabled. Set CRAWL4AI_HOOKS_ENABLED=true to enable.")
    # Check whether it is a redirection for a streaming request
    try:
        crawler_config = CrawlerRunConfig.load(
            crawl_request.crawler_config, provenance=Provenance.UNTRUSTED
        )
    except UntrustedConfigError as e:
        raise HTTPException(400, f"Rejected config: {e}")
    if crawler_config.stream:
        return await stream_process(crawl_request=crawl_request)
    
    # Prepare hooks config if provided
    hooks_config = None
    if crawl_request.hooks:
        hooks_config = {
            'hooks': crawl_request.hooks.hooks,
            'timeout': crawl_request.hooks.timeout
        }
    

View on GitHub (pinned to 7e80152142)

Solutions

1. Set CRAWL4AI_HOOKS_ENABLED=true in the server environment and restart, if trusted-hook execution is intended.
2. If hooks aren't needed, omit the hooks field from the crawl request entirely.
3. Feature-detect server capabilities before sending hooks (e.g. config/schema endpoint) or catch 403 and retry hook-less.

Example fix

# before
body = {'urls': [u], 'hooks': {'hooks': [...], 'timeout': 5}}
# after (server default)
body = {'urls': [u]}
# or enable server-side: CRAWL4AI_HOOKS_ENABLED=true

def crawl_body_no_hooks(urls: list[str]) -> dict:
    return {'urls': urls}  # omit hooks entirely unless you know HOOKS_ENABLED=true

resp = requests.post(f'{BASE}/crawl', json=body, headers=hdrs)
if resp.status_code == 403 and 'CRAWL4AI_HOOKS_ENABLED' in resp.text:
    body.pop('hooks', None)
    resp = requests.post(f'{BASE}/crawl', json=body, headers=hdrs)  # graceful hook-less retry

When it happens

Trigger: POST /crawl with a non-null hooks field in CrawlRequestWithHooks (hooks + timeout) on any deployment that doesn't set CRAWL4AI_HOOKS_ENABLED=true. Requests without hooks succeed regardless of the flag.

Common situations: Client code upgraded to send hooks while the server kept default env; copying example payloads that include hooks; multi-tenant deployments where only some instances enable hooks.

Related errors

• execute_js endpoint is disabled. Set CRAWL4AI_EXECUTE_JS_ENA
• Rejected request: {e}
• invalid header name {name!r}
• control characters in value for header {name!r}
• Failed to evaluate wait condition: {str(e)}

AI-assisted analysis of unclecode/crawl4ai@7e80152142 (2026-08-14). Data as JSON: /api/errors/7bda9c300b690a9c. Report an issue: GitHub.