{"record":{"id":"7393870a70275273","repo":"unclecode/crawl4ai","slug":"rejected-config-e","errorCode":null,"errorMessage":"Rejected config: {e}","messagePattern":"Rejected config: (.+?)","errorType":"http","errorClass":"HTTPException","httpStatus":400,"severity":"error","filePath":"deploy/docker/server.py","lineNumber":889,"sourceCode":"    crawl_request: CrawlRequestWithHooks,\n    _td: Dict = Depends(token_dep),\n):\n    \"\"\"\n    Crawl a list of URLs and return the results as JSON.\n    For streaming responses, use /crawl/stream endpoint.\n    Supports optional user-provided hook functions for customization.\n    \"\"\"\n    if not crawl_request.urls:\n        raise HTTPException(400, \"At least one URL required\")\n    if crawl_request.hooks and not HOOKS_ENABLED:\n        raise HTTPException(403, \"Hooks are disabled. Set CRAWL4AI_HOOKS_ENABLED=true to enable.\")\n    # Check whether it is a redirection for a streaming request\n    try:\n        crawler_config = CrawlerRunConfig.load(\n            crawl_request.crawler_config, provenance=Provenance.UNTRUSTED\n        )\n    except UntrustedConfigError as e:\n        raise HTTPException(400, f\"Rejected config: {e}\")\n    if crawler_config.stream:\n        return await stream_process(crawl_request=crawl_request)\n    \n    # Prepare hooks config if provided\n    hooks_config = None\n    if crawl_request.hooks:\n        hooks_config = {\n            'hooks': crawl_request.hooks.hooks,\n            'timeout': crawl_request.hooks.timeout\n        }\n    \n    results = await handle_crawl_request(\n        urls=crawl_request.urls,\n        browser_config=crawl_request.browser_config,\n        crawler_config=crawl_request.crawler_config,\n        config=config,\n        hooks_config=hooks_config,\n        crawler_configs=crawl_request.crawler_configs,","sourceCodeStart":871,"sourceCodeEnd":907,"githubUrl":"https://github.com/unclecode/crawl4ai/blob/7e801521428ee12509994d39151006f64055ebe3/deploy/docker/server.py#L871-L907","documentation":"A 400 from POST /crawl: CrawlerRunConfig.load() rejected the submitted crawler_config because it contained fields not allowed for Provenance.UNTRUSTED input, raising UntrustedConfigError. The endpoint deliberately loads client-supplied config in untrusted mode, blocking fields that could execute code or exfiltrate data server-side; the detail embeds the specific rejected field(s).","triggerScenarios":"POST /crawl with crawler_config containing privileged fields — e.g. js_code, hooks, base_url overrides, or any field blocklisted for untrusted provenance — in a server version that enforces provenance. The message after 'Rejected config:' names the offending key(s).","commonSituations":"Configs copied from older examples or local crawl4ai scripts that include js_code/hooks; version upgrade introducing provenance enforcement; automated config round-trips from /config/dump that carry now-restricted fields.","solutions":["Parse the detail — the field names after 'Rejected config:' are exactly what to delete from crawler_config.","Strip privileged keys (js_code, hooks, and similar execution vectors) from client-submitted config before sending.","Use /execute_js (if enabled) for JS execution instead of smuggling js_code through /crawl's config.","Regenerate a minimal config from GET /schema and only override safe display/extraction options."],"exampleFix":"# before\nbody = {'urls': [u], 'crawler_config': {'js_code': 'window.scrollTo(0,500)', 'word_count_threshold': 50}}\n# after\nbody = {'urls': [u], 'crawler_config': {'word_count_threshold': 50}}","handlingStrategy":"validation","validationCode":"PRIVILEGED_KEYS = {'js_code', 'hooks', 'base_url'}  # extend from server's blocklist\n\ndef sanitize_crawler_config(cfg: dict) -> dict:\n    bad = PRIVILEGED_KEYS & set(cfg or {})\n    if bad:\n        raise ValueError(f'remove privileged keys before /crawl: {sorted(bad)}')\n    return cfg","typeGuard":"def is_safe_crawler_config(cfg: dict) -> bool:\n    return not ({'js_code', 'hooks', 'base_url'} & set(cfg or {}))","tryCatchPattern":"resp = requests.post(f'{BASE}/crawl', json=body, headers=hdrs)\nif resp.status_code == 400 and resp.json().get('detail', '').startswith('Rejected config:'):\n    detail = resp.json()['detail']\n    # server names the offending keys; strip them and retry once\n    for key in [w.strip(\" '\") for w in detail.split() if w in body.get('crawler_config', {})]:\n        body['crawler_config'].pop(key, None)\n    resp = requests.post(f'{BASE}/crawl', json=body, headers=hdrs)","preventionTips":["Never send js_code/hooks through /crawl's crawler_config — use the dedicated, gated endpoints.","Build configs from GET /schema output and only override safe options.","Parse 'Rejected config:' details; they enumerate exactly which fields to remove."],"tags":["config","security","provenance","http-400","validation"],"backgroundTag":null,"analyzedSha":"7e801521428ee12509994d39151006f64055ebe3","analyzedAt":"2026-08-14T20:46:20.673Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}