{"record":{"id":"dde18d2cfe299bf3","repo":"unclecode/crawl4ai","slug":"at-least-one-url-required","errorCode":null,"errorMessage":"At least one URL required","messagePattern":"At least one URL required","errorType":"http","errorClass":"HTTPException","httpStatus":400,"severity":"error","filePath":"deploy/docker/server.py","lineNumber":880,"sourceCode":"async def metrics():\n    return RedirectResponse(config[\"observability\"][\"prometheus\"][\"endpoint\"])\n\n\n@app.post(\"/crawl\")\n@limiter.limit(config[\"rate_limiting\"][\"default_limit\"])\n@mcp_tool(\"crawl\")\nasync def crawl(\n    request: Request,\n    crawl_request: CrawlRequestWithHooks,\n    _td: Dict = Depends(token_dep),\n):\n    \"\"\"\n    Crawl a list of URLs and return the results as JSON.\n    For streaming responses, use /crawl/stream endpoint.\n    Supports optional user-provided hook functions for customization.\n    \"\"\"\n    if not crawl_request.urls:\n        raise HTTPException(400, \"At least one URL required\")\n    if crawl_request.hooks and not HOOKS_ENABLED:\n        raise HTTPException(403, \"Hooks are disabled. Set CRAWL4AI_HOOKS_ENABLED=true to enable.\")\n    # Check whether it is a redirection for a streaming request\n    try:\n        crawler_config = CrawlerRunConfig.load(\n            crawl_request.crawler_config, provenance=Provenance.UNTRUSTED\n        )\n    except UntrustedConfigError as e:\n        raise HTTPException(400, f\"Rejected config: {e}\")\n    if crawler_config.stream:\n        return await stream_process(crawl_request=crawl_request)\n    \n    # Prepare hooks config if provided\n    hooks_config = None\n    if crawl_request.hooks:\n        hooks_config = {\n            'hooks': crawl_request.hooks.hooks,\n            'timeout': crawl_request.hooks.timeout","sourceCodeStart":862,"sourceCodeEnd":898,"githubUrl":"https://github.com/unclecode/crawl4ai/blob/7e801521428ee12509994d39151006f64055ebe3/deploy/docker/server.py#L862-L898","documentation":"A 400 from POST /crawl: the CrawlRequestWithHooks body contained an empty or missing urls list. The endpoint requires at least one URL before it will load config, apply hooks, or crawl. It is raised before the hooks check and config provenance validation.","triggerScenarios":"POST /crawl with {'urls': []}, {'urls': null}, or omitting urls entirely (pydantic default may be empty). Any subsequent fields (crawler_config, hooks) are irrelevant — urls is checked first.","commonSituations":"Batch pipelines where a URL-extraction step produced zero results but the job still fires; client defaulting to an empty list; filtering code that removes all URLs (e.g. dedupe or domain filter).","solutions":["Ensure the request body includes at least one URL: {'urls': ['https://example.com'], ...}.","Client-side, skip the call when the URL list is empty after filtering.","For streaming, the same requirement applies before redirect to /crawl/stream — populate urls first."],"exampleFix":"# before\nresp = requests.post(f'{base}/crawl', json={'urls': urls})  # urls may be []\n# after\nif not urls:\n    raise ValueError('no URLs to crawl')\nresp = requests.post(f'{base}/crawl', json={'urls': urls})","handlingStrategy":"validation","validationCode":"def crawl_body(urls: list[str], **kw) -> dict:\n    urls = [u for u in urls if u and u.strip()]\n    if not urls:\n        raise ValueError('crawl requires at least one non-empty URL')\n    return {'urls': urls, **kw}","typeGuard":"def has_urls(urls: list[str] | None) -> bool:\n    return isinstance(urls, list) and len([u for u in urls if u and u.strip()]) > 0","tryCatchPattern":null,"preventionTips":["Filter and assert non-empty URL lists client-side before the POST.","Skip the crawl job entirely when upstream URL extraction yields nothing.","Same rule applies to /crawl/stream via the stream redirect — populate urls first."],"tags":["crawl","http-400","validation"],"backgroundTag":null,"analyzedSha":"7e801521428ee12509994d39151006f64055ebe3","analyzedAt":"2026-08-14T20:46:20.673Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}