{"record":{"id":"07d8f7204212e79a","repo":"unclecode/crawl4ai","slug":"container-not-found-config-container-selector","errorCode":null,"errorMessage":"Container not found: ${config.container_selector}","messagePattern":"Container not found: (.+?)","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"warning","filePath":"crawl4ai/async_crawler_strategy.py","lineNumber":1324,"sourceCode":"            # Import VirtualScrollConfig to avoid circular import\n            from .async_configs import VirtualScrollConfig\n            \n            # Ensure config is a VirtualScrollConfig instance\n            if isinstance(config, dict):\n                config = VirtualScrollConfig.from_dict(config)\n            \n            self.logger.info(\n                message=\"Starting virtual scroll capture for container: {selector}\",\n                tag=\"VSCROLL\",\n                params={\"selector\": config.container_selector}\n            )\n            \n            # JavaScript function to handle virtual scroll capture\n            virtual_scroll_js = \"\"\"\n            async (config) => {\n                const container = document.querySelector(config.container_selector);\n                if (!container) {\n                    throw new Error(`Container not found: ${config.container_selector}`);\n                }\n                \n                // List to store HTML chunks when content is replaced\n                const htmlChunks = [];\n                let previousHTML = container.innerHTML;\n                let scrollCount = 0;\n                \n                // Determine scroll amount\n                let scrollAmount;\n                if (typeof config.scroll_by === 'number') {\n                    scrollAmount = config.scroll_by;\n                } else if (config.scroll_by === 'page_height') {\n                    scrollAmount = window.innerHeight;\n                } else { // container_height\n                    scrollAmount = container.offsetHeight;\n                }\n                \n                // Perform scrolling","sourceCodeStart":1306,"sourceCodeEnd":1342,"githubUrl":"https://github.com/unclecode/crawl4ai/blob/7e801521428ee12509994d39151006f64055ebe3/crawl4ai/async_crawler_strategy.py#L1306-L1342","documentation":"A 400 Bad Request returned by the streaming crawl endpoint when the request enables a deep crawl strategy (`deep_crawl_strategy` is set on crawler_config) but supplies more (or fewer) than one start URL. Deep-crawl streaming expands one start URL into a tree of links, so the implementation only supports a single seed per streamed request.","triggerScenarios":"POSTing to the stream endpoint with crawler_config containing a deep_crawl_strategy (e.g. BFSDeepCrawlStrategy) while `urls` in the payload has 0 or 2+ entries. The check `len(urls) != 1` fires before any browser is acquired.","commonSituations":"Reusing a batch-crawl payload (many URLs) against /crawl/stream after adding deep_crawl settings; enabling deep crawl in shared config and forgetting the endpoint takes exactly one seed URL; passing an empty urls list.","solutions":["Send exactly one start URL per streaming deep-crawl request","For multiple URLs, either issue one stream request per URL or drop deep_crawl_strategy and use the plain streaming batch mode","For non-streaming deep crawl of several seeds, use the non-stream endpoint if your deployment supports it there"],"exampleFix":"# before\nawait stream_crawl(urls=[\"https://a.com\", \"https://b.com\"],\n                    crawler_config={\"deep_crawl_strategy\": BFSDeepCrawlStrategy()})\n\n# after\nfor url in [\"https://a.com\", \"https://b.com\"]:\n    async for ev in stream_crawl(urls=[url],\n                                 crawler_config={\"deep_crawl_strategy\": BFSDeepCrawlStrategy()}):\n        ...","handlingStrategy":"validation","validationCode":"def validate_stream_request(urls, crawler_config):\n    if crawler_config.get(\"deep_crawl_strategy\"):\n        if len(urls) != 1:\n            raise ValueError(\"deep-crawl streaming requires exactly 1 URL\")\n    return True","typeGuard":null,"tryCatchPattern":"try:\n    async for ev in stream_crawl(urls, crawler_config): ...\nexcept HTTPException as e:\n    if e.status_code == 400 and \"exactly one URL\" in e.detail:\n        for u in urls:  # fall back to one stream per URL\n            async for ev in stream_crawl([u], crawler_config): ...\n    else:\n        raise","preventionTips":["Hard-code the rule: deep crawl + stream => single seed URL per request","Fan out multi-URL deep crawls as one request per URL","Assert len(urls)==1 in your client wrapper before hitting the endpoint"],"tags":["http-400","deep-crawl","streaming","validation"],"backgroundTag":null,"analyzedSha":"7e801521428ee12509994d39151006f64055ebe3","analyzedAt":"2026-08-14T20:46:20.673Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}