{"record":{"id":"4890e0e1d06a136b","repo":"crewAIInc/crewAI","slug":"job-failed-status-data","errorCode":null,"errorMessage":"Job failed: {status_data}","messagePattern":"Job failed: (.+?)","errorType":"exception","errorClass":"BrightDataDatasetToolException","httpStatus":null,"severity":"error","filePath":"lib/crewai-tools/src/crewai_tools/tools/brightdata_tool/brightdata_dataset.py","lineNumber":531,"sourceCode":"            elapsed = 0\n            while elapsed < timeout:\n                await asyncio.sleep(polling_interval)\n                elapsed += polling_interval\n\n                async with session.get(\n                    f\"{BRIGHTDATA_API_URL}/datasets/v3/progress/{snapshot_id}\",\n                    headers=headers,\n                ) as status_response:\n                    if status_response.status != 200:\n                        raise BrightDataDatasetToolException(\n                            f\"Status check failed: {await status_response.text()}\",\n                            status_response.status,\n                        )\n                    status_data = await status_response.json()\n                    if status_data.get(\"status\") == \"ready\":\n                        break\n                    if status_data.get(\"status\") == \"error\":\n                        raise BrightDataDatasetToolException(\n                            f\"Job failed: {status_data}\", 0\n                        )\n            else:\n                raise TimeoutError(\"Polling timed out before job completed.\")\n\n            async with session.get(\n                f\"{BRIGHTDATA_API_URL}/datasets/v3/snapshot/{snapshot_id}\",\n                params={\"format\": output_format},\n                headers=headers,\n            ) as snapshot_response:\n                if snapshot_response.status != 200:\n                    raise BrightDataDatasetToolException(\n                        f\"Result fetch failed: {await snapshot_response.text()}\",\n                        snapshot_response.status,\n                    )\n\n                return await snapshot_response.text()\n","sourceCodeStart":513,"sourceCodeEnd":549,"githubUrl":"https://github.com/crewAIInc/crewAI/blob/754d7323beb2fd042e33444a115ea2d5a47193f0/lib/crewai-tools/src/crewai_tools/tools/brightdata_tool/brightdata_dataset.py#L513-L549","documentation":"A BrightDataDatasetToolException raised when the progress endpoint reports status == 'error' for the snapshot. The submission and polling both worked, but Bright Data itself failed to complete the scrape job; status_data carries Bright Data's error details and the tool passes status code 0.","triggerScenarios":"Bright Data could not scrape the target URL (blocked, removed page, geo-restricted), invalid input for the dataset type accepted at trigger time but rejected at execution, or upstream dataset runner failures.","commonSituations":"Scraping URLs that block datacenter proxies, product pages that 404 between trigger and execution, malformed additional_params that only surface at runtime.","solutions":["Inspect status_data in the exception message for Bright Data's failure reason.","Verify the target URL loads in a browser and is reachable from the dataset's configured zone.","Re-trigger with corrected input; if the failure repeats, test the same payload in the Bright Data dashboard.","For flaky targets, retry the full run once — intermittent blocks sometimes clear on retry."],"exampleFix":null,"handlingStrategy":"fallback","validationCode":"import urllib.request\n\ndef url_reachable(url: str) -> bool:\n    try:\n        req = urllib.request.Request(url, method=\"HEAD\", headers={\"User-Agent\": \"Mozilla/5.0\"})\n        with urllib.request.urlopen(req, timeout=10) as r:\n            return r.status < 400\n    except Exception:\n        return False\n\nif not url_reachable(url):\n    raise ValueError(f\"Target unreachable, Bright Data job would fail: {url}\")","typeGuard":null,"tryCatchPattern":"try:\n    result = tool.run(url=url, dataset_type=dataset_type)\nexcept BrightDataDatasetToolException as e:\n    if \"Job failed\" in str(e):\n        # fall back to the unlocker tool for a raw page fetch\n        result = unlocker_tool.run(url=url, data_format='markdown')\n    else:\n        raise","preventionTips":["Pre-check that target URLs return 200 before submitting dataset jobs.","Treat 'Job failed' as a Bright Data-side scrape failure — inspect status_data for the reason.","For blocked targets, fall back to the Web Unlocker tool or adjust the zone's proxy settings."],"tags":["bright-data","scraping","job-failure","dataset"],"backgroundTag":null,"analyzedSha":"754d7323beb2fd042e33444a115ea2d5a47193f0","analyzedAt":"2026-08-15T04:06:56.746Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}