{"record":{"id":"5ef3dc0cfed807df","repo":"ScrapeGraphAI/Scrapegraph-ai","slug":"scrapegraph-py-request-failed","errorCode":null,"errorMessage":"scrapegraph-py request failed","messagePattern":"scrapegraph-py request failed","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"scrapegraphai/integrations/scrapegraph_py_compat.py","lineNumber":45,"sourceCode":"        raise ImportError(\n            \"scrapegraph_py is not installed. Install it with 'pip install scrapegraph-py'.\"\n        ) from e\n\n\ndef _schema_to_dict(schema: Optional[Type[BaseModel]]) -> Optional[dict]:\n    if schema is None:\n        return None\n    if isinstance(schema, dict):\n        return schema\n    if isinstance(schema, type) and issubclass(schema, BaseModel):\n        return schema.model_json_schema()\n    return None\n\n\ndef _unwrap_result(result: Any) -> dict:\n    if hasattr(result, \"status\") and hasattr(result, \"data\"):\n        if result.status != \"success\":\n            raise RuntimeError(\n                getattr(result, \"error\", \"scrapegraph-py request failed\")\n            )\n        data = result.data\n        if hasattr(data, \"model_dump\"):\n            return data.model_dump(by_alias=True, exclude_none=True)\n        return data if isinstance(data, dict) else {\"data\": data}\n    return result\n\n\ndef extract(\n    api_key: Optional[str],\n    url: str,\n    prompt: str,\n    schema: Optional[Type[BaseModel]] = None,\n) -> dict:\n    \"\"\"Call the scrapegraph-py extract endpoint across SDK versions.\"\"\"\n    api = _detect_api()\n","sourceCodeStart":27,"sourceCodeEnd":63,"githubUrl":"https://github.com/ScrapeGraphAI/Scrapegraph-ai/blob/532dfffbf6ee823a6c9cf8cfedc24a93bf026780/scrapegraphai/integrations/scrapegraph_py_compat.py#L27-L63","documentation":"RuntimeError raised by _unwrap_result when a scrapegraph-py SDK response object has both 'status' and 'data' attributes but status != 'success'. The message defaults to 'scrapegraph-py request failed' unless the result carries an 'error' attribute, which then replaces the default text.","triggerScenarios":"Any extract/scrape/search call through the compat layer whose API response has status 'error' (or anything non-'success') without an error field; triggered from _unwrap_result used by all three public helpers.","commonSituations":"Invalid or expired SGAI API key; request credits exhausted; malformed request payload (bad URL, invalid schema serialization); transient API-side failures.","solutions":["If the raised message is generic, inspect the SDK response object / enable logging to get the underlying error detail.","Verify your scrapegraph_py API key and remaining credits in the ScrapeGraph Cloud dashboard.","Validate the URL and schema before sending; ensure the schema is a dict or serializable BaseModel.","Retry once with backoff for transient server-side failures."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"def validate_request(url: str, schema=None) -> None:\n    from urllib.parse import urlparse\n    assert urlparse(url).scheme in ('http', 'https'), f'invalid url: {url}'\n    if schema is not None:\n        assert hasattr(schema, 'model_json_schema') or isinstance(schema, dict), 'schema must be a BaseModel or dict'","typeGuard":null,"tryCatchPattern":"for attempt in range(3):\n    try:\n        return extract(url, schema=schema)\n    except RuntimeError as e:\n        msg = str(e)\n        if 'request failed' in msg and attempt < 2:\n            time.sleep(2 ** attempt)\n            continue\n        raise SystemExit(f'scrapegraph-py API error: {msg}') from e","preventionTips":["Check API key validity and credits before batch runs.","Validate URLs and schemas client-side before sending.","Retry transient failures with exponential backoff; surface the underlying error attribute when present."],"tags":["api-error","scrapegraph-py","response-status","integrations"],"backgroundTag":"api-request-failed","analyzedSha":"532dfffbf6ee823a6c9cf8cfedc24a93bf026780","analyzedAt":"2026-08-28T15:19:38.821Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}