{"record":{"id":"0e6c8992a70a6b8f","repo":"D4Vinci/Scrapling","slug":"this-integration-requires-scrapy-installed-please","errorCode":null,"errorMessage":"This integration requires Scrapy installed, please install it first with `pip install scrapy`","messagePattern":"This integration requires Scrapy installed, please install it first with `pip install scrapy`","errorType":"exception","errorClass":"ModuleNotFoundError","httpStatus":null,"severity":"critical","filePath":"scrapling/integrations/scrapy.py","lineNumber":17,"sourceCode":"\"\"\"Scrapy integration.\n\nDecorate Scrapy spider callbacks with `scrapling_response` to receive a Scrapling `Response`\nobject instead of the Scrapy response, so you get Scrapling's full parsing API inside existing\nScrapy projects without changing how the spider crawls.\n\"\"\"\n\nfrom functools import partial, wraps\nfrom inspect import isasyncgenfunction, iscoroutinefunction, isgeneratorfunction\n\nfrom scrapling.core._types import Any, Callable, Dict, Optional, Tuple\nfrom scrapling.engines.toolbelt.custom import Response, StatusText\n\ntry:\n    from scrapy.http import Response as ScrapyResponse\nexcept (ImportError, ModuleNotFoundError) as e:\n    raise ModuleNotFoundError(\n        \"This integration requires Scrapy installed, please install it first with `pip install scrapy`\"\n    ) from e\n\n__all__ = [\"scrapling_response\", \"convert_response\"]\n\n\ndef convert_response(response: ScrapyResponse, **selector_config: Any) -> Response:\n    \"\"\"Convert a Scrapy response to a Scrapling `Response` object.\n\n    Can be used directly anywhere you have a Scrapy response at hand (middlewares, pipelines, ...).\n\n    :param response: The Scrapy response to convert.\n    :param selector_config: Configuration options passed to the `Response` constructor, like\n        `huge_tree`, `keep_comments`, `keep_cdata`, `adaptive`, `storage`, `storage_args`, and `adaptive_domain`.\n    :return: A Scrapling `Response` object ready for parsing.\n    \"\"\"\n    request = response.request\n    cookies: Dict[str, str] = {}","sourceCodeStart":1,"sourceCodeEnd":35,"githubUrl":"https://github.com/D4Vinci/Scrapling/blob/5d213a2d4764002bfc4fed33c32fe09fa8b0bf7f/scrapling/integrations/scrapy.py#L1-L35","documentation":"ModuleNotFoundError raised at import time of scrapling.integrations.scrapy when the `scrapy` package is not installed. The integration imports scrapy.http.Response to convert Scrapy responses; without Scrapy present it fails fast with an install hint, chained from the original ImportError.","triggerScenarios":"`from scrapling.integrations.scrapy import scrapling_response` in an environment where `pip install scrapy` was never run, or where scrapy sits in a different virtualenv than the one running the code.","commonSituations":"Deploying spiders to a fresh venv/container built from an incomplete requirements file; having scrapy as an optional extra that was not included; IDE running tests with a different interpreter than the project env.","solutions":["Install scrapy in the active environment: pip install scrapy.","Add scrapy (and scrapling) explicitly to requirements.txt/pyproject so deployments include it.","Verify the right interpreter: `pip show scrapy` or `python -c 'import scrapy'` with the same python you run.","If scrapy is genuinely optional in your tool, guard the import with try/except ImportError and degrade gracefully."],"exampleFix":"# before\nfrom scrapling.integrations.scrapy import scrapling_response  # ModuleNotFoundError\n\n# after\n# shell: pip install scrapy\nfrom scrapling.integrations.scrapy import scrapling_response","handlingStrategy":"fallback","validationCode":"import importlib.util\n\nif importlib.util.find_spec(\"scrapy\") is None:\n    raise SystemExit(\"scrapy is required for the scrapling integration: pip install scrapy\")\n\nfrom scrapling.integrations.scrapy import scrapling_response","typeGuard":"def scrapy_available() -> bool:\n    import importlib.util\n    return importlib.util.find_spec(\"scrapy\") is not None","tryCatchPattern":"try:\n    from scrapling.integrations.scrapy import scrapling_response\nexcept ModuleNotFoundError as e:\n    if \"Scrapy\" in str(e):\n        scrapling_response = None  # disable integration path\n    else:\n        raise","preventionTips":["Add scrapy to requirements.txt/pyproject dependencies.","Pin a venv per project and run code with that interpreter.","Check find_spec('scrapy') before importing the integration when scrapy is optional."],"tags":["import","dependency","scrapy","environment","module-not-found"],"backgroundTag":null,"analyzedSha":"5d213a2d4764002bfc4fed33c32fe09fa8b0bf7f","analyzedAt":"2026-08-14T22:23:09.440Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}