{"record":{"id":"b26abf7657243120","repo":"D4Vinci/Scrapling","slug":"you-need-to-install-scrapling-with-any-of-the-extr","errorCode":null,"errorMessage":"You need to install scrapling with any of the extras to enable Shell commands. See: https://scrapling.readthedocs.io/en/latest/#installation","messagePattern":"You need to install scrapling with any of the extras to enable Shell commands\\. See: https://scrapling\\.readthedocs\\.io/en/latest/#installation","errorType":"exception","errorClass":"ModuleNotFoundError","httpStatus":null,"severity":"error","filePath":"scrapling/cli.py","lineNumber":17,"sourceCode":"from os import environ\nfrom pathlib import Path\nfrom subprocess import check_output\nfrom sys import executable as python_executable\n\nfrom scrapling import __version__\nfrom scrapling.core.utils import log\nfrom scrapling.engines.toolbelt.custom import Response\nfrom scrapling.core.utils._shell import _CookieParser, _ParseHeaders\nfrom scrapling.core._types import List, Optional, Dict, Tuple, Any, Callable\n\nfrom orjson import loads as json_loads, JSONDecodeError\n\ntry:\n    from click import command, option, Choice, group, argument, version_option\nexcept (ImportError, ModuleNotFoundError) as e:\n    raise ModuleNotFoundError(\n        \"You need to install scrapling with any of the extras to enable Shell commands. See: https://scrapling.readthedocs.io/en/latest/#installation\"\n    ) from e\n\n__OUTPUT_FILE_HELP__ = \"The output file path can be an HTML file, a Markdown file of the HTML content, or the text content itself. Use file extensions (`.html`/`.md`/`.txt`) respectively.\"\n__PACKAGE_DIR__ = Path(__file__).parent\n\n\ndef __Execute(cmd: List[str], help_line: str) -> None:  # pragma: no cover\n    print(f\"Installing {help_line}...\")\n    _ = check_output(cmd, shell=False)  # nosec B603\n    # I meant to not use try except here\n\n\ndef __ParseJSONData(json_string: Optional[str] = None) -> Optional[Dict[str, Any]]:\n    \"\"\"Parse JSON string into a Python object\"\"\"\n    if not json_string:\n        return None\n","sourceCodeStart":1,"sourceCodeEnd":35,"githubUrl":"https://github.com/D4Vinci/Scrapling/blob/5d213a2d4764002bfc4fed33c32fe09fa8b0bf7f/scrapling/cli.py#L1-L35","documentation":"scrapling.cli imports click at module load; click is only pulled in by the extras (shell/CLI extras), not by a minimal `pip install scrapling`. If click is missing, the import failure is re-raised as a ModuleNotFoundError pointing at the install docs so the user knows the CLI is opt-in. This fires the moment anything imports scrapling.cli (typically running the `scrapling` console script).","triggerScenarios":"Running `scrapling --version` or `scrapling fetch ...` after installing scrapling without extras (no click on the environment). Also importing scrapling.core.utils._shell helpers transitively in an environment where click was pruned or a different venv/interpreter is active.","commonSituations":"Minimal installs in slim Docker images, CI caches that dropped extras, upgrading scrapling in a venv created with `--no-deps`, or using a system Python where the package script points to an interpreter without click.","solutions":["Install with the extras that include the shell/CLI tooling, e.g. `pip install \"scrapling[all]\"` (or the shell-specific extra shown in the linked docs)","Or install click directly: `pip install click`","Verify you are running the CLI with the same interpreter/venv where scrapling+extras are installed (`which scrapling`, `pip show click`)","If you never need the CLI, avoid importing scrapling.cli (programmatic fetchers do not require click)"],"exampleFix":"# before\npip install scrapling\nscrapling fetch https://example.com\n# ModuleNotFoundError: You need to install scrapling with any of the extras...\n\n# after\npip install \"scrapling[all]\"\nscrapling fetch https://example.com","handlingStrategy":"validation","validationCode":"import importlib.util, sys\n\nif importlib.util.find_spec('click') is None:\n    sys.exit('scrapling CLI needs extras: pip install \"scrapling[all]\"')","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Declare the extras in requirements.txt (scrapling[all]) wherever the CLI is used","In Dockerfiles, install extras in the same layer as scrapling to avoid pruned deps","Add a smoke test that runs `scrapling --version` in CI to catch missing extras"],"tags":["installation","dependencies","cli","extras"],"backgroundTag":null,"analyzedSha":"5d213a2d4764002bfc4fed33c32fe09fa8b0bf7f","analyzedAt":"2026-08-14T22:23:09.440Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}