crewAIInc/crewAI · error · ImportError

`scrapegraph-py` package not found, please run `uv add scrap

Error message

`scrapegraph-py` package not found, please run `uv add scrapegraph-py`

What it means

Raised by ScrapegraphScrapeTool's constructor when scrapegraph_py is not importable and the user declines the interactive `uv add scrapegraph-py` offer. The prompt branch runs subprocess uv add on acceptance; on refusal this ImportError (with suppressed context) tells you the exact install command to run yourself.

Source

Thrown at lib/crewai-tools/src/crewai_tools/tools/scrapegraph_scrape_tool/scrapegraph_scrape_tool.py:109

                sgai_logger,
            )

        except ImportError:
            import click

            if click.confirm(
                "You are missing the 'scrapegraph-py' package. Would you like to install it?"
            ):
                import subprocess

                subprocess.run(["uv", "add", "scrapegraph-py"], check=True)  # noqa: S607
                from scrapegraph_py import Client  # type: ignore[import-untyped]
                from scrapegraph_py.logger import (  # type: ignore[import-untyped]
                    sgai_logger,
                )

            else:
                raise ImportError(
                    "`scrapegraph-py` package not found, please run `uv add scrapegraph-py`"
                ) from None

        self.api_key = api_key or os.getenv("SCRAPEGRAPH_API_KEY")
        self._client = Client(api_key=self.api_key)

        if not self.api_key:
            raise ValueError("Scrapegraph API key is required")

        if website_url is not None:
            self._validate_url(website_url)
            self.website_url = website_url
            self.description = f"A tool that uses Scrapegraph AI to intelligently scrape {website_url}'s content."
            self.args_schema = FixedScrapegraphScrapeToolSchema

        if user_prompt is not None:
            self.user_prompt = user_prompt

View on GitHub (pinned to 754d7323be)

Solutions

Install the package: uv add scrapegraph-py (or pip install scrapegraph-py) in the active environment
Verify installation went to the same interpreter the tool runs under
Pre-install in CI/Docker images so the prompt is never reached
For scripted runs, pre-install rather than relying on interactive consent

Example fix

# before
$ python agent.py
# > You are missing the 'scrapegraph-py' package. Would you like to install it? n
# ImportError

# after
$ uv add scrapegraph-py
$ python agent.py

Defensive patterns

Strategy: validation

Validate before calling

import importlib.util, shutil

def scrapegraph_ready() -> bool:
    return importlib.util.find_spec("scrapegraph_py") is not None or shutil.which("uv") is not None

assert scrapegraph_ready(), "Install first: uv add scrapegraph-py"

Try / catch

try:
    tool = ScrapegraphScrapeTool(api_key=key)
except ImportError as e:
    raise SystemExit(f"Missing dependency: {e}") from e

Prevention

Pre-install scrapegraph-py in all environments and CI images
Never depend on the interactive click.confirm path in automation
Ensure uv is installed and on PATH if you expect the auto-install offer to work
Check find_spec('scrapegraph_py') during startup smoke tests

When it happens

Trigger: Constructing ScrapegraphScrapeTool in an environment without scrapegraph-py installed and answering 'n' to the click.confirm prompt — including non-interactive shells where confirm cannot return True.

Common situations: CI/docker runs with piped stdin where the prompt aborts; developers declining auto-install to control dependency versions; uv-based projects where the user prefers manual dependency management.

Related errors

AI-assisted analysis of crewAIInc/crewAI@754d7323be (2026-08-15). Data as JSON: /api/errors/a75b8d1acdf008e6. Report an issue: GitHub.