crewAIInc/crewAI · error · ImportError

`scrapfly-sdk` package not found, please run `uv add scrapfl

Error message

`scrapfly-sdk` package not found, please run `uv add scrapfly-sdk`

What it means

ImportError raised by ScrapflyScrapeWebsiteTool when the scrapfly-sdk package is not importable and the user declines the interactive install prompt (click.confirm). The tool asks 'Would you like to install it?' and runs `uv add scrapfly-sdk`; if you answer 'no' (or stdin is non-interactive so confirm fails/is declined), the ImportError is raised with the install command in the message. It fires during tool construction, before any scraping happens.

Source

Thrown at lib/crewai-tools/src/crewai_tools/tools/scrapfly_scrape_website_tool/scrapfly_scrape_website_tool.py:63

    def __init__(self, api_key: str):
        super().__init__(
            name="Scrapfly web scraping API tool",
            description="Scrape a webpage url using Scrapfly and return its content as markdown or text",
        )
        try:
            from scrapfly import ScrapflyClient  # type: ignore[import-untyped]
        except ImportError:
            import click

            if click.confirm(
                "You are missing the 'scrapfly-sdk' package. Would you like to install it?"
            ):
                import subprocess

                subprocess.run(["uv", "add", "scrapfly-sdk"], check=True)  # noqa: S607
            else:
                raise ImportError(
                    "`scrapfly-sdk` package not found, please run `uv add scrapfly-sdk`"
                ) from None
        self.scrapfly = ScrapflyClient(key=api_key or os.getenv("SCRAPFLY_API_KEY"))

    def _run(
        self,
        url: str,
        scrape_format: str = "markdown",
        scrape_config: dict[str, Any] | None = None,
        ignore_scrape_failures: bool | None = None,
    ) -> str | None:
        from scrapfly import ScrapeConfig

        url = validate_url(url)
        scrape_config = scrape_config if scrape_config is not None else {}
        try:
            response = self.scrapfly.scrape(  # type: ignore[union-attr]
                ScrapeConfig(url, format=scrape_format, **scrape_config)

View on GitHub (pinned to 754d7323be)

Solutions

1. Install the dependency before constructing the tool: run `uv add scrapfly-sdk` (or `pip install scrapfly-sdk`).
2. For Docker/CI, add scrapfly-sdk to the image/requirements so the interactive prompt never triggers.
3. Pre-set SCRAPFLY_API_KEY as well, since the constructor reads it right after the import.

Example fix

# before (CI, no TTY -> prompt auto-declined)
tool = ScrapflyScrapeWebsiteTool()  # ImportError

# after
# Dockerfile / CI step:
#   uv add scrapfly-sdk
tool = ScrapflyScrapeWebsiteTool()

def scrapfly_available() -> bool:
    try:
        import scrapfly  # noqa: F401
        return True
    except ImportError:
        return False

if not scrapfly_available():
    subprocess.run(["uv", "add", "scrapfly-sdk"], check=True)

try:
    tool = ScrapflyScrapeWebsiteTool()
except ImportError as e:
    if "scrapfly-sdk" in str(e):
        raise SystemExit("Run `uv add scrapfly-sdk` before using ScrapflyScrapeWebsiteTool") from e
    raise

When it happens

Trigger: Instantiating ScrapflyScrapeWebsiteTool without scrapfly-sdk installed: ImportError on 'from scrapfly import ScrapflyClient', the click.confirm prompt appears, and the user declines (or the environment auto-declines).

Common situations: Non-interactive environments (CI, Docker, agent runs) where click.confirm cannot prompt and defaults to no; forgetting to add the optional dependency after pulling in crewai-tools; deploying to a slim container that never had scrapfly-sdk.

Related errors

• `selenium` and `webdriver-manager` package not found, please
• `serpapi` package not found, please install with `uv add ser
• The 'tavily-python' package is required. 'click' and 'subpro
• The 'tavily-python' package is required. 'click' and 'subpro
• The 'tavily-python' package is required. 'click' and 'subpro

AI-assisted analysis of crewAIInc/crewAI@754d7323be (2026-08-15). Data as JSON: /api/errors/300287fb2db6afb7. Report an issue: GitHub.