crewAIInc/crewAI · error · ValueError

Scrapegraph API key is required

Error message

Scrapegraph API key is required

What it means

Raised by ScrapegraphScrapeTool's constructor when neither an api_key argument nor the SCRAPEGRAPH_API_KEY environment variable yields a key. Note the source order: self.api_key is resolved, a Client is constructed with it, and only then is the emptiness check performed — so the check fires after client creation but before the tool becomes usable.

Source

Thrown at lib/crewai-tools/src/crewai_tools/tools/scrapegraph_scrape_tool/scrapegraph_scrape_tool.py:117

            ):
                import subprocess

                subprocess.run(["uv", "add", "scrapegraph-py"], check=True)  # noqa: S607
                from scrapegraph_py import Client  # type: ignore[import-untyped]
                from scrapegraph_py.logger import (  # type: ignore[import-untyped]
                    sgai_logger,
                )

            else:
                raise ImportError(
                    "`scrapegraph-py` package not found, please run `uv add scrapegraph-py`"
                ) from None

        self.api_key = api_key or os.getenv("SCRAPEGRAPH_API_KEY")
        self._client = Client(api_key=self.api_key)

        if not self.api_key:
            raise ValueError("Scrapegraph API key is required")

        if website_url is not None:
            self._validate_url(website_url)
            self.website_url = website_url
            self.description = f"A tool that uses Scrapegraph AI to intelligently scrape {website_url}'s content."
            self.args_schema = FixedScrapegraphScrapeToolSchema

        if user_prompt is not None:
            self.user_prompt = user_prompt

        if self.enable_logging:
            sgai_logger.set_logging(level="INFO")

    @staticmethod
    def _validate_url(url: str) -> None:
        """Validate URL format."""
        try:
            result = urlparse(url)

View on GitHub (pinned to 754d7323be)

Solutions

1. Set the environment variable: export SCRAPEGRAPH_API_KEY=your_key, then rerun
2. Or pass it explicitly: ScrapegraphScrapeTool(api_key='your_key')
3. Load your .env before tool construction (load_dotenv()) and verify with os.environ.get('SCRAPEGRAPH_API_KEY')
4. Get a key from the Scrapegraph AI dashboard if you don't have one yet

Example fix

# before
tool = ScrapegraphScrapeTool()  # ValueError: API key is required

# after
import os
tool = ScrapegraphScrapeTool(api_key=os.environ['SCRAPEGRAPH_API_KEY'])

import os

def has_scrapegraph_key() -> bool:
    return bool(os.environ.get("SCRAPEGRAPH_API_KEY"))

assert has_scrapegraph_key(), "Set SCRAPEGRAPH_API_KEY or pass api_key="

try:
    tool = ScrapegraphScrapeTool(api_key=os.environ.get("SCRAPEGRAPH_API_KEY"))
except ValueError as e:
    if "API key" in str(e):
        raise SystemExit("Missing SCRAPEGRAPH_API_KEY — get one at scrapegraph.ai and export it") from e
    raise

When it happens

Trigger: ScrapegraphScrapeTool() (or with website_url) created while SCRAPEGRAPH_API_KEY is unset/empty and no api_key argument is given — common in fresh shells, CI, or when the key was exported in a different terminal session.

Common situations: Forgot to export SCRAPEGRAPH_API_KEY; key stored in an unloaded .env file; environment-specific shells (direnv not activated); typo in the variable name.

Related errors

• API key must be provided either through constructor or MINDS
• BRAVE_API_KEY environment variable is required for BraveSear
• BRIGHT_DATA_API_KEY environment variable is required.
• You must pass oxylabs username and password when instantiati
• You must pass oxylabs username and password when instantiati

AI-assisted analysis of crewAIInc/crewAI@754d7323be (2026-08-15). Data as JSON: /api/errors/91be094292661a25. Report an issue: GitHub.