crewAIInc/crewAI · error · ValueError

Invalid URL

Error message

Invalid URL: {e!s}

What it means

Pydantic field_validator error from SeleniumScrapingToolSchema raised as the except branch of the urlparse try-block: it wraps any unexpected exception raised while parsing the URL (message interpolated as 'Invalid URL: {e}'). In practice the inner raises are ValueErrors re-wrapped here, so the message chains the original parsing failure detail. It indicates the URL string could not be structurally parsed at all.

Solutions

1. Read the text after 'Invalid URL:' — it carries the underlying parse error identifying the malformed part.
2. Sanitize/percent-encode the URL before passing it (urllib.parse.quote for unsafe components).
3. Log the offending value on ValidationError to find which upstream producer emitted it.

Example fix

# before
tool = SeleniumScrapingTool(website_url=raw_user_input, css_element="article")  # Invalid URL: ...

# after
from urllib.parse import urlparse
parsed = urlparse(raw_user_input.strip())
if not (parsed.scheme in ("http", "https") and parsed.netloc):
    raise ValueError(f"bad url from user: {raw_user_input!r}")
tool = SeleniumScrapingTool(website_url=parsed.geturl(), css_element="article")

from urllib.parse import urlparse

def is_parseable_web_url(v: str) -> bool:
    try:
        p = urlparse(v)
        return p.scheme in ("http", "https") and bool(p.netloc) and not any(ch.isspace() for ch in v)
    except Exception:
        return False

def is_web_url(v) -> bool:
    """Narrow unknown input to a parseable http(s) URL without whitespace."""
    if not isinstance(v, str):
        return False
    try:
        p = urlparse(v.strip())
    except Exception:
        return False
    return p.scheme in ("http", "https") and bool(p.netloc) and not any(c.isspace() for c in v)

from pydantic import ValidationError

try:
    tool = SeleniumScrapingTool(website_url=raw, css_element=css)
except ValidationError as e:
    log.error("rejected url %r: %s", raw, e)  # msg carries the underlying parse reason
    raise

When it happens

Trigger: Passing values to website_url that make urlparse raise rather than return an incomplete result — e.g. certain non-string-like or severely malformed inputs; commonly seen as the wrapper around the 'Invalid URL format' branch when the inner ValueError propagates through the except.

Common situations: Dynamic URL assembly producing control characters or broken percent-encoding; values from untrusted user input that only look like URLs; debugging sessions where the raw exception text under 'Invalid URL:' reveals which parse step failed.

Related errors

• Invalid URL format
• URL cannot be empty
• URL cannot contain whitespace
• URL is too long (max 2048 characters)
• URL must start with http:// or https://

AI-assisted analysis of crewAIInc/crewAI@754d7323be (2026-08-15). Data as JSON: /api/errors/9a281435f6794b80. Report an issue: GitHub.

Appendix: source

Thrown at lib/crewai-tools/src/crewai_tools/tools/selenium_scraping_tool/selenium_scraping_tool.py:43

    @field_validator("website_url")
    @classmethod
    def validate_website_url(cls, v: str) -> str:
        if not v:
            raise ValueError("Website URL cannot be empty")

        if len(v) > 2048:  # Common maximum URL length
            raise ValueError("URL is too long (max 2048 characters)")

        if not re.match(r"^https?://", v):
            raise ValueError("URL must start with http:// or https://")

        try:
            result = urlparse(v)
            if not all([result.scheme, result.netloc]):
                raise ValueError("Invalid URL format")
        except Exception as e:
            raise ValueError(f"Invalid URL: {e!s}") from e

        if re.search(r"\s", v):
            raise ValueError("URL cannot contain whitespace")

        return v


class SeleniumScrapingTool(BaseTool):
    name: str = "Read a website content"
    description: str = "A tool that can be used to read a website content."
    args_schema: type[BaseModel] = SeleniumScrapingToolSchema
    website_url: str | None = None
    driver: Any | None = None
    cookie: dict[str, Any] | None = None
    wait_time: int | None = 3
    css_element: str | None = None
    return_html: bool | None = False
    _by: Any | None = None

View on GitHub (pinned to 754d7323be)