crewAIInc/crewAI · error · ValueError

Invalid URL: {e!s}

Error message

Invalid URL: {e!s}

What it means

Pydantic field_validator error from SeleniumScrapingToolSchema raised as the except branch of the urlparse try-block: it wraps any unexpected exception raised while parsing the URL (message interpolated as 'Invalid URL: {e}'). In practice the inner raises are ValueErrors re-wrapped here, so the message chains the original parsing failure detail. It indicates the URL string could not be structurally parsed at all.

Source

Thrown at lib/crewai-tools/src/crewai_tools/tools/selenium_scraping_tool/selenium_scraping_tool.py:43

    @field_validator("website_url")
    @classmethod
    def validate_website_url(cls, v: str) -> str:
        if not v:
            raise ValueError("Website URL cannot be empty")

        if len(v) > 2048:  # Common maximum URL length
            raise ValueError("URL is too long (max 2048 characters)")

        if not re.match(r"^https?://", v):
            raise ValueError("URL must start with http:// or https://")

        try:
            result = urlparse(v)
            if not all([result.scheme, result.netloc]):
                raise ValueError("Invalid URL format")
        except Exception as e:
            raise ValueError(f"Invalid URL: {e!s}") from e

        if re.search(r"\s", v):
            raise ValueError("URL cannot contain whitespace")

        return v


class SeleniumScrapingTool(BaseTool):
    name: str = "Read a website content"
    description: str = "A tool that can be used to read a website content."
    args_schema: type[BaseModel] = SeleniumScrapingToolSchema
    website_url: str | None = None
    driver: Any | None = None
    cookie: dict[str, Any] | None = None
    wait_time: int | None = 3
    css_element: str | None = None
    return_html: bool | None = False
    _by: Any | None = None

View on GitHub (pinned to 754d7323be)

Solutions

1. Read the text after 'Invalid URL:' — it carries the underlying parse error identifying the malformed part.
2. Sanitize/percent-encode the URL before passing it (urllib.parse.quote for unsafe components).
3. Log the offending value on ValidationError to find which upstream producer emitted it.

Example fix

# before
tool = SeleniumScrapingTool(website_url=raw_user_input, css_element="article")  # Invalid URL: ...

# after
from urllib.parse import urlparse
parsed = urlparse(raw_user_input.strip())
if not (parsed.scheme in ("http", "https") and parsed.netloc):
    raise ValueError(f"bad url from user: {raw_user_input!r}")
tool = SeleniumScrapingTool(website_url=parsed.geturl(), css_element="article")

from urllib.parse import urlparse

def is_parseable_web_url(v: str) -> bool:
    try:
        p = urlparse(v)
        return p.scheme in ("http", "https") and bool(p.netloc) and not any(ch.isspace() for ch in v)
    except Exception:
        return False

def is_web_url(v) -> bool:
    """Narrow unknown input to a parseable http(s) URL without whitespace."""
    if not isinstance(v, str):
        return False
    try:
        p = urlparse(v.strip())
    except Exception:
        return False
    return p.scheme in ("http", "https") and bool(p.netloc) and not any(c.isspace() for c in v)

from pydantic import ValidationError

try:
    tool = SeleniumScrapingTool(website_url=raw, css_element=css)
except ValidationError as e:
    log.error("rejected url %r: %s", raw, e)  # msg carries the underlying parse reason
    raise

When it happens

Trigger: Passing values to website_url that make urlparse raise rather than return an incomplete result — e.g. certain non-string-like or severely malformed inputs; commonly seen as the wrapper around the 'Invalid URL format' branch when the inner ValueError propagates through the except.

Common situations: Dynamic URL assembly producing control characters or broken percent-encoding; values from untrusted user input that only look like URLs; debugging sessions where the raw exception text under 'Invalid URL:' reveals which parse step failed.

Related errors

• Website URL cannot be empty
• URL is too long (max 2048 characters)
• URL must start with http:// or https://
• Invalid URL format
• URL cannot contain whitespace

AI-assisted analysis of crewAIInc/crewAI@754d7323be (2026-08-15). Data as JSON: /api/errors/9a281435f6794b80. Report an issue: GitHub.