crewAIInc/crewAI · error · ValueError

Invalid URL format

Error message

Invalid URL format

What it means

Pydantic field_validator error from SeleniumScrapingToolSchema raised when urlparse(v) succeeds but the parsed URL lacks a scheme or netloc component (all([result.scheme, result.netloc]) is False). This catches URLs that pass the earlier http/https prefix check but are structurally incomplete — most notably 'https://' with no host, or malformed inputs the regex prefix happened to accept.

Source

Thrown at lib/crewai-tools/src/crewai_tools/tools/selenium_scraping_tool/selenium_scraping_tool.py:41

        description="Mandatory css reference for element to scrape from the website",
    )

    @field_validator("website_url")
    @classmethod
    def validate_website_url(cls, v: str) -> str:
        if not v:
            raise ValueError("Website URL cannot be empty")

        if len(v) > 2048:  # Common maximum URL length
            raise ValueError("URL is too long (max 2048 characters)")

        if not re.match(r"^https?://", v):
            raise ValueError("URL must start with http:// or https://")

        try:
            result = urlparse(v)
            if not all([result.scheme, result.netloc]):
                raise ValueError("Invalid URL format")
        except Exception as e:
            raise ValueError(f"Invalid URL: {e!s}") from e

        if re.search(r"\s", v):
            raise ValueError("URL cannot contain whitespace")

        return v


class SeleniumScrapingTool(BaseTool):
    name: str = "Read a website content"
    description: str = "A tool that can be used to read a website content."
    args_schema: type[BaseModel] = SeleniumScrapingToolSchema
    website_url: str | None = None
    driver: Any | None = None
    cookie: dict[str, Any] | None = None
    wait_time: int | None = 3
    css_element: str | None = None

View on GitHub (pinned to 754d7323be)

Solutions

1. Ensure the URL has a real hostname: 'https://example.com/page', not 'https:///page'.
2. If constructing URLs from parts, validate each component (host non-empty) before joining.
3. Test the URL with urllib.parse.urlparse yourself and require both scheme and netloc.

Example fix

# before
url = f"https://{host}/page"  # host == "" -> "https:///page"
tool = SeleniumScrapingTool(website_url=url, css_element="article")

# after
if not host:
    raise ValueError("host must not be empty")
url = f"https://{host}/page"
tool = SeleniumScrapingTool(website_url=url, css_element="article")

from urllib.parse import urlparse

p = urlparse(url)
if not (p.scheme in ("http", "https") and p.netloc):
    raise ValueError(f"URL missing host: {url!r}")

from pydantic import ValidationError

try:
    tool = SeleniumScrapingTool(website_url=url, css_element=css)
except ValidationError as e:
    if "Invalid URL format" in str(e):
        raise ValueError(f"assemble URLs with a non-empty host: got {url!r}") from e
    raise

When it happens

Trigger: Passing 'https:///path' (empty host), 'https://?q=1', or values where urlparse yields an empty netloc despite an https:// prefix; note the inner raise is inside a try whose except wraps it as 'Invalid URL: {e}' only if an exception propagates — the direct raise produces this exact message.

Common situations: String-building URLs with a bug that drops the host (f'https://{host}{path}' with empty host); truncated URLs from copy-paste; URLs assembled from config where the domain variable is empty.

Related errors

• Website URL cannot be empty
• URL is too long (max 2048 characters)
• URL must start with http:// or https://
• Invalid URL: {e!s}
• URL cannot contain whitespace

AI-assisted analysis of crewAIInc/crewAI@754d7323be (2026-08-15). Data as JSON: /api/errors/3b249ff4516c00af. Report an issue: GitHub.