crewAIInc/crewAI · error · ValueError

URL is too long (max 2048 characters)

Error message

URL is too long (max 2048 characters)

What it means

Pydantic field_validator error from SeleniumScrapingToolSchema when website_url exceeds 2048 characters, the common maximum URL length accepted by browsers and servers. Selenium/Chrome cannot reliably navigate to URLs beyond this size, so the validator rejects them early with a clear limit instead of a cryptic browser failure.

Source

Thrown at lib/crewai-tools/src/crewai_tools/tools/selenium_scraping_tool/selenium_scraping_tool.py:33

    """Input for SeleniumScrapingTool."""

    website_url: str = Field(
        ...,
        description="Mandatory website url to read the file. Must start with http:// or https://",
    )
    css_element: str = Field(
        ...,
        description="Mandatory css reference for element to scrape from the website",
    )

    @field_validator("website_url")
    @classmethod
    def validate_website_url(cls, v: str) -> str:
        if not v:
            raise ValueError("Website URL cannot be empty")

        if len(v) > 2048:  # Common maximum URL length
            raise ValueError("URL is too long (max 2048 characters)")

        if not re.match(r"^https?://", v):
            raise ValueError("URL must start with http:// or https://")

        try:
            result = urlparse(v)
            if not all([result.scheme, result.netloc]):
                raise ValueError("Invalid URL format")
        except Exception as e:
            raise ValueError(f"Invalid URL: {e!s}") from e

        if re.search(r"\s", v):
            raise ValueError("URL cannot contain whitespace")

        return v


class SeleniumScrapingTool(BaseTool):

View on GitHub (pinned to 754d7323be)

Solutions

1. Shorten the URL: move oversized query parameters to a POST body on the target site, or use a shortened/redirect URL.
2. Double-check the value being passed — a >2048-char 'URL' usually means the wrong data (document, JSON) was supplied.
3. If a long encoded parameter is unavoidable, host the payload at a short URL and scrape that.

Example fix

# before
tool = SeleniumScrapingTool(website_url=f"https://api.example.com/view?data={huge_b64}", css_element="body")
#  -> ValueError: URL is too long (max 2048 characters)

# after: host the payload, scrape the short URL
tool = SeleniumScrapingTool(website_url="https://example.com/view/abc123", css_element="body")

MAX_URL = 2048
if len(url) > MAX_URL:
    raise ValueError(f"URL exceeds {MAX_URL} chars — shorten or host the payload elsewhere")

from pydantic import ValidationError

try:
    tool = SeleniumScrapingTool(website_url=url, css_element=css)
except ValidationError as e:
    if "too long" in str(e):
        raise ValueError("URL payload too large for GET scraping; use a shortened link") from e
    raise

When it happens

Trigger: Passing a URL longer than 2048 chars — typically a GET URL carrying a huge query string, an encoded blob/token in the query, or accidentally passing a full document/JSON payload instead of a URL.

Common situations: URLs with long encoded state, JWTs, or base64 payloads in the query string; accidentally passing HTML content or a data URI where a URL was expected; pasting the wrong variable into website_url.

Related errors

• Website URL cannot be empty
• URL must start with http:// or https://
• Invalid URL format
• Invalid URL: {e!s}
• URL cannot contain whitespace

AI-assisted analysis of crewAIInc/crewAI@754d7323be (2026-08-15). Data as JSON: /api/errors/a644ba2befe6a7ad. Report an issue: GitHub.