crewAIInc/crewAI · error · ValueError

URL must start with http:// or https://

Error message

URL must start with http:// or https://

What it means

Pydantic field_validator error from SeleniumScrapingToolSchema when website_url does not begin with http:// or https:// (regex ^https?://). Selenium's Chrome driver can technically open file:// or other schemes, but this tool restricts scraping to web URLs. It fires at schema-validation time, before any driver is launched.

Source

Thrown at lib/crewai-tools/src/crewai_tools/tools/selenium_scraping_tool/selenium_scraping_tool.py:36

        ...,
        description="Mandatory website url to read the file. Must start with http:// or https://",
    )
    css_element: str = Field(
        ...,
        description="Mandatory css reference for element to scrape from the website",
    )

    @field_validator("website_url")
    @classmethod
    def validate_website_url(cls, v: str) -> str:
        if not v:
            raise ValueError("Website URL cannot be empty")

        if len(v) > 2048:  # Common maximum URL length
            raise ValueError("URL is too long (max 2048 characters)")

        if not re.match(r"^https?://", v):
            raise ValueError("URL must start with http:// or https://")

        try:
            result = urlparse(v)
            if not all([result.scheme, result.netloc]):
                raise ValueError("Invalid URL format")
        except Exception as e:
            raise ValueError(f"Invalid URL: {e!s}") from e

        if re.search(r"\s", v):
            raise ValueError("URL cannot contain whitespace")

        return v


class SeleniumScrapingTool(BaseTool):
    name: str = "Read a website content"
    description: str = "A tool that can be used to read a website content."
    args_schema: type[BaseModel] = SeleniumScrapingToolSchema

View on GitHub (pinned to 754d7323be)

Solutions

1. Include the scheme: website_url='https://example.com/page'.
2. Normalize input before constructing the tool: prepend 'https://' when no scheme is present.
3. If you need local-file scraping, use Selenium directly — this tool intentionally disallows it.

Example fix

# before
tool = SeleniumScrapingTool(website_url="example.com/article", css_element="article")

# after
url = url if url.startswith(("http://", "https://")) else f"https://{url}"
tool = SeleniumScrapingTool(website_url=url, css_element="article")

def with_scheme(url: str) -> str:
    url = url.strip()
    if not url.startswith(("http://", "https://")):
        url = "https://" + url
    return url

tool = SeleniumScrapingTool(website_url=with_scheme(raw_url), css_element="article")

from pydantic import ValidationError

try:
    tool = SeleniumScrapingTool(website_url=url, css_element=css)
except ValidationError as e:
    if "must start with" in str(e):
        tool = SeleniumScrapingTool(website_url=with_scheme(url), css_element=css)
    else:
        raise

When it happens

Trigger: Passing 'example.com' (no scheme), 'ftp://server/file', 'www.example.com/page', or a local path '/tmp/page.html' as website_url.

Common situations: User-supplied URLs missing the scheme (very common — people type bare domains); config storing URLs without protocol; accidentally passing a file path intended for local scraping; an LLM agent emitting a bare domain.

Related errors

• Website URL cannot be empty
• URL is too long (max 2048 characters)
• Invalid URL format
• Invalid URL: {e!s}
• URL cannot contain whitespace

AI-assisted analysis of crewAIInc/crewAI@754d7323be (2026-08-15). Data as JSON: /api/errors/e8717d412d9f7631. Report an issue: GitHub.