{"record":{"id":"e8717d412d9f7631","repo":"crewAIInc/crewAI","slug":"url-must-start-with-http-or-https","errorCode":null,"errorMessage":"URL must start with http:// or https://","messagePattern":"URL must start with http:// or https://","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"lib/crewai-tools/src/crewai_tools/tools/selenium_scraping_tool/selenium_scraping_tool.py","lineNumber":36,"sourceCode":"        ...,\n        description=\"Mandatory website url to read the file. Must start with http:// or https://\",\n    )\n    css_element: str = Field(\n        ...,\n        description=\"Mandatory css reference for element to scrape from the website\",\n    )\n\n    @field_validator(\"website_url\")\n    @classmethod\n    def validate_website_url(cls, v: str) -> str:\n        if not v:\n            raise ValueError(\"Website URL cannot be empty\")\n\n        if len(v) > 2048:  # Common maximum URL length\n            raise ValueError(\"URL is too long (max 2048 characters)\")\n\n        if not re.match(r\"^https?://\", v):\n            raise ValueError(\"URL must start with http:// or https://\")\n\n        try:\n            result = urlparse(v)\n            if not all([result.scheme, result.netloc]):\n                raise ValueError(\"Invalid URL format\")\n        except Exception as e:\n            raise ValueError(f\"Invalid URL: {e!s}\") from e\n\n        if re.search(r\"\\s\", v):\n            raise ValueError(\"URL cannot contain whitespace\")\n\n        return v\n\n\nclass SeleniumScrapingTool(BaseTool):\n    name: str = \"Read a website content\"\n    description: str = \"A tool that can be used to read a website content.\"\n    args_schema: type[BaseModel] = SeleniumScrapingToolSchema","sourceCodeStart":18,"sourceCodeEnd":54,"githubUrl":"https://github.com/crewAIInc/crewAI/blob/754d7323beb2fd042e33444a115ea2d5a47193f0/lib/crewai-tools/src/crewai_tools/tools/selenium_scraping_tool/selenium_scraping_tool.py#L18-L54","documentation":"Pydantic field_validator error from SeleniumScrapingToolSchema when website_url does not begin with http:// or https:// (regex ^https?://). Selenium's Chrome driver can technically open file:// or other schemes, but this tool restricts scraping to web URLs. It fires at schema-validation time, before any driver is launched.","triggerScenarios":"Passing 'example.com' (no scheme), 'ftp://server/file', 'www.example.com/page', or a local path '/tmp/page.html' as website_url.","commonSituations":"User-supplied URLs missing the scheme (very common — people type bare domains); config storing URLs without protocol; accidentally passing a file path intended for local scraping; an LLM agent emitting a bare domain.","solutions":["Include the scheme: website_url='https://example.com/page'.","Normalize input before constructing the tool: prepend 'https://' when no scheme is present.","If you need local-file scraping, use Selenium directly — this tool intentionally disallows it."],"exampleFix":"# before\ntool = SeleniumScrapingTool(website_url=\"example.com/article\", css_element=\"article\")\n\n# after\nurl = url if url.startswith((\"http://\", \"https://\")) else f\"https://{url}\"\ntool = SeleniumScrapingTool(website_url=url, css_element=\"article\")","handlingStrategy":"validation","validationCode":"def with_scheme(url: str) -> str:\n    url = url.strip()\n    if not url.startswith((\"http://\", \"https://\")):\n        url = \"https://\" + url\n    return url\n\ntool = SeleniumScrapingTool(website_url=with_scheme(raw_url), css_element=\"article\")","typeGuard":null,"tryCatchPattern":"from pydantic import ValidationError\n\ntry:\n    tool = SeleniumScrapingTool(website_url=url, css_element=css)\nexcept ValidationError as e:\n    if \"must start with\" in str(e):\n        tool = SeleniumScrapingTool(website_url=with_scheme(url), css_element=css)\n    else:\n        raise","preventionTips":["Normalize bare domains by prepending https:// before tool construction.","Users and LLMs frequently omit the scheme — make normalization a standard step in URL ingestion."],"tags":["validation","url","selenium","scheme"],"backgroundTag":null,"analyzedSha":"754d7323beb2fd042e33444a115ea2d5a47193f0","analyzedAt":"2026-08-15T04:06:56.746Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}