{"record":{"id":"9e32bc539271cf14","repo":"crewAIInc/crewAI","slug":"website-url-must-be-provided","errorCode":null,"errorMessage":"Website URL must be provided.","messagePattern":"Website URL must be provided\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"lib/crewai-tools/src/crewai_tools/tools/scrape_website_tool/scrape_website_tool.py","lineNumber":75,"sourceCode":"            )\n\n        if website_url is not None:\n            self.website_url = website_url\n            self.description = (\n                f\"A tool that can be used to read {website_url}'s content.\"\n            )\n            self.args_schema = FixedScrapeWebsiteToolSchema\n            self._generate_description()\n            if cookies is not None:\n                self.cookies = {cookies[\"name\"]: os.getenv(cookies[\"value\"]) or \"\"}\n\n    def _run(\n        self,\n        **kwargs: Any,\n    ) -> Any:\n        website_url: str | None = kwargs.get(\"website_url\", self.website_url)\n        if website_url is None:\n            raise ValueError(\"Website URL must be provided.\")\n\n        page = safe_get(\n            website_url,\n            timeout=15,\n            headers=self.headers,\n            cookies=self.cookies if self.cookies else {},\n        )\n\n        page.encoding = page.apparent_encoding\n        parsed = BeautifulSoup(page.text, \"html.parser\")\n\n        text = \"The following text is scraped website content:\\n\\n\"\n        text += parsed.get_text(\" \")\n        text = re.sub(\"[ \\t]+\", \" \", text)\n        return re.sub(\"\\\\s+\\n\\\\s+\", \"\\n\", text)\n","sourceCodeStart":57,"sourceCodeEnd":91,"githubUrl":"https://github.com/crewAIInc/crewAI/blob/754d7323beb2fd042e33444a115ea2d5a47193f0/lib/crewai-tools/src/crewai_tools/tools/scrape_website_tool/scrape_website_tool.py#L57-L91","documentation":"Raised by ScrapeWebsiteTool._run when website_url resolves to None — kwargs.get('website_url', self.website_url) found neither a call-time argument nor an instance attribute. It fails before any HTTP request, and unlike the element scraper this tool has no other required parameters.","triggerScenarios":"ScrapeWebsiteTool() constructed bare and run without a website_url kwarg; or calling with a misspelled/differently named key (e.g. 'url') so kwargs.get returns None and no default was set.","commonSituations":"Agents omitting the URL argument; tools constructed without the URL and expected to be parameterized per-call; version changes to the expected argument name; schema drift between the tool description and the LLM's emitted arguments.","solutions":["Pass website_url at run time: tool.run(website_url='https://example.com')","Or fix it at construction: ScrapeWebsiteTool(website_url='https://example.com')","Confirm the kwarg name is exactly website_url per FixedScrapeWebsiteToolSchema","Make the URL field required in the agent-facing tool schema so the LLM always supplies it"],"exampleFix":"# before\ntool = ScrapeWebsiteTool()\ntool.run()  # ValueError: Website URL must be provided.\n\n# after\ntool.run(website_url='https://example.com')\n","handlingStrategy":"validation","validationCode":"from urllib.parse import urlparse\n\ndef valid_target(url: str | None) -> bool:\n    return isinstance(url, str) and bool(urlparse(url).scheme)\n\nassert valid_target(target_url), \"website_url (with scheme) is required\"","typeGuard":null,"tryCatchPattern":"try:\n    tool.run(website_url=url)\nexcept ValueError as e:\n    if \"must be provided\" in str(e):\n        raise ValueError(\"website_url missing — pass it to ScrapeWebsiteTool(...) or run(website_url=...)\") from e\n    raise","preventionTips":["Set website_url at construction when it is known and static","Pass website_url explicitly on every run() call otherwise","Use the exact kwarg name 'website_url'","Make the URL required in the agent tool schema so the LLM always emits it"],"tags":["validation","missing-argument","scraper","url"],"backgroundTag":null,"analyzedSha":"754d7323beb2fd042e33444a115ea2d5a47193f0","analyzedAt":"2026-08-15T04:06:56.746Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}