{"record":{"id":"c20784f6731b879d","repo":"crewAIInc/crewAI","slug":"both-website-url-and-css-element-must-be-provided","errorCode":null,"errorMessage":"Both website_url and css_element must be provided.","messagePattern":"Both website_url and css_element must be provided\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"lib/crewai-tools/src/crewai_tools/tools/scrape_element_from_website/scrape_element_from_website.py","lineNumber":83,"sourceCode":"            self.args_schema = FixedScrapeElementFromWebsiteToolSchema\n            self._generate_description()\n            if cookies is not None:\n                self.cookies = {cookies[\"name\"]: os.getenv(cookies[\"value\"]) or \"\"}\n\n    def _run(\n        self,\n        **kwargs: Any,\n    ) -> Any:\n        if not BEAUTIFULSOUP_AVAILABLE:\n            raise ImportError(\n                \"beautifulsoup4 is not installed. Please install it with `pip install crewai-tools[beautifulsoup4]`\"\n            )\n\n        website_url = kwargs.get(\"website_url\", self.website_url)\n        css_element = kwargs.get(\"css_element\", self.css_element)\n\n        if website_url is None or css_element is None:\n            raise ValueError(\"Both website_url and css_element must be provided.\")\n\n        page = safe_get(\n            website_url,\n            headers=self.headers,\n            cookies=self.cookies if self.cookies else {},\n            timeout=30,\n        )\n        parsed = BeautifulSoup(page.content, \"html.parser\")\n        elements = parsed.select(css_element)\n        return \"\\n\".join([element.get_text() for element in elements])\n","sourceCodeStart":65,"sourceCodeEnd":94,"githubUrl":"https://github.com/crewAIInc/crewAI/blob/754d7323beb2fd042e33444a115ea2d5a47193f0/lib/crewai-tools/src/crewai_tools/tools/scrape_element_from_website/scrape_element_from_website.py#L65-L94","documentation":"Raised by ScrapeElementFromWebsiteTool._run when either website_url or css_element ends up None. Values come from kwargs first, then the instance attributes set at construction; if the tool was created bare (no website_url) and the call omitted either key, this ValueError fires before any HTTP request.","triggerScenarios":"ScrapeElementFromWebsiteTool() instantiated without website_url, then run with only one of website_url/css_element; or a schema mismatch where the agent/caller passes differently-named keys so kwargs.get returns None.","commonSituations":"LLM agents omitting the css_element argument because the tool description didn't make it mandatory; renaming of args between versions; constructing with website_url but calling with 'url' instead of 'website_url'.","solutions":["Pass both website_url and css_element explicitly to run(): tool.run(website_url='https://example.com', css_element='div.main')","Or set them at construction: ScrapeElementFromWebsiteTool(website_url='https://example.com', css_element='div.main')","Verify the exact expected kwarg names against FixedScrapeElementFromWebsiteToolSchema","When an agent drives the tool, make both fields required in the tool schema/description"],"exampleFix":"# before\ntool = ScrapeElementFromWebsiteTool()\ntool.run(website_url='https://example.com')  # css_element None -> ValueError\n\n# after\ntool.run(website_url='https://example.com', css_element='div.content')\n","handlingStrategy":"validation","validationCode":"def valid_scrape_args(kwargs: dict) -> bool:\n    return bool(kwargs.get(\"website_url\")) and bool(kwargs.get(\"css_element\"))\n\nassert valid_scrape_args(call_kwargs), \"website_url and css_element are both required\"","typeGuard":"from typing import TypedDict\n\nclass ScrapeElementArgs(TypedDict, total=True):\n    website_url: str\n    css_element: str","tryCatchPattern":"try:\n    tool.run(website_url=url, css_element=sel)\nexcept ValueError as e:\n    if \"must be provided\" in str(e):\n        raise ValueError(\"ScrapeElementFromWebsiteTool requires both website_url and css_element\") from e\n    raise","preventionTips":["Always supply both website_url and css_element (constructor or run kwargs)","Make both fields required in the agent-facing tool schema","Use the exact kwarg names from FixedScrapeElementFromWebsiteToolSchema","Default css_element at construction when scraping a known site layout"],"tags":["validation","missing-argument","scraper","css-selector"],"backgroundTag":null,"analyzedSha":"754d7323beb2fd042e33444a115ea2d5a47193f0","analyzedAt":"2026-08-15T04:06:56.746Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}