{"record":{"id":"ee8ce018792faeb1","repo":"unclecode/crawl4ai","slug":"either-html-or-url-must-be-provided","errorCode":null,"errorMessage":"Either 'html' or 'url' must be provided","messagePattern":"Either 'html' or 'url' must be provided","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"crawl4ai/extraction_strategy.py","lineNumber":1813,"sourceCode":" validate (bool): If True, validate the schema against the HTML and\n refine via LLM feedback loop. Defaults to False (zero overhead).\n max_refinements (int): Max refinement rounds when validate=True. Defaults to 3.\n usage (TokenUsage, optional): Token usage accumulator. If provided,\n token counts from all LLM calls (including inference and\n validation retries) are added to it in-place.\n **kwargs: Additional args passed to LLM processor.\n\n Returns:\n dict: Generated schema following the JsonElementExtractionStrategy format.\n\n Raises:\n ValueError: If neither html nor url is provided.\n \"\"\"\n from .utils import aperform_completion_with_backoff, preprocess_html_for_schema\n\n # Validate inputs\n if html is None and (url is None or (isinstance(url, list) and len(url) == 0)):\n raise ValueError(\"Either 'html' or 'url' must be provided\")\n\n # Check deprecated parameters\n for name, message in JsonElementExtractionStrategy._GENERATE_SCHEMA_UNWANTED_PROPS.items():\n if locals()[name] is not None:\n raise AttributeError(f\"Setting '{name}' is deprecated. {message}\")\n\n if llm_config is None:\n llm_config = create_llm_config()\n\n # Save original HTML(s) before preprocessing (for validation against real HTML)\n original_htmls = []\n\n # Fetch HTML from URL(s) if provided\n if url is not None:\n from .async_webcrawler import AsyncWebCrawler\n from .async_configs import BrowserConfig, CrawlerRunConfig, CacheMode\n\n browser_config = BrowserConfig(","sourceCodeStart":1795,"sourceCodeEnd":1831,"githubUrl":"https://github.com/unclecode/crawl4ai/blob/7e801521428ee12509994d39151006f64055ebe3/crawl4ai/extraction_strategy.py#L1795-L1831","documentation":"Raised by JsonElementExtractionStrategy.generate_schema when both html is None and url is None (or url is an empty list). The schema generator needs at least one page of real HTML to infer selectors from, so it refuses to run without sample content.","triggerScenarios":"Calling generate_schema() with no arguments; passing url=[] (an empty list passes the isinstance check and len==0 branch); passing url=None and forgetting html; conditionally building arguments where both branches end up None.","commonSituations":"Dynamic pipelines where the URL list was filtered to empty before the call; refactoring code that used to pass html but now passes a variable that is None on some paths; notebook prototyping with placeholder arguments.","solutions":["Pass either html=\"...\" or a non-empty url / list of URLs","If loading content dynamically, assert the value is truthy before calling generate_schema","When accepting user input, validate that at least one of the two parameters is populated"],"exampleFix":"// before\nschema = await JsonElementExtractionStrategy.generate_schema() # ValueError\n\n// after\nschema = await JsonElementExtractionStrategy.generate_schema(url=\"https://example.com/products\")","handlingStrategy":"validation","validationCode":"has_html = bool(html and html.strip())\nhas_url = bool(url) and not (isinstance(url, list) and len(url) == 0)\nif not (has_html or has_url):\n raise ValueError(\"provide html= or a non-empty url= before generating a schema\")","typeGuard":"def has_schema_input(html, url) -> bool:\n if html and html.strip():\n return True\n if isinstance(url, str) and url:\n return True\n return isinstance(url, list) and len(url) > 0","tryCatchPattern":null,"preventionTips":["Assert at least one of html/url is truthy before calling generate_schema","Beware url=[] — an empty list is treated as missing","Log which input path supplied the sample HTML in dynamic pipelines"],"tags":["extraction","schema-generation","validation"],"backgroundTag":null,"analyzedSha":"7e801521428ee12509994d39151006f64055ebe3","analyzedAt":"2026-08-14T20:46:20.673Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}