crewAIInc/crewAI · error · ValueError

url is required either in constructor or method call

Error message

url is required either in constructor or method call

What it means

Raised in BrightDataDatasetTool._run when the url parameter resolves to falsy both at call-site level and constructor level (url = url or self.url). The Bright Data dataset API is URL-driven — it tells the dataset which page to scrape — so a missing URL is rejected before validation of anything else downstream.

Source

Thrown at lib/crewai-tools/src/crewai_tools/tools/brightdata_tool/brightdata_dataset.py:570

        url: str | None = None,
        dataset_type: str | None = None,
        format: str | None = None,
        zipcode: str | None = None,
        additional_params: dict[str, Any] | None = None,
        **kwargs: Any,
    ) -> Any:
        dataset_type = dataset_type or self.dataset_type
        output_format = format or self.format
        url = url or self.url
        zipcode = zipcode or self.zipcode
        additional_params = additional_params or self.additional_params

        if not dataset_type:
            raise ValueError(
                "dataset_type is required either in constructor or method call"
            )
        if not url:
            raise ValueError("url is required either in constructor or method call")

        valid_output_formats = {"json", "ndjson", "jsonl", "csv"}
        if output_format not in valid_output_formats:
            raise ValueError(
                f"Unsupported output format: {output_format}. Must be one of {', '.join(valid_output_formats)}."
            )

        api_key = os.getenv("BRIGHT_DATA_API_KEY")
        if not api_key:
            raise ValueError("BRIGHT_DATA_API_KEY environment variable is required.")

        try:
            return asyncio.run(
                self.get_dataset_data_async(
                    dataset_type=dataset_type,
                    output_format=output_format,
                    url=url,
                    zipcode=zipcode,

View on GitHub (pinned to 754d7323be)

Solutions

1. Pass the target page URL: tool.run(url='https://www.amazon.com/dp/B0XXXX', dataset_type='amazon_product').
2. Or set it once in the constructor: BrightDataDatasetTool(dataset_type='amazon_product', url=product_url).
3. Ensure the URL is a fully qualified http(s) URL string, not empty.

Example fix

# before
result = tool.run(dataset_type='amazon_product')

# after
result = tool.run(dataset_type='amazon_product', url='https://www.amazon.com/dp/B0XXXX')

def validate_target_url(url: str | None) -> str:
    if not url or not url.startswith(("http://", "https://")):
        raise ValueError("A fully qualified http(s) URL is required for Bright Data datasets")
    return url

url = validate_target_url(url or tool.url)
result = tool.run(url=url, dataset_type='amazon_product')

def is_runnable_dataset_call(kwargs: dict, tool) -> bool:
    return bool((kwargs.get("url") or tool.url) and (kwargs.get("dataset_type") or tool.dataset_type))

try:
    result = tool.run(dataset_type='amazon_product')
except ValueError as e:
    if "url is required" in str(e):
        result = tool.run(dataset_type='amazon_product', url=get_url_from_task())
    else:
        raise

When it happens

Trigger: Calling tool.run(dataset_type='amazon_product') with no url anywhere; passing url='' or url=None explicitly; constructor built with only dataset_type and format.

Common situations: Assuming dataset_type alone triggers a discovery job (it does not — this tool always needs a target URL), agents dropping the url argument, config templates that leave url blank.

Related errors

• url is required either in constructor or method call
• dataset_type is required either in constructor or method cal
• query is required either in constructor or method call
• Invalid URL scheme: {self.url}
• URL scheme must be 'http' or 'https'

AI-assisted analysis of crewAIInc/crewAI@754d7323be (2026-08-15). Data as JSON: /api/errors/aba29f87b72438f8. Report an issue: GitHub.