crewAIInc/crewAI · error · BrightDataDatasetToolException

Trigger failed: {await trigger_response.text()}

Error message

Trigger failed: {await trigger_response.text()}

What it means

A BrightDataDatasetToolException raised when the POST to {BRIGHTDATA_API_URL}/datasets/v3/trigger returns a non-200 status. This is the first HTTP step of the trigger-then-poll workflow; failure means Bright Data rejected the job submission itself (auth, bad dataset payload, quota), and the response body text plus status code are attached.

Source

Thrown at lib/crewai-tools/src/crewai_tools/tools/brightdata_tool/brightdata_dataset.py:506

        dataset_id = ""
        dataset = self.filter_dataset_by_id(dataset_type)

        if len(dataset) == 1:
            dataset_id = dataset[0]["dataset_id"]
        else:
            raise ValueError(
                f"Unable to find the dataset for {dataset_type}. Please make sure to pass a valid one"
            )

        async with aiohttp.ClientSession() as session:
            async with session.post(
                f"{BRIGHTDATA_API_URL}/datasets/v3/trigger",
                params={"dataset_id": dataset_id, "include_errors": "true"},
                json=[request_data],
                headers=headers,
            ) as trigger_response:
                if trigger_response.status != 200:
                    raise BrightDataDatasetToolException(
                        f"Trigger failed: {await trigger_response.text()}",
                        trigger_response.status,
                    )
                trigger_data = await trigger_response.json()
                snapshot_id = trigger_data.get("snapshot_id")

            elapsed = 0
            while elapsed < timeout:
                await asyncio.sleep(polling_interval)
                elapsed += polling_interval

                async with session.get(
                    f"{BRIGHTDATA_API_URL}/datasets/v3/progress/{snapshot_id}",
                    headers=headers,
                ) as status_response:
                    if status_response.status != 200:
                        raise BrightDataDatasetToolException(
                            f"Status check failed: {await status_response.text()}",

View on GitHub (pinned to 754d7323be)

Solutions

  1. Read the response text embedded in the exception — it states the exact Bright Data rejection reason.
  2. Verify BRIGHT_DATA_API_KEY is current and the target dataset is activated in your Bright Data account.
  3. Validate request_data against the dataset type's expected input schema before calling.
  4. If 429/5xx, retry with backoff (wrap the call and catch BrightDataDatasetToolException, inspecting .status_code).
Defensive patterns

Strategy: retry

Validate before calling

import os

assert os.getenv("BRIGHT_DATA_API_KEY"), "BRIGHT_DATA_API_KEY required before triggering datasets"
assert isinstance(request_data, dict) and request_data, "request_data must be a non-empty dict"

Try / catch

from crewai_tools.tools.brightdata_tool import BrightDataDatasetToolException

for attempt in range(3):
    try:
        result = tool.run(url=url, dataset_type=dataset_type)
        break
    except BrightDataDatasetToolException as e:
        if e.status_code in (429, 500, 502, 503) and attempt < 2:
            time.sleep(2 ** attempt)
            continue
        raise  # 4xx (auth/bad request) — do not retry

Prevention

When it happens

Trigger: Expired or wrong BRIGHT_DATA_API_KEY (401/403), malformed request_data for the dataset type (400), unactivated dataset/zone in the Bright Data account (422), or insufficient quota — any non-200 from the trigger endpoint.

Common situations: Free-trial accounts without the dataset unlocked, API key rotated but env var stale, request_data schema drift after Bright Data API updates.

Related errors

AI-assisted analysis of crewAIInc/crewAI@754d7323be (2026-08-15). Data as JSON: /api/errors/ae0310597a554c99. Report an issue: GitHub.