firecrawl/firecrawl · error · Exception

Failed to scrape URL. Error: {error_content}

Error message

Failed to scrape URL. Error: {error_content}

What it means

Raised by async scrape_url in the fall-through else branch — the response had neither success+data nor an 'error' field. The error_content is response.get('error', str(response)), which since the 'error' key is absent becomes str(response), often an unhelpful dict dump.

Source

Thrown at apps/python-sdk/firecrawl/v1/client.py:3745

        if 'jsonOptions' in scrape_params and scrape_params['jsonOptions'] and 'schema' in scrape_params['jsonOptions']:
            scrape_params['jsonOptions']['schema'] = self._ensure_schema_dict(scrape_params['jsonOptions']['schema'])

        # Make async request
        endpoint = f'/v1/scrape'
        response = await self._async_post_request(
            f'{self.api_url}{endpoint}',
            scrape_params,
            _headers
        )

        if response.get('success') and 'data' in response:
            return V1ScrapeResponse(**response['data'])
        elif "error" in response:
            raise Exception(f'Failed to scrape URL. Error: {response["error"]}')
        else:
            # Use the response content directly if possible, otherwise a generic message
            error_content = response.get('error', str(response))
            raise Exception(f'Failed to scrape URL. Error: {error_content}')

    async def batch_scrape_urls(
        self,
        urls: List[str],
        *,
        formats: Optional[List[Literal["markdown", "html", "rawHtml", "content", "links", "screenshot", "screenshot@fullPage", "extract", "json"]]] = None,
        headers: Optional[Dict[str, str]] = None,
        include_tags: Optional[List[str]] = None,
        exclude_tags: Optional[List[str]] = None,
        only_main_content: Optional[bool] = None,
        wait_for: Optional[int] = None,
        timeout: Optional[int] = 30000,
        location: Optional[V1LocationConfig] = None,
        mobile: Optional[bool] = None,
        skip_tls_verification: Optional[bool] = None,
        remove_base64_images: Optional[bool] = None,
        block_ads: Optional[bool] = None,
        proxy: Optional[Literal["basic", "stealth", "enhanced", "auto"]] = None,

View on GitHub (pinned to 656bffcc28)

Solutions

1. Upgrade the python-sdk to a version compatible with your Firecrawl server version.
2. Log the full response to diagnose the unexpected shape; the str(response) in the message is the only clue.
3. If self-hosted, align server and SDK versions.

Example fix

// before
data = await app.scrape_url(url)  # unknown failure shape
// after
# upgrade SDK and server in lockstep; pin compatible versions
# pip install firecrawl-py==X.Y.Z matching your server tag

null

null

try:
    data = await app.scrape_url(url)
except Exception as e:
    log.error('unexpected scrape response: %s', e)
    # check SDK/server version alignment, then upgrade and retry
    raise

When it happens

Trigger: Server returned an unexpected JSON shape — a partial/malformed response, a maintenance page rendered as JSON, or an undocumented response variant the SDK does not recognize.

Common situations: API version mismatch (SDK talking to a newer/older server), a server-side bug returning an unusual body, or a proxy injecting its own JSON.

Related errors

• Failed to map URL. Error: {response}
• Batch scrape job failed to start
• Failed to scrape URL. Error: {response["error"]}
• Failed to scrape URL. Error: {response["error"]}
• Async v2 client not implemented yet: ${name}

AI-assisted analysis of firecrawl/firecrawl@656bffcc28 (2026-08-12). Data as JSON: /api/errors/88292288b09a3680. Report an issue: GitHub.