unclecode/crawl4ai · error · HTTPException

{"error": str(e), "server_memory_delta_mb": mem_delta_mb, "s

Error message

{"error": str(e), "server_memory_delta_mb": mem_delta_mb, "server_peak_memory_mb": max(peak_mem_mb if peak_mem_mb else 0, end_mem_mb_error or 0)} (json.dumps)

What it means

Error "{"error": str(e), "server_memory_delta_mb": mem_delta_mb, "server_peak_memory_mb": max(peak_mem_mb if peak_mem_mb else 0, end_mem_mb_error or 0)} (json.dumps)" thrown in unclecode/crawl4ai.

Source

Thrown at deploy/docker/api.py:841

    except Exception as e:
        logger.error(f"Crawl error: {str(e)}", exc_info=True)

        # Track request error
        try:
            from monitor import get_monitor
            await get_monitor().track_request_end(
                request_id, success=False, error=str(e), status_code=500
            )
        except:
            pass

        # Measure memory even on error if possible
        end_mem_mb_error = _get_memory_mb()
        if start_mem_mb is not None and end_mem_mb_error is not None:
            mem_delta_mb = end_mem_mb_error - start_mem_mb

        raise HTTPException(
            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
            detail=json.dumps({ # Send structured error
                "error": str(e),
                "server_memory_delta_mb": mem_delta_mb,
                "server_peak_memory_mb": max(peak_mem_mb if peak_mem_mb else 0, end_mem_mb_error or 0)
            })
        )
    finally:
        if crawler:
            await release_crawler(crawler)

async def handle_stream_crawl_request(
    urls: List[str],
    browser_config: dict,
    crawler_config: dict,
    config: dict,
    hooks_config: Optional[dict] = None
) -> Tuple[AsyncWebCrawler, AsyncGenerator, Optional[Dict]]:

View on GitHub (pinned to 7e80152142)

Solutions

1. Investigate the exception message in the 'error' field and the memory metrics to find the root cause.
2. Reduce concurrent crawls or page complexity if server_peak_memory_mb is high.

Example fix

# Inspect response JSON: {'error': ..., 'server_memory_delta_mb': ..., 'server_peak_memory_mb': ...}

When it happens

Trigger: Thrown at deploy/docker/api.py:841 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of unclecode/crawl4ai@7e80152142 (2026-08-14). Data as JSON: /api/errors/e4b6064098340191. Report an issue: GitHub.