unclecode/crawl4ai · error · HTTPException

(json.dumps)

Error message

{"error": str(e), "server_memory_delta_mb": mem_delta_mb, "server_peak_memory_mb": max(peak_mem_mb if peak_mem_mb else 0, end_mem_mb_error or 0)} (json.dumps)

What it means

Error "{"error": str(e), "server_memory_delta_mb": mem_delta_mb, "server_peak_memory_mb": max(peak_mem_mb if peak_mem_mb else 0, end_mem_mb_error or 0)} (json.dumps)" thrown in unclecode/crawl4ai.

Solutions

1. Investigate the exception message in the 'error' field and the memory metrics to find the root cause.
2. Reduce concurrent crawls or page complexity if server_peak_memory_mb is high.

Example fix

# Inspect response JSON: {'error': ..., 'server_memory_delta_mb': ..., 'server_peak_memory_mb': ...}

When it happens

Trigger: Thrown at deploy/docker/api.py:841 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of unclecode/crawl4ai@7e80152142 (2026-08-14). Data as JSON: /api/errors/e4b6064098340191. Report an issue: GitHub.

Appendix: source

Thrown at deploy/docker/api.py:841

    except Exception as e:
        logger.error(f"Crawl error: {str(e)}", exc_info=True)

        # Track request error
        try:
            from monitor import get_monitor
            await get_monitor().track_request_end(
                request_id, success=False, error=str(e), status_code=500
            )
        except:
            pass

        # Measure memory even on error if possible
        end_mem_mb_error = _get_memory_mb()
        if start_mem_mb is not None and end_mem_mb_error is not None:
            mem_delta_mb = end_mem_mb_error - start_mem_mb

        raise HTTPException(
            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
            detail=json.dumps({ # Send structured error
                "error": str(e),
                "server_memory_delta_mb": mem_delta_mb,
                "server_peak_memory_mb": max(peak_mem_mb if peak_mem_mb else 0, end_mem_mb_error or 0)
            })
        )
    finally:
        if crawler:
            await release_crawler(crawler)

async def handle_stream_crawl_request(
    urls: List[str],
    browser_config: dict,
    crawler_config: dict,
    config: dict,
    hooks_config: Optional[dict] = None
) -> Tuple[AsyncWebCrawler, AsyncGenerator, Optional[Dict]]:

View on GitHub (pinned to 7e80152142)