{"record":{"id":"ee5c9081bbe07fdf","repo":"unclecode/crawl4ai","slug":"artifact-too-large","errorCode":null,"errorMessage":"Artifact too large","messagePattern":"Artifact too large","errorType":"http","errorClass":"HTTPException","httpStatus":413,"severity":"error","filePath":"deploy/docker/server.py","lineNumber":639,"sourceCode":"\n        raw_html = results[0].html\n        from crawl4ai.utils import preprocess_html_for_schema\n        processed_html = preprocess_html_for_schema(raw_html)\n        return JSONResponse({\"html\": processed_html, \"url\": body.url, \"success\": True})\n    except Exception as e:\n        raise HTTPException(500, detail=str(e))\n    finally:\n        if crawler:\n            await release_crawler(crawler)\n\n# ── artifact store helpers ───────────────────────────────────\ndef _store_artifact(kind: str, data: bytes) -> dict:\n    \"\"\"Write to the sandboxed store; map quota/size errors to HTTP codes.\"\"\"\n    from artifacts import write_artifact, ArtifactTooLarge, QuotaExceeded\n    try:\n        meta = write_artifact(kind, data)\n    except ArtifactTooLarge:\n        raise HTTPException(413, \"Artifact too large\")\n    except QuotaExceeded:\n        raise HTTPException(507, \"Artifact storage quota exceeded\")\n    return {\n        \"artifact_id\": meta[\"artifact_id\"],\n        \"url\": f\"/artifacts/{meta['artifact_id']}\",\n        \"mime\": meta[\"mime\"],\n        \"size\": meta[\"size\"],\n    }\n\n\n@app.get(\"/artifacts/{artifact_id}\")\nasync def get_artifact(artifact_id: str, _td: Dict = Depends(token_dep)):\n    \"\"\"Fetch a previously generated artifact by its opaque id (authed).\"\"\"\n    from artifacts import resolve_artifact, ArtifactNotFound\n    try:\n        path, mime = resolve_artifact(artifact_id)\n    except ArtifactNotFound:\n        raise HTTPException(404, \"Artifact not found\")","sourceCodeStart":621,"sourceCodeEnd":657,"githubUrl":"https://github.com/unclecode/crawl4ai/blob/7e801521428ee12509994d39151006f64055ebe3/deploy/docker/server.py#L621-L657","documentation":"A 413 from _store_artifact(): write_artifact() raised ArtifactTooLarge, meaning the generated artifact (PNG screenshot, PDF, etc.) exceeds the sandboxed store's per-artifact size cap. The store enforces limits so single payloads can't fill the disk.","triggerScenarios":"POST /screenshot or /pdf on a page whose rendered PNG/PDF is larger than the configured per-artifact maximum; e.g. an extremely long page captured full-page at high resolution.","commonSituations":"Full-page screenshots of infinite-scroll or very long pages; PDFs of huge documents; a lowered store size cap in the deployment config versus earlier versions.","solutions":["Reduce output size: for screenshots set wait_for_images only when needed and target shorter pages; for PDFs, print sub-pages instead of one giant page.","Raise the per-artifact size cap in the artifacts store configuration if the deployment genuinely needs large artifacts.","Split the crawl: capture per-section URLs and store several smaller artifacts.","Check artifact store config (ARTIFACT_MAX_SIZE-style env/settings) in the container."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"# Client-side heuristic: avoid full-page captures of extremely long pages\nimport requests\n\ndef page_too_long(url: str, approx_max_px: int = 30000) -> bool:\n    r = requests.get(url, timeout=10)\n    # rough heuristic on content length; tune to your store's cap\n    return len(r.content) > approx_max_px * 40","typeGuard":null,"tryCatchPattern":"resp = requests.post(f'{BASE}/screenshot', json=body, headers=hdrs)\nif resp.status_code == 413:\n    # artifact too large: shrink the capture (split page, lower resolution) and retry","preventionTips":["Split very long pages into section URLs and capture each separately.","Know your store's per-artifact cap and keep captures under it.","Handle 413 as a signal to reduce output size, not as a transient error to retry."],"tags":["artifacts","http-413","storage","size-limit"],"backgroundTag":null,"analyzedSha":"7e801521428ee12509994d39151006f64055ebe3","analyzedAt":"2026-08-14T20:46:20.673Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}