{"record":{"id":"265d8f769644b2b8","repo":"unclecode/crawl4ai","slug":"artifact-storage-quota-exceeded-265d8f","errorCode":null,"errorMessage":"Artifact storage quota exceeded","messagePattern":"Artifact storage quota exceeded","errorType":"http","errorClass":"HTTPException","httpStatus":507,"severity":"error","filePath":"deploy/docker/server.py","lineNumber":641,"sourceCode":"        from crawl4ai.utils import preprocess_html_for_schema\n        processed_html = preprocess_html_for_schema(raw_html)\n        return JSONResponse({\"html\": processed_html, \"url\": body.url, \"success\": True})\n    except Exception as e:\n        raise HTTPException(500, detail=str(e))\n    finally:\n        if crawler:\n            await release_crawler(crawler)\n\n# ── artifact store helpers ───────────────────────────────────\ndef _store_artifact(kind: str, data: bytes) -> dict:\n    \"\"\"Write to the sandboxed store; map quota/size errors to HTTP codes.\"\"\"\n    from artifacts import write_artifact, ArtifactTooLarge, QuotaExceeded\n    try:\n        meta = write_artifact(kind, data)\n    except ArtifactTooLarge:\n        raise HTTPException(413, \"Artifact too large\")\n    except QuotaExceeded:\n        raise HTTPException(507, \"Artifact storage quota exceeded\")\n    return {\n        \"artifact_id\": meta[\"artifact_id\"],\n        \"url\": f\"/artifacts/{meta['artifact_id']}\",\n        \"mime\": meta[\"mime\"],\n        \"size\": meta[\"size\"],\n    }\n\n\n@app.get(\"/artifacts/{artifact_id}\")\nasync def get_artifact(artifact_id: str, _td: Dict = Depends(token_dep)):\n    \"\"\"Fetch a previously generated artifact by its opaque id (authed).\"\"\"\n    from artifacts import resolve_artifact, ArtifactNotFound\n    try:\n        path, mime = resolve_artifact(artifact_id)\n    except ArtifactNotFound:\n        raise HTTPException(404, \"Artifact not found\")\n    return FileResponse(path, media_type=mime, headers={\"X-Content-Type-Options\": \"nosniff\"})\n","sourceCodeStart":623,"sourceCodeEnd":659,"githubUrl":"https://github.com/unclecode/crawl4ai/blob/7e801521428ee12509994d39151006f64055ebe3/deploy/docker/server.py#L623-L659","documentation":"A 507 from _store_artifact(): write_artifact() raised QuotaExceeded — the sandboxed artifact store's total storage quota is exhausted. Unlike the 413 (single artifact too large), this means cumulative stored data has hit the global cap.","triggerScenarios":"Many /screenshot or /pdf calls accumulating artifacts until the store's total quota is full; the next write of any size fails with 507.","commonSituations":"Long-running service without artifact retention/cleanup; batch screenshot jobs; container volume sized too small for the workload.","solutions":["Prune old artifacts (the store's cleanup/retention mechanism) or wipe the store volume if artifacts are disposable.","Raise the store quota / volume size for the deployment.","Add client-side retention: only request artifact storage when needed, or fetch-and-delete after consumption.","Monitor store usage and alert before the quota is reached."],"exampleFix":null,"handlingStrategy":"fallback","validationCode":null,"typeGuard":null,"tryCatchPattern":"resp = requests.post(f'{BASE}/pdf', json={'url': url}, headers=hdrs)\nif resp.status_code == 507:\n    # quota exhausted: fetch-and-consume inline bytes, skip storage-dependent flows\n    raise RuntimeError('artifact store full — prune artifacts or raise quota')","preventionTips":["Run artifact retention/cleanup on a schedule before the quota fills.","Monitor store usage and alert below 100%.","Design clients to consume artifacts immediately so the store can be pruned aggressively."],"tags":["artifacts","http-507","quota","storage"],"backgroundTag":null,"analyzedSha":"7e801521428ee12509994d39151006f64055ebe3","analyzedAt":"2026-08-14T20:46:20.673Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}