{"record":{"id":"01f1a651dfeeb11d","repo":"affaan-m/ECC","slug":"json-artifact-exceeds-local-size-limit","errorCode":null,"errorMessage":"JSON artifact exceeds local size limit","messagePattern":"JSON artifact exceeds local size limit","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"skills/taste-application/scripts/tasteforge/integration.py","lineNumber":127,"sourceCode":"        raise\n\n\ndef _read_local(raw: str, *, parse_json: bool, expected_size: int | None = None,\n                expected_hash: str | None = None) -> Any:\n    path = Path(raw)\n    if not path.is_absolute() or str(path) != raw or \"..\" in path.parts:\n        raise ValueError(\"artifact path must be canonical and absolute\")\n    parent = descriptor = None\n    try:\n        flags = os.O_RDONLY | os.O_NOFOLLOW | os.O_NONBLOCK\n        parent = _parent_fd(path)\n        before = os.stat(path.name, dir_fd=parent, follow_symlinks=False)\n        if not stat.S_ISREG(before.st_mode) or getattr(before, \"st_flags\", 0) & 0x40000000:\n            raise ValueError(\"artifact must be a resident regular file\")\n        if expected_size is None:\n            expected_size = before.st_size\n        if parse_json and expected_size > _MAX_JSON:\n            raise ValueError(\"JSON artifact exceeds local size limit\")\n        if before.st_size != expected_size:\n            raise ValueError(\"artifact byte count mismatch\")\n        descriptor = os.open(path.name, flags, dir_fd=parent)\n        if _identity(before) != _identity(os.fstat(descriptor)):\n            raise ValueError(\"artifact changed before reading\")\n        digest, chunks, count = hashlib.sha256(), [], 0\n        while data := os.read(descriptor, 65536):\n            count += len(data)\n            if count > expected_size:\n                raise ValueError(\"artifact byte count exceeded during reading\")\n            digest.update(data)\n            if parse_json:\n                chunks.append(data)\n        # Rewalk the named path: a pinned old directory fd can outlive a rename.\n        fresh_parent = _parent_fd(path)\n        try:\n            after = os.stat(path.name, dir_fd=fresh_parent, follow_symlinks=False)\n        finally:","sourceCodeStart":109,"sourceCodeEnd":145,"githubUrl":"https://github.com/affaan-m/ECC/blob/8321021c54d670126ce3b2969d5deb880b4b0c2a/skills/taste-application/scripts/tasteforge/integration.py#L109-L145","documentation":"When reading a JSON artifact (`parse_json=True`) and no explicit `expected_size` was supplied by the caller beyond the stat, `_read_local` enforces a maximum JSON payload size `_MAX_JSON`. This protects the process from unbounded memory consumption when joining and parsing the file's chunks. A file larger than that cap is refused before any bytes are read.","triggerScenarios":"Calling `load_application_request` (or `_artifact` with `parse_json=True`) on a JSON file whose size exceeds `_MAX_JSON` bytes; an upstream tool accidentally writing a huge log as JSON with a `.json` extension.","commonSituations":"An application-request JSON that accumulated enormous embedded data (base64 blobs, full dumps); a runaway generator concatenating artifacts; pointing the loader at a data file that was never meant to be a request document.","solutions":["Shrink or split the JSON artifact so it fits under the local size limit, or move bulk data into sidecar files referenced by hash.","If the file is legitimately large, switch the pipeline to a non-JSON (or streamed) artifact path rather than `parse_json=True`.","Check the size before calling: `os.path.getsize(p) <= _MAX_JSON`, and regenerate the artifact if it exceeds it.","Verify the artifact generator isn't duplicating content (e.g. writing debug payloads into the JSON)."],"exampleFix":"// before\nreq = load_application_request(\"/out/request.json\")  # 250MB JSON dump\n// after\nimport os\nif os.path.getsize(\"/out/request.json\") > _MAX_JSON:\n    slim_request(\"/out/request.json\")  # strip oversized embedded payloads first\nreq = load_application_request(\"/out/request.json\")","handlingStrategy":"validation","validationCode":"import os\ndef assert_json_fits(path: str, max_json: int) -> None:\n    size = os.path.getsize(path)\n    if size > max_json:\n        raise ValueError(f\"JSON artifact {size} bytes exceeds limit {max_json}\")","typeGuard":null,"tryCatchPattern":"try:\n    req = load_application_request(p)\nexcept ValueError as e:\n    if str(e) == \"JSON artifact exceeds local size limit\":\n        slim_or_split_artifact(p)   # move bulk data to sidecar files, then retry\n        req = load_application_request(p)\n    else:\n        raise","preventionTips":["Keep bulk payloads (blobs, dumps) in separate hash-referenced files, not inline in JSON.","Add a size check to the artifact generator so oversized JSON fails at production time.","Monitor artifact sizes in CI to catch runaway generators before they hit the limit.","Point loaders only at true request documents; never reuse the JSON loader for arbitrary data dumps."],"tags":["filesystem","size-limit","json"],"backgroundTag":"file-size-limit-exceeded","analyzedSha":"8321021c54d670126ce3b2969d5deb880b4b0c2a","analyzedAt":"2026-09-16T10:08:13.343Z","contentChangedAt":"2026-09-16T10:08:13.343Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}