{"record":{"id":"44de6dc24739cf28","repo":"pypa/pip","slug":"too-large-data","errorCode":null,"errorMessage":"Too large data","messagePattern":"Too large data","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/pip/_vendor/msgpack/fallback.py","lineNumber":851,"sourceCode":"    def pack_map_header(self, n):\n        if n >= 2**32:\n            raise ValueError\n        self._pack_map_header(n)\n        if self._autoreset:\n            ret = self._buffer.getvalue()\n            self._buffer = BytesIO()\n            return ret\n\n    def pack_ext_type(self, typecode, data):\n        if not isinstance(typecode, int):\n            raise TypeError(\"typecode must have int type.\")\n        if not 0 <= typecode <= 127:\n            raise ValueError(\"typecode should be 0-127\")\n        if not isinstance(data, bytes):\n            raise TypeError(\"data must have bytes type\")\n        L = len(data)\n        if L > 0xFFFFFFFF:\n            raise ValueError(\"Too large data\")\n        if L == 1:\n            self._buffer.write(b\"\\xd4\")\n        elif L == 2:\n            self._buffer.write(b\"\\xd5\")\n        elif L == 4:\n            self._buffer.write(b\"\\xd6\")\n        elif L == 8:\n            self._buffer.write(b\"\\xd7\")\n        elif L == 16:\n            self._buffer.write(b\"\\xd8\")\n        elif L <= 0xFF:\n            self._buffer.write(b\"\\xc7\" + struct.pack(\"B\", L))\n        elif L <= 0xFFFF:\n            self._buffer.write(b\"\\xc8\" + struct.pack(\">H\", L))\n        else:\n            self._buffer.write(b\"\\xc9\" + struct.pack(\">I\", L))\n        self._buffer.write(struct.pack(\"B\", typecode))\n        self._buffer.write(data)","sourceCodeStart":833,"sourceCodeEnd":869,"githubUrl":"https://github.com/pypa/pip/blob/f399c3718970b1b0e2478dac5296eb62679a9b86/src/pip/_vendor/msgpack/fallback.py#L833-L869","documentation":"Raised by msgpack's pure-Python Packer.pack_ext_type when the data payload length exceeds 0xFFFFFFFF (4 GiB − 1). The msgpack ext format encodes the data length in a 32-bit field, so the maximum single ext payload is just under 4 GiB. This is the ext-type-specific equivalent of the bin/string size limits.","triggerScenarios":"Calling packer.pack_ext_type(code, data) where len(data) > 0xFFFFFFFF (4,294,967,295 bytes). Encountered when embedding large blobs as custom extension types.","commonSituations":"Serializing large binary blobs (media files, serialized models) as a single ext type; insufficient chunking strategy for extension data.","solutions":["Split the payload into multiple ext messages each under 4 GiB and reassemble on unpack.","Store the large payload externally and pack only a reference/URL in the ext data.","Use the standard bin type with chunking instead of a custom ext type for very large blobs.","Pre-check len(data) <= 0xFFFFFFFF before calling pack_ext_type."],"exampleFix":"# before\npacker.pack_ext_type(1, huge_bytes)  # raises if > 4 GiB\n\n# after — chunk the payload\nMAX = 0xFFFFFFFF\nchunks = [huge_bytes[i:i+MAX] for i in range(0, len(huge_bytes), MAX)]\nfor i, chunk in enumerate(chunks):\n    packer.pack_ext_type(1, chunk)  # caller tracks ordering","handlingStrategy":"validation","validationCode":"MSGPACK_MAX_EXT_DATA = 0xFFFFFFFF\n\ndef is_serializable_ext_data(data: bytes) -> bool:\n    return len(data) <= MSGPACK_MAX_EXT_DATA","typeGuard":"def is_small_enough_ext_data(data) -> bool:\n    return isinstance(data, bytes) and len(data) <= 0xFFFFFFFF","tryCatchPattern":"try:\n    packer.pack_ext_type(code, data)\nexcept ValueError as e:\n    if 'Too large' in str(e):\n        # split and pack as multiple ext messages\n        for i in range(0, len(data), 0xFFFFFFFF):\n            packer.pack_ext_type(code, data[i:i+0xFFFFFFFF])\n    else:\n        raise","preventionTips":["Check len(data) <= 4 GiB before calling pack_ext_type.","Chunk large ext payloads.","Store very large blobs externally."],"tags":["msgpack","ext-type","size-limit"],"backgroundTag":null,"analyzedSha":"f399c3718970b1b0e2478dac5296eb62679a9b86","analyzedAt":"2026-08-08T23:01:42.227Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}