{"record":{"id":"03939c7396773be1","repo":"apache/cassandra","slug":"insufficient-remaining-bytes-to-deserialize-inner","errorCode":null,"errorMessage":"Insufficient remaining bytes to deserialize Inner node off-heap","messagePattern":"Insufficient remaining bytes to deserialize Inner node off-heap","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"src/java/org/apache/cassandra/utils/MerkleTree.java","lineNumber":1328,"sourceCode":"\n                computed = true;\n            }\n\n            return this;\n        }\n\n        static OnHeapInner deserializeWithoutIdent(DataInputPlus in, IPartitioner p, int version) throws IOException\n        {\n            Token token = Token.serializer.deserialize(in, p, version);\n            OnHeapNode  left = OnHeapNode.deserialize(in, p, version);\n            OnHeapNode right = OnHeapNode.deserialize(in, p, version);\n            return new OnHeapInner(token, left, right);\n        }\n\n        int serializeOffHeap(ByteBuffer buffer, IPartitioner partitioner) throws IOException\n        {\n            if (buffer.remaining() < OffHeapInner.maxOffHeapSize(partitioner))\n                throw new IllegalStateException(\"Insufficient remaining bytes to deserialize Inner node off-heap\");\n\n            final int offset = buffer.position();\n\n            int tokenSize = partitioner.getTokenFactory().byteSize(token);\n            buffer.putShort(offset + OffHeapInner.TOKEN_LENGTH_OFFSET, Shorts.checkedCast(tokenSize));\n            buffer.position(offset + OffHeapInner.TOKEN_BYTES_OFFSET);\n            partitioner.getTokenFactory().serialize(token, buffer);\n\n            int  leftPointer =  left.serializeOffHeap(buffer, partitioner);\n            int rightPointer = right.serializeOffHeap(buffer, partitioner);\n\n            buffer.putInt(offset + OffHeapInner.LEFT_CHILD_POINTER_OFFSET,  leftPointer);\n            buffer.putInt(offset + OffHeapInner.RIGHT_CHILD_POINTER_OFFSET, rightPointer);\n\n            int  leftHashOffset = OffHeapInner.hashBytesOffset(leftPointer);\n            int rightHashOffset = OffHeapInner.hashBytesOffset(rightPointer);\n\n            for (int i = 0; i < HASH_SIZE; i += 8)","sourceCodeStart":1310,"sourceCodeEnd":1346,"githubUrl":"https://github.com/apache/cassandra/blob/88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1/src/java/org/apache/cassandra/utils/MerkleTree.java#L1310-L1346","documentation":"MerkleTree.Inner.serializeOffHeap writes an off-heap representation of an inner node into a caller-provided ByteBuffer. Before writing it verifies the buffer has at least maxOffHeapSize(partitioner) bytes remaining; if not it throws this IllegalStateException. The message says 'deserialize' but is thrown on the serialize path — it is a buffer-capacity guard against truncating a node mid-write.","triggerScenarios":"Calling Inner.serializeOffHeap with a ByteBuffer whose remaining() is smaller than OffHeapInner.maxOffHeapSize(partitioner), typically when the buffer was sized without accounting for the token size or the short/long header fields, or when the buffer was partially consumed before the call.","commonSituations":"Anti-entropy/repair code serializing merkle trees into fixed-size buffers after the partitioner was changed (e.g. RandomPartitioner vs Murmur3 token sizes), or hand-rolled buffer sizing that underestimates token byte size.","solutions":["Size the destination ByteBuffer to at least OffHeapInner.maxOffHeapSize(partitioner) before calling serializeOffHeap","Recompute buffer size with the current partitioner's token factory byte size; do not cache sizes across partitioner changes","Check buffer.remaining() / ensure the buffer position is reset (clear/rewind) before serializing","Inspect call sites that write multiple nodes into one buffer and make sure each node's space is reserved"],"exampleFix":"// before\nByteBuffer buf = ByteBuffer.allocate(128);\ninner.serializeOffHeap(buf, partitioner);\n// after\nByteBuffer buf = ByteBuffer.allocate(OffHeapInner.maxOffHeapSize(partitioner));\ninner.serializeOffHeap(buf, partitioner);","handlingStrategy":"validation","validationCode":"if (buffer.remaining() < OffHeapInner.maxOffHeapSize(partitioner)) throw new IllegalStateException(\"buffer too small for inner node\");","typeGuard":null,"tryCatchPattern":"try { inner.serializeOffHeap(buffer, partitioner); } catch (IllegalStateException e) { /* resize buffer to maxOffHeapSize(partitioner) and retry */ }","preventionTips":["Always size buffers via OffHeapInner.maxOffHeapSize(partitioner), never hardcode","Reset buffer position (clear/rewind) before serializing","Recompute sizes whenever the partitioner changes","Unit-test serialization with the largest token size your partitioner can produce"],"tags":["merkle-tree","buffer-overflow","serialization"],"backgroundTag":"internal-invariant-violation","analyzedSha":"88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1","analyzedAt":"2026-09-10T07:29:22.284Z","contentChangedAt":"2026-09-10T07:29:22.284Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}