{"record":{"id":"2f2b1e04c783bd47","repo":"apache/pulsar","slug":"invalid-messageidv5-data-bad-inner-id-length","errorCode":null,"errorMessage":"Invalid MessageIdV5 data: bad inner id length","messagePattern":"Invalid MessageIdV5 data: bad inner id length","errorType":"exception","errorClass":"java.io.IOException","httpStatus":null,"severity":"error","filePath":"pulsar-client-v5/src/main/java/org/apache/pulsar/client/impl/v5/MessageIdV5.java","lineNumber":326,"sourceCode":"                    buf.get(topicBytes);\n                    String topic = new String(topicBytes, StandardCharsets.UTF_8);\n                    Map<Long, MessageId> inner = readSegmentVector(buf);\n                    multiTopic.put(topic, inner);\n                }\n            }\n        }\n\n        return new MessageIdV5(v4Id, segmentId, positions, parentTopic, multiTopic);\n    }\n\n    private static Map<Long, MessageId> readSegmentVector(ByteBuffer buf) throws IOException {\n        int count = buf.getInt();\n        Map<Long, MessageId> out = new HashMap<>(count);\n        for (int i = 0; i < count; i++) {\n            long segId = buf.getLong();\n            int idLen = buf.getInt();\n            if (idLen < 0 || idLen > buf.remaining()) {\n                throw new IOException(\"Invalid MessageIdV5 data: bad inner id length\");\n            }\n            byte[] idBytes = new byte[idLen];\n            buf.get(idBytes);\n            out.put(segId, MessageId.fromByteArray(idBytes));\n        }\n        return out;\n    }\n\n    @Override\n    public int compareTo(org.apache.pulsar.client.api.v5.MessageId other) {\n        if (!(other instanceof MessageIdV5 o)) {\n            throw new IllegalArgumentException(\"Cannot compare with \" + other.getClass());\n        }\n        int cmp = Long.compare(this.segmentId, o.segmentId);\n        if (cmp != 0) {\n            return cmp;\n        }\n        return this.v4MessageId.compareTo(o.v4MessageId);","sourceCodeStart":308,"sourceCodeEnd":344,"githubUrl":"https://github.com/apache/pulsar/blob/820761864ed8e2a7d2e52dd9763ad2ae117c1395/pulsar-client-v5/src/main/java/org/apache/pulsar/client/impl/v5/MessageIdV5.java#L308-L344","documentation":"readSegmentVector parses section 3: a count, then per entry an 8-byte segment id, a 4-byte inner-id length, and the id bytes. If an inner id length is negative or exceeds remaining bytes, the buffer is corrupt/misaligned, so it throws IOException('Invalid MessageIdV5 data: bad inner id length'). Called from fromByteArray and inner.","triggerScenarios":"Deserializing a truncated or offset-shifted blob so the declared id length runs past the buffer end; byte-order mismatch turning the length into a negative/huge value; a count field mismatched with the actual number of entries written.","commonSituations":"Custom persistence of MessageIdV5 with framing bugs; partial writes from a crashed producer; hand-editing or re-framing checkpoint blobs; mixing little-endian writers with the big-endian ByteBuffer reader.","solutions":["Ensure both writer and reader use the same byte order and framing (ByteBuffer big-endian default).","Verify the position-vector count matches the number of segment entries actually persisted.","Regenerate the blob via toByteArray/from the live checkpoint instead of repairing bytes manually.","Add a length/version header when persisting ids so truncation is detected earlier."],"exampleFix":"// before\nbyte[] blob = store.load(key); // possibly partial write\nMessageIdV5 id = MessageIdV5.fromByteArray(blob);\n// after\nbyte[] blob = store.load(key);\nif (blob.length != expectedLen) { // persisted length header\n    throw new IOException(\"Partial blob, discarding checkpoint\");\n}\nMessageIdV5 id = MessageIdV5.fromByteArray(blob);","handlingStrategy":"validation","validationCode":"ByteBuffer buf = ByteBuffer.wrap(data);\nbuf.getLong(); // segmentId\nint v4Len = buf.getInt();\nbuf.position(buf.position() + v4Len);\nif (buf.remaining() < 4) {\n    throw new IOException(\"No room for segment-vector count — blob truncated\");\n}\nint count = buf.getInt();\nif (8L + 4L * count > buf.remaining()) {\n    throw new IOException(\"Segment vector entries exceed remaining bytes\");\n}","typeGuard":null,"tryCatchPattern":"try {\n    MessageIdV5 id = MessageIdV5.fromByteArray(data);\n} catch (IOException e) {\n    log.warn(\"Corrupt segment vector, rebuilding positions from broker\", e);\n}","preventionTips":["Match byte order between writer and reader (ByteBuffer defaults to big-endian).","Validate per-entry lengths against remaining bytes in any custom parser.","Avoid hand-rolled serialization of MessageIdV5; use toByteArray/fromByteArray exclusively."],"tags":["deserialization","message-id","byte-buffer"],"backgroundTag":"invalid-message-id-data","analyzedSha":"820761864ed8e2a7d2e52dd9763ad2ae117c1395","analyzedAt":"2026-09-06T00:14:20.138Z","contentChangedAt":"2026-09-06T00:14:20.138Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}