apache/pulsar · error · java.io.IOException

Invalid MessageIdV5 data: bad parent-topic length

Error message

Invalid MessageIdV5 data: bad parent-topic length

What it means

Section 4 of the MessageIdV5 encoding is an optional parent topic string, prefixed by a 4-byte length where -1 means absent. If a non-negative length exceeds the bytes remaining in the buffer, the data is corrupt or misaligned, so fromByteArray throws IOException('Invalid MessageIdV5 data: bad parent-topic length').

Source

Thrown at pulsar-client-v5/src/main/java/org/apache/pulsar/client/impl/v5/MessageIdV5.java:291

            throw new IOException("Invalid MessageIdV5 data: bad v4 length");
        }
        byte[] v4Bytes = new byte[v4Length];
        buf.get(v4Bytes);
        MessageId v4Id = MessageId.fromByteArray(v4Bytes);

        // Section 3: position vector (single-topic / per-segment).
        Map<Long, MessageId> positions = Map.of();
        if (buf.hasRemaining()) {
            positions = readSegmentVector(buf);
        }

        // Section 4: parent topic. Length -1 sentinel means "absent".
        String parentTopic = null;
        if (buf.hasRemaining()) {
            int parentLen = buf.getInt();
            if (parentLen >= 0) {
                if (parentLen > buf.remaining()) {
                    throw new IOException("Invalid MessageIdV5 data: bad parent-topic length");
                }
                byte[] parentBytes = new byte[parentLen];
                buf.get(parentBytes);
                parentTopic = new String(parentBytes, StandardCharsets.UTF_8);
            }
        }

        // Section 5: cross-topic vector. Count -1 sentinel means "absent".
        Map<String, Map<Long, MessageId>> multiTopic = null;
        if (buf.hasRemaining()) {
            int topicCount = buf.getInt();
            if (topicCount >= 0) {
                multiTopic = new HashMap<>(topicCount);
                for (int i = 0; i < topicCount; i++) {
                    int topicLen = buf.getInt();
                    byte[] topicBytes = new byte[topicLen];
                    buf.get(topicBytes);
                    String topic = new String(topicBytes, StandardCharsets.UTF_8);

View on GitHub (pinned to 820761864e)

Solutions

1. Serialize/deserialize with matching library versions on both sides.
2. Check the write path flushes the complete blob (verify stored length equals expected).
3. Validate your buffer offsets: the position vector (section 3) must be fully consumed before the parent-topic section.
4. Rebuild the id via toByteArray from the original MessageIdV5 object.

Example fix

// before
ByteBuffer buf = ByteBuffer.wrap(data); // data truncated
MessageIdV5 id = MessageIdV5.fromByteArray(data);
// after
if (data.length < expectedMinSize(topicName)) {
    throw new IOException("Stored id truncated (" + data.length + " bytes), re-resolve");
}
MessageIdV5 id = MessageIdV5.fromByteArray(data);

// validate overall blob size before parsing sections
int minLen = 12 + positionVectorSize + 4; // header + vector + parent-topic length field
if (data.length < minLen) {
    throw new IOException("Blob truncated: expected >= " + minLen + ", got " + data.length);
}

try {
    MessageIdV5 id = MessageIdV5.fromByteArray(data);
} catch (IOException e) {
    log.warn("Truncated/corrupt v5 id, discarding checkpoint", e);
}

When it happens

Trigger: Deserializing a truncated blob where the parent-topic length was read but the string bytes are missing; a length field written in the wrong byte order producing a huge positive value; blobs assembled with a wrong offset skipping the position vector.

Common situations: Storage layer truncating trailing bytes; hand-rolled serialization of MessageIdV5 that omits the position-vector section; cross-version formats where the parent-topic section was added later and old readers misparse new blobs.

Related errors

• Invalid MessageIdV5 data: too short
• Invalid checkpoint data: empty
• Invalid MessageIdV5 data: bad v4 length
• Invalid MessageIdV5 data: bad inner id length
• Cannot handle message with null messageId

AI-assisted analysis of apache/pulsar@820761864e (2026-09-06). Data as JSON: /api/errors/6ccfd8253b5fce8c. Report an issue: GitHub.