apache/pulsar · error · IOException

Invalid checkpoint data: empty

Error message

Invalid checkpoint data: empty

What it means

CheckpointV5.fromByteArray deserializes a checkpoint from its byte encoding, which must start with a type byte. If the byte array is null or has zero length, there is no type byte to read, so it throws IOException('Invalid checkpoint data: empty'). This guards against corrupt or truncated stored checkpoints.

Source

Thrown at pulsar-client-v5/src/main/java/org/apache/pulsar/client/impl/v5/CheckpointV5.java:79

            byte[] idBytes = entry.getValue().toByteArray();
            serializedIds.put(entry.getKey(), idBytes);
            totalSize += 8 + 4 + idBytes.length;
        }

        ByteBuffer buf = ByteBuffer.allocate(totalSize);
        buf.put(TYPE_REGULAR);
        buf.putInt(segmentPositions.size());
        for (var entry : serializedIds.entrySet()) {
            buf.putLong(entry.getKey());
            buf.putInt(entry.getValue().length);
            buf.put(entry.getValue());
        }
        return buf.array();
    }

    static Checkpoint fromByteArray(byte[] data) throws IOException {
        if (data == null || data.length < 1) {
            throw new IOException("Invalid checkpoint data: empty");
        }

        ByteBuffer buf = ByteBuffer.wrap(data);
        byte type = buf.get();

        return switch (type) {
            case TYPE_EARLIEST -> EARLIEST;
            case TYPE_LATEST -> LATEST;
            case TYPE_REGULAR -> {
                int numEntries = buf.getInt();
                Map<Long, org.apache.pulsar.client.api.MessageId> positions = new HashMap<>();
                for (int i = 0; i < numEntries; i++) {
                    long segmentId = buf.getLong();
                    int msgIdLen = buf.getInt();
                    byte[] msgIdBytes = new byte[msgIdLen];
                    buf.get(msgIdBytes);
                    positions.put(segmentId,
                            org.apache.pulsar.client.api.MessageId.fromByteArray(msgIdBytes));

View on GitHub (pinned to 820761864e)

Solutions

1. Check the checkpoint byte array is non-null and non-empty before calling fromByteArray.
2. Regenerate the checkpoint by re-consuming/re-acknowledging to write a fresh checkpoint.
3. Verify the storage backend did not truncate the record (check write flush, record size metadata).

Example fix

// before
Checkpoint cp = CheckpointV5.fromByteArray(raw);
// after
if (raw == null || raw.length < 1) {
    throw new IOException("No checkpoint stored, starting fresh");
}
Checkpoint cp = CheckpointV5.fromByteArray(raw);

if (data == null || data.length < 1) {
    // no checkpoint stored — start fresh
    return null;
}

try {
    checkpoint = CheckpointV5.fromByteArray(data);
} catch (IOException e) {
    log.warn("Unreadable checkpoint, starting fresh", e);
    checkpoint = null;
}

When it happens

Trigger: Calling CheckpointV5.fromByteArray(null) or fromByteArray(new byte[0]) — typically from a corrupted or empty checkpoint record read back from storage.

Common situations: Restoring consumer state after a crash where the checkpoint file/record was truncated or never written; migration tooling feeding legacy/empty blobs into the v5 deserializer.

Related errors

• Unknown checkpoint type: ${type}
• Invalid MessageIdV5 data: too short
• Invalid MessageIdV5 data: bad parent-topic length
• payload (${payloadSize} bytes) cannot be decoded with schema
• Failed to decode message from topic ${topic} with schemaId $

AI-assisted analysis of apache/pulsar@820761864e (2026-09-06). Data as JSON: /api/errors/65af354944d33f8e. Report an issue: GitHub.