apache/pulsar · error · MetadataStoreException

Invalid MetaValue data, no enough header data. expect %d, ac

Error message

Invalid MetaValue data, no enough header data. expect %d, actual %d

What it means

MetaValue.parse deserializes the RocksDB-stored value blob, which begins with a header: an int headerSize, a format version int, and metadata fields. This error means the stored byte array is shorter than the declared header size, so the header cannot be read fully. The library throws it to guard against corrupt, truncated, or non-MetaValue data stored under a path.

Source

Thrown at pulsar-metadata/src/main/java/org/apache/pulsar/metadata/impl/RocksdbMetadataStore.java:163

            buffer.putLong(createdTimestamp);
            buffer.putLong(modifiedTimestamp);
            buffer.put((byte) (ephemeral ? 1 : 0));
            buffer.put(data);
            return result;
        }

        public static MetaValue parse(byte[] dataBytes) throws MetadataStoreException {
            if (dataBytes == null) {
                return null;
            }
            if (dataBytes.length < 4) {
                throw new MetadataStoreException("Invalid MetaValue data, size=" + dataBytes.length);
            }
            ByteBuffer buffer = ByteBuffer.wrap(dataBytes);
            MetaValue metaValue = new MetaValue();
            int headerSize = buffer.getInt();
            if (dataBytes.length < headerSize) {
                throw new MetadataStoreException(
                        String.format("Invalid MetaValue data, no enough header data. expect %d, actual %d",
                                headerSize, dataBytes.length));
            }
            int formatVersion = buffer.getInt();
            if (formatVersion >= FORMAT_VERSION_V1) {
                metaValue.version = buffer.getLong();
                metaValue.owner = buffer.getLong();
                metaValue.createdTimestamp = buffer.getLong();
                metaValue.modifiedTimestamp = buffer.getLong();
                metaValue.ephemeral = buffer.get() > 0;
            } else {
                throw new MetadataStoreException("Invalid MetaValue format version=" + formatVersion);
            }
            buffer.position(headerSize);
            metaValue.data = new byte[buffer.remaining()];
            buffer.get(metaValue.data);
            return metaValue;
        }

View on GitHub (pinned to 820761864e)

Solutions

1. Inspect the offending path's raw value in RocksDB and delete it so it can be rewritten (e.g. via a rocksdb tool or the store delete API bypassing parse).
2. Verify the data directory is not shared with, or populated by, another application or an older incompatible data layout.
3. Restore the RocksDB data directory from a known-good backup.
4. If corruption is widespread, wipe the metadata store and re-create metadata (recovery from source systems).

static boolean looksLikeMetaValue(byte[] data) {
    if (data == null || data.length < 8) return false;
    int headerSize = ByteBuffer.wrap(data).getInt();
    return headerSize > 0 && headerSize <= data.length;
}

static boolean isParseable(byte[] data) {
    try { MetaValue.parse(data); return true; } catch (MetadataStoreException e) { return false; }
}

try {
    MetaValue mv = MetaValue.parse(raw);
} catch (MetadataStoreException e) {
    log.warn("Corrupt metadata value, will recreate", e);
    // delete and rewrite the record
}

When it happens

Trigger: Calling storeGet/storeDelete/storePut on a RocksdbMetadataStore path whose raw value bytes were written by another tool, hand-corrupted in RocksDB, or truncated on disk, such that dataBytes.length < the headerSize int at offset 0.

Common situations: RocksDB data directory copied or restored partially from backup; a different application sharing the same data directory wrote its own bytes; disk corruption; manually editing values in the DB.

Related errors

• Invalid MetaValue format version=${version}
• payload (${payloadSize} bytes) cannot be decoded with schema
• Failed to decode message from topic ${topic} with schemaId $
• Failed to load config into existing configuration data
• Invalid checkpoint data: empty

AI-assisted analysis of apache/pulsar@820761864e (2026-09-06). Data as JSON: /api/errors/dd9c4f3ca5ae37df. Report an issue: GitHub.