apache/pulsar · critical · ReplicationException.CompatibilityException

Invalid data found

Error message

Invalid data found

What it means

When the stored LAYOUT znode data cannot be parsed as LedgerRereplicationLayoutFormat (parseFromTextFormat throws a RuntimeException), checkLayout wraps it in ReplicationException.CompatibilityException with message "Invalid data found". It means the layout znode contains corrupted or foreign data.

Solutions

  1. Dump the LAYOUT znode content and check whether it matches the expected LedgerRereplicationLayoutFormat text form.
  2. Restore the znode from a healthy backup/snapshot of the metadata store.
  3. On a fresh/test cluster only, delete and let the manager re-create the layout znode.
  4. Check metadata store health for underlying corruption and repair per the store's tooling.
Defensive patterns

Strategy: validation

Validate before calling

try {
    new LedgerRereplicationLayoutFormat().parseFromTextFormat(store.get(layoutPath).join().get().getValue());
} catch (RuntimeException pe) {
    // layout znode is corrupt: restore from backup before starting the broker
}

Try / catch

try {
    urManager.checkLayout();
} catch (ReplicationException.CompatibilityException e) {
    // e.getCause() holds the parse failure; restore or recreate the layout znode
}

Prevention

When it happens

Trigger: Reading a LAYOUT znode whose bytes are not valid layout text — truncated writes, data written by an incompatible tool, manual znode edits, or corruption in the metadata store.

Common situations: Partial/failed writes during crashes; someone hand-edited znodes; migrating data between stores with encoding differences; metadata store corruption after disk issues.

Related errors

AI-assisted analysis of apache/pulsar@820761864e (2026-09-06). Data as JSON: /api/errors/f4b68dcb45d19923. Report an issue: GitHub.

Appendix: source

Thrown at pulsar-metadata/src/main/java/org/apache/pulsar/metadata/bookkeeper/PulsarLedgerUnderreplicationManager.java:208

                } catch (ExecutionException | InterruptedException e) {
                    if (!(e.getCause() instanceof MetadataStoreException.BadVersionException)) {
                        throw new RuntimeException(e);
                    }
                }
            } else {
                byte[] layoutData = store.get(layoutPath).join().get().getValue();

                LedgerRereplicationLayoutFormat layout = new LedgerRereplicationLayoutFormat();

                try {
                    layout.parseFromTextFormat(layoutData);
                    if (!layout.getType().equals(LAYOUT)
                            || layout.getVersion() != LAYOUT_VERSION) {
                        throw new ReplicationException.CompatibilityException(
                                "Incompatible layout found (" + LAYOUT + ":" + LAYOUT_VERSION + ")");
                    }
                } catch (RuntimeException pe) {
                    throw new ReplicationException.CompatibilityException(
                            "Invalid data found", pe);
                }
                break;
            }
        }
    }

    private long getLedgerId(String path) throws NumberFormatException {
        Matcher m = ID_EXTRACTION_PATTERN.matcher(path);
        if (m.find()) {
            return Long.parseLong(m.group(1));
        } else {
            throw new NumberFormatException("Couldn't find ledgerid in path");
        }
    }

    private static String getParentPath(String base, long ledgerId) {
        String subdir1 = String.format("%04x", ledgerId >> 48 & 0xffff);

View on GitHub (pinned to 820761864e)