apache/pulsar · error · MetadataFormatException

Unrecognized package metadata format

Error message

Unrecognized package metadata format

What it means

MetadataFormatException thrown by PackageMetadataUtil.fromBytes when the payload neither starts with the JSON leading byte '{' nor the Java serialization magic bytes (0xAC 0xED), so its format cannot be recognized.

Source

Thrown at pulsar-package-management/core/src/main/java/org/apache/pulsar/packages/management/core/common/PackageMetadataUtil.java:91

    public static PackageMetadata fromBytes(byte[] bytes, boolean allowLegacyJavaSerialization)
            throws MetadataFormatException {
        if (bytes == null || bytes.length == 0) {
            throw new MetadataFormatException("Empty package metadata");
        }
        int firstNonWhitespace = indexOfFirstNonWhitespace(bytes);
        if (firstNonWhitespace >= 0 && bytes[firstNonWhitespace] == JSON_LEADING_BYTE) {
            return readJson(bytes);
        }
        if (bytes.length >= 2 && bytes[0] == JAVA_MAGIC_BYTE_0 && bytes[1] == JAVA_MAGIC_BYTE_1) {
            if (!allowLegacyJavaSerialization) {
                throw new MetadataFormatException(
                        "Package metadata is in legacy Java serialization format but reading it is disabled. "
                                + "Enable packagesManagementAllowLegacyJavaSerialization or re-upload the package.");
            }
            return readLegacy(bytes);
        }
        throw new MetadataFormatException("Unrecognized package metadata format");
    }

    private static PackageMetadata readJson(byte[] bytes) throws MetadataFormatException {
        try {
            return JSON_READER.readValue(bytes);
        } catch (IOException e) {
            throw new MetadataFormatException("Failed to parse package metadata as JSON: " + e.getMessage());
        }
    }

    private static PackageMetadata readLegacy(byte[] bytes) throws MetadataFormatException {
        try (ObjectInputStream ois = new ObjectInputStream(new ByteArrayInputStream(bytes))) {
            ois.setObjectInputFilter(LEGACY_FILTER);
            Object o = ois.readObject();
            if (!(o instanceof PackageMetadata)) {
                throw new MetadataFormatException("Unexpected metadata type: "
                        + (o == null ? "null" : o.getClass().getName()));
            }

View on GitHub (pinned to 820761864e)

Solutions

1. Re-upload the package to rewrite a valid JSON metadata blob.
2. Inspect the first bytes of the stored payload to identify the actual format (compression, BOM, corruption).
3. Check for proxy/storage layers that transform bytes (compression or encryption) and disable the transform.
4. If the data is legacy from an unknown source, restore it from a known-good backup or re-create the package.

Example fix

// before
byte[] blob = gzip(decompress(raw)); // transformed payload stored
// after
byte[] blob = raw; // store plain JSON bytes as the format expects

static boolean looksLikeKnownFormat(byte[] b) {
    if (b == null || b.length == 0) return false;
    if (b[0] == '{') return true;                     // JSON
    return b.length >= 2 && b[0] == (byte) 0xAC && b[1] == (byte) 0xED; // java serialization
}

try {
    PackageMetadata m = PackageMetadataUtil.fromBytes(bytes, allowLegacy);
} catch (MetadataFormatException e) {
    if (e.getMessage() != null && e.getMessage().contains("Unrecognized")) {
        // dump first bytes hex for diagnosis, then restore from backup or re-upload
    }
}

When it happens

Trigger: Reading a metadata blob that was written by an incompatible writer, corrupted (e.g. compressed/encrypted/truncated), or has leading whitespace/bytes prepended before the JSON.

Common situations: Manual edits or migrations of package storage data, storing gzip-compressed metadata where raw bytes were expected, or a bug writing a BOM/extra bytes before the JSON payload.

Related errors

• Empty package metadata
• Package metadata is in legacy Java serialization format but
• Failed to parse package metadata as JSON: ${message}
• Cursor %s mark-delete position %s is ahead of the last posit
• Package Management Service is not enabled in the broker.

AI-assisted analysis of apache/pulsar@820761864e (2026-09-06). Data as JSON: /api/errors/f24519cefd437bd7. Report an issue: GitHub.