Protobuf message not recognized at candidate offset

Error message

Protobuf message not recognized at candidate offset {}, falling back

What it means

tryDeserialize attempts to parse bytes at a candidate offset inside a larger buffer as a Protobuf message. If parsing fails (message bytes don't match the expected schema at that offset), it logs this warning and returns null so the caller can slide to the next offset and try again — an expected part of offset-scanning deserialization for streams without framing.

Solutions

Ensure the producer frames messages (length prefix) so deserialization starts at exact offsets
Verify the SchemaRegistry resolves the same .proto descriptor used at write time
If warnings are frequent at offset 0, the data is likely not Protobuf of the expected type — check the topic/file format
Treat the null return as designed fallback behavior; only investigate if no offset ever succeeds

Defensive patterns

Strategy: try-catch

Validate before calling

// verify bytes look like the expected message before full parse
if (message.length - offset < minMessageSize) return null;

Type guard

boolean isProtobufSlice(byte[] buf, int off) { try { schema.parseFrom(buf, off, buf.length - off); return true; } catch (InvalidProtocolBufferException e) { return false; } }

Try / catch

try { return inner.deserialize(new ByteArrayInputStream(message, offset, remaining)); } catch (IOException | RuntimeException e) { return null; } // fall back to next offset

Prevention

Frame messages with length prefixes
Keep producer/consumer descriptors in sync via the schema registry
Check data format if fallbacks happen at every offset

When it happens

Trigger: The byte slice [offset, remaining) is not a valid message of the registered Protobuf type — e.g., scanning for a message start inside a blob, corrupt/truncated record, or a message written by a different schema version.

Common situations: Misaligned offsets when messages were serialized without length prefixes; schema mismatch between producer and consumer (wrong .proto registered in the registry); corrupted storage payloads.

Understand the failure class

Background: "cannot parse invalid wire-format data", "cannot unmarshal", "failed unmarshalling": protobuf unmarshal errors explained — this error's family across 10 libraries.

Related errors

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/1f4a122c019990cc. Report an issue: GitHub.

Appendix: source

Thrown at seatunnel-formats/seatunnel-format-protobuf/src/main/java/org/apache/seatunnel/format/protobuf/SchemaRegistryAwareProtobufDeserializationSchema.java:189

     *
     * @param message the original message byte array
     * @param offset the starting offset in the array
     * @param length the total length of the array
     * @param allowEmptyPayload whether a structurally validated header may have a zero-byte payload
     * @return deserialized SeaTunnelRow, or null if parsing fails
     */
    private SeaTunnelRow tryDeserialize(
            byte[] message, int offset, int length, boolean allowEmptyPayload) {
        int remaining = length - offset;
        if (remaining < (allowEmptyPayload ? 0 : 2)) {
            return null;
        }

        try (ByteArrayInputStream inputStream =
                new ByteArrayInputStream(message, offset, remaining)) {
            return inner.deserialize(inputStream);
        } catch (IOException | RuntimeException e) {
            LOG.warn(
                    "Protobuf message not recognized at candidate offset {}, falling back",
                    offset,
                    e);
            return null;
        }
    }

    @Override
    public SeaTunnelDataType<SeaTunnelRow> getProducedType() {
        return this.rowType;
    }
}

View on GitHub (pinned to cf67b549a7)