What causes "Invalid or out-of-order tag ${tag}" in apache/kafka?

Ensure the writer uses a sorted map (TreeMap/NavigableMap) with unique integer tags so write emits them in ascending order.

apache/kafka · error · RuntimeException

Invalid or out-of-order tag ${tag}

Error message

Invalid or out-of-order tag ${tag}

What it means

Thrown by TaggedFields.read when a tagged-field tag is not strictly greater than the previous one. Tagged fields (KIP-482) must appear in ascending tag order so the receiver can stream-decode them; a repeat or out-of-order tag signals malformed or tampered input. Unlike most errors here this is raised as a plain RuntimeException because the contract is broken at the framing level.

Source

Thrown at clients/src/main/java/org/apache/kafka/common/protocol/types/TaggedFields.java:92

            } else {
                ByteUtils.writeUnsignedVarint(field.type.sizeOf(entry.getValue()), buffer);
                field.type.write(buffer, entry.getValue());
            }
        }
    }

    @Override
    public NavigableMap<Integer, Object> read(ByteBuffer buffer) {
        int numTaggedFields = ByteUtils.readUnsignedVarint(buffer);
        if (numTaggedFields == 0) {
            return Collections.emptyNavigableMap();
        }
        NavigableMap<Integer, Object> objects = new TreeMap<>();
        int prevTag = -1;
        for (int i = 0; i < numTaggedFields; i++) {
            int tag = ByteUtils.readUnsignedVarint(buffer);
            if (tag <= prevTag) {
                throw new RuntimeException("Invalid or out-of-order tag " + tag);
            }
            prevTag = tag;
            int size = ByteUtils.readUnsignedVarint(buffer);
            if (size < 0)
                throw new SchemaException("field size " + size + " cannot be negative");
            if (size > buffer.remaining())
                throw new SchemaException("Error reading field of size " + size + ", only " + buffer.remaining() + " bytes available");

            Field field = fields.get(tag);
            if (field == null) {
                byte[] bytes = new byte[size];
                buffer.get(bytes);
                objects.put(tag, new RawTaggedField(tag, bytes));
            } else {
                objects.put(tag, field.type.read(buffer));
            }
        }
        return objects;

View on GitHub (pinned to c31c9215e1)

Solutions

Ensure the writer uses a sorted map (TreeMap/NavigableMap) with unique integer tags so write emits them in ascending order.
Verify the buffer position aligns with the tagged-fields section (i.e. the parent struct fields were read first).
If forwarding unknown tagged fields, preserve the original RawTaggedField entries in ascending tag order.
Regenerate message specs (./gradlew processMessages) so tag assignments are authoritative and never duplicated.

Example fix

// before - HashMap allows out-of-order writes
Map<Integer, Object> tagged = new HashMap<>();
tagged.put(2, v2);
tagged.put(1, v1);

// after - sorted, unique tags
tagged = new TreeMap<>(tagged);

Defensive patterns

Strategy: try-catch

Validate before calling

// Tags are decoded inside TaggedFields.read and must be strictly ascending.
// The reader cannot reorder them beforehand. If YOU are writing tagged fields,
// emit them in ascending tag order using a NavigableMap so this never triggers:
NavigableMap<Integer, Object> out = new TreeMap<>();   // ascending by contract
out.put(2, valueFor2);
out.put(5, valueFor5);
taggedFields.write(buffer, out);                        // safe: TreeMap iterates ascending

Type guard

// On the WRITE path: ensure the object is a NavigableMap whose keys ascend.
static boolean isAscendingTaggedMap(Object o) {
    if (!(o instanceof NavigableMap)) return false;
    NavigableMap<?, ?> m = (NavigableMap<?, ?>) o;
    Integer prev = -1;
    for (Object k : m.keySet()) {
        if (!(k instanceof Integer) || (Integer) k <= prev) return false;
        prev = (Integer) k;
    }
    return true;
}

Try / catch

// NOTE: line 92 throws RuntimeException, NOT SchemaException. Catch broadly.
try {
    NavigableMap<Integer, Object> tf = taggedFields.read(buffer);
} catch (RuntimeException e) {
    if (e.getMessage() != null && e.getMessage().startsWith("Invalid or out-of-order tag")) {
        // corrupt or maliciously ordered tagged fields on the wire.
        log.warn("Rejecting frame with unordered tags: {}", e.getMessage());
        throw new CorruptFrameException(e);
    }
    throw e;
}

Prevention

When writing tagged fields, always pass a TreeMap/NavigableMap so iteration order is ascending by tag.
Never insert raw tag values out of order; the wire format forbids duplicate or non-increasing tags.
Remember this failure surfaces as RuntimeException, not SchemaException, so don't only catch SchemaException.
Reject and close the connection on this error; it indicates a malformed or hostile peer.

When it happens

Trigger: TaggedFields.read loops numTaggedFields times; for each it reads an unsigned varint tag and checks `tag <= prevTag` (prevTag starts at -1). Triggered by a writer that emitted tags unordered, duplicated a tag, or by buffer misalignment causing garbage varints to be read as tags.

Common situations: Custom code that builds a NavigableMap incorrectly (non-sorted or duplicate keys) before calling TaggedFields.write; a man-in-the-middle or corrupted frame; reading a payload with a buffer position that is off by some bytes so the tag varint decodes to a small/repeat value; tests that hand-craft tagged-field bytes in the wrong order.

Related errors

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/7db75c5995925f33.json. Report an issue: GitHub.