{"id":"7db75c5995925f33","repo":"apache/kafka","slug":"invalid-or-out-of-order-tag-tag","errorCode":null,"errorMessage":"Invalid or out-of-order tag ${tag}","messagePattern":"Invalid or out-of-order tag (.+?)","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"clients/src/main/java/org/apache/kafka/common/protocol/types/TaggedFields.java","lineNumber":92,"sourceCode":"            } else {\n                ByteUtils.writeUnsignedVarint(field.type.sizeOf(entry.getValue()), buffer);\n                field.type.write(buffer, entry.getValue());\n            }\n        }\n    }\n\n    @Override\n    public NavigableMap<Integer, Object> read(ByteBuffer buffer) {\n        int numTaggedFields = ByteUtils.readUnsignedVarint(buffer);\n        if (numTaggedFields == 0) {\n            return Collections.emptyNavigableMap();\n        }\n        NavigableMap<Integer, Object> objects = new TreeMap<>();\n        int prevTag = -1;\n        for (int i = 0; i < numTaggedFields; i++) {\n            int tag = ByteUtils.readUnsignedVarint(buffer);\n            if (tag <= prevTag) {\n                throw new RuntimeException(\"Invalid or out-of-order tag \" + tag);\n            }\n            prevTag = tag;\n            int size = ByteUtils.readUnsignedVarint(buffer);\n            if (size < 0)\n                throw new SchemaException(\"field size \" + size + \" cannot be negative\");\n            if (size > buffer.remaining())\n                throw new SchemaException(\"Error reading field of size \" + size + \", only \" + buffer.remaining() + \" bytes available\");\n\n            Field field = fields.get(tag);\n            if (field == null) {\n                byte[] bytes = new byte[size];\n                buffer.get(bytes);\n                objects.put(tag, new RawTaggedField(tag, bytes));\n            } else {\n                objects.put(tag, field.type.read(buffer));\n            }\n        }\n        return objects;","sourceCodeStart":74,"sourceCodeEnd":110,"githubUrl":"https://github.com/apache/kafka/blob/c31c9215e131f8c17e79f8901b48c13ee6aa8e7a/clients/src/main/java/org/apache/kafka/common/protocol/types/TaggedFields.java#L74-L110","documentation":"Thrown by TaggedFields.read when a tagged-field tag is not strictly greater than the previous one. Tagged fields (KIP-482) must appear in ascending tag order so the receiver can stream-decode them; a repeat or out-of-order tag signals malformed or tampered input. Unlike most errors here this is raised as a plain RuntimeException because the contract is broken at the framing level.","triggerScenarios":"TaggedFields.read loops numTaggedFields times; for each it reads an unsigned varint tag and checks `tag <= prevTag` (prevTag starts at -1). Triggered by a writer that emitted tags unordered, duplicated a tag, or by buffer misalignment causing garbage varints to be read as tags.","commonSituations":"Custom code that builds a NavigableMap incorrectly (non-sorted or duplicate keys) before calling TaggedFields.write; a man-in-the-middle or corrupted frame; reading a payload with a buffer position that is off by some bytes so the tag varint decodes to a small/repeat value; tests that hand-craft tagged-field bytes in the wrong order.","solutions":["Ensure the writer uses a sorted map (TreeMap/NavigableMap) with unique integer tags so write emits them in ascending order.","Verify the buffer position aligns with the tagged-fields section (i.e. the parent struct fields were read first).","If forwarding unknown tagged fields, preserve the original RawTaggedField entries in ascending tag order.","Regenerate message specs (./gradlew processMessages) so tag assignments are authoritative and never duplicated."],"exampleFix":"// before - HashMap allows out-of-order writes\nMap<Integer, Object> tagged = new HashMap<>();\ntagged.put(2, v2);\ntagged.put(1, v1);\n\n// after - sorted, unique tags\ntagged = new TreeMap<>(tagged);","handlingStrategy":"try-catch","validationCode":"// Tags are decoded inside TaggedFields.read and must be strictly ascending.\n// The reader cannot reorder them beforehand. If YOU are writing tagged fields,\n// emit them in ascending tag order using a NavigableMap so this never triggers:\nNavigableMap<Integer, Object> out = new TreeMap<>();   // ascending by contract\nout.put(2, valueFor2);\nout.put(5, valueFor5);\ntaggedFields.write(buffer, out);                        // safe: TreeMap iterates ascending","typeGuard":"// On the WRITE path: ensure the object is a NavigableMap whose keys ascend.\nstatic boolean isAscendingTaggedMap(Object o) {\n    if (!(o instanceof NavigableMap)) return false;\n    NavigableMap<?, ?> m = (NavigableMap<?, ?>) o;\n    Integer prev = -1;\n    for (Object k : m.keySet()) {\n        if (!(k instanceof Integer) || (Integer) k <= prev) return false;\n        prev = (Integer) k;\n    }\n    return true;\n}","tryCatchPattern":"// NOTE: line 92 throws RuntimeException, NOT SchemaException. Catch broadly.\ntry {\n    NavigableMap<Integer, Object> tf = taggedFields.read(buffer);\n} catch (RuntimeException e) {\n    if (e.getMessage() != null && e.getMessage().startsWith(\"Invalid or out-of-order tag\")) {\n        // corrupt or maliciously ordered tagged fields on the wire.\n        log.warn(\"Rejecting frame with unordered tags: {}\", e.getMessage());\n        throw new CorruptFrameException(e);\n    }\n    throw e;\n}","preventionTips":["When writing tagged fields, always pass a TreeMap/NavigableMap so iteration order is ascending by tag.","Never insert raw tag values out of order; the wire format forbids duplicate or non-increasing tags.","Remember this failure surfaces as RuntimeException, not SchemaException, so don't only catch SchemaException.","Reject and close the connection on this error; it indicates a malformed or hostile peer."],"tags":["protocol","serialization","tagged-fields","flexible-versions","kafka-clients"],"analyzedSha":"c31c9215e131f8c17e79f8901b48c13ee6aa8e7a","analyzedAt":"2026-08-03T12:34:05.770Z","schemaVersion":2}