apache/kafka · error · SchemaException

Error reading field '${name}': ${detail}

Error message

Error reading field '${name}': ${detail}

What it means

Thrown by Schema.read when the element Type.read for a struct field raises any exception. The wrapper attaches the offending field's name and the underlying cause's message (or class name), turning a generic buffer failure into a field-localized error. It usually masks a more specific exception from the field's Type (e.g. ArrayOf, STRING, INT*).

Source

Thrown at clients/src/main/java/org/apache/kafka/common/protocol/types/Schema.java:120

            return cachedStruct;
        }
        Object[] objects = new Object[fields.length];
        for (int i = 0; i < fields.length; i++) {
            try {
                if (tolerateMissingFieldsWithDefaults) {
                    if (buffer.hasRemaining()) {
                        objects[i] = fields[i].def.type.read(buffer);
                    } else if (fields[i].def.hasDefaultValue) {
                        objects[i] = fields[i].def.defaultValue;
                    } else {
                        throw new SchemaException("Missing value for field '" + fields[i].def.name +
                                "' which has no default value.");
                    }
                } else {
                    objects[i] = fields[i].def.type.read(buffer);
                }
            } catch (Exception e) {
                throw new SchemaException("Error reading field '" + fields[i].def.name + "': " +
                                          (e.getMessage() == null ? e.getClass().getName() : e.getMessage()));
            }
        }
        return new Struct(this, objects);
    }

    /**
     * The size of the given record
     */
    @Override
    public int sizeOf(Object o) {
        int size = 0;
        Struct r = (Struct) o;
        for (BoundField field : fields) {
            try {
                size += field.def.type.sizeOf(r.get(field));
            } catch (Exception e) {
                throw new SchemaException("Error computing size for field '" + field.def.name + "': " +

View on GitHub (pinned to c31c9215e1)

Solutions

1. Read the named field, then check the underlying message (e.g. 'Error reading array of size ...') which pinpoints the real failure in the element type.
2. Confirm the schema's field order/types match the API key + API version on the wire.
3. Verify the ByteBuffer is not exhausted and was not over-sliced before this struct.
4. For log/record data, validate integrity (kafka-dump-log) and confirm the producer version matches.

Example fix

// before - generic catch hides the real cause
catch (SchemaException e) { log.error("read failed", e); }

// after - surface the underlying field type error
try {
    return (Struct) schema.read(buf);
} catch (SchemaException e) {
    throw new IOException("failed reading field set beginning at offset " + start + ": " + e.getMessage(), e);
}

// The 'detail' comes from the inner Type.read; you cannot pre-validate it
// generically. Ensure the buffer has at least one byte per remaining field
// as a cheap sanity check before read.
int remainingFields = schema.fields().length - fieldsAlreadyRead;
if (buffer.remaining() < remainingFields) {
    // likely truncated payload; read() will wrap the cause into this error
    log.debug("buffer may be short for {} fields", remainingFields);
}

try {
    Struct s = (Struct) schema.read(buffer);
} catch (org.apache.kafka.common.protocol.types.SchemaException e) {
    if (e.getMessage().startsWith("Error reading field '")) {
        // The suffix is the inner cause (e.g. a nested ArrayOf/TaggedFields
        // failure). Surface the field name and the inner detail separately.
        String field = extractQuoted(e.getMessage(), "Error reading field '");
        log.error("Failed to deserialize field '{}': {}", field, innerDetail(e));
        throw new MalformedRecordException(field, e);
    }
    throw e;
}

When it happens

Trigger: Schema.read calls fields[i].def.type.read(buffer) inside a try/catch; any exception - SchemaException from ArrayOf/CompactArrayOf/TaggedFields, BufferUnderflowException, or a NumberFormatException from a primitive reader - is rethrown with this message naming fields[i].def.name.

Common situations: Decoding with the wrong API version's schema so a later field reads bytes that belong to the next field; truncated payload causing BufferUnderflowException partway through a struct; a nested array/string field hit one of its own length guards; corrupted log replay.

Related errors

• Array size ${size} cannot be negative
• Error reading array of size ${size}, only ${remaining} bytes
• Error writing field '${name}': ${detail}
• Buffer underflow while parsing consumer protocol's header
• Malformed consumer protocol subscription

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/22aad3b765e3ae58.json. Report an issue: GitHub.