What causes "Failed to close deserializers" in apache/kafka?

Inspect the wrapped cause in the stack trace to identify which deserializer (key vs value) and what resource failed.

apache/kafka · error · KafkaException

Failed to close deserializers

Error message

Failed to close deserializers

What it means

KafkaException("Failed to close deserializers", cause) thrown from Deserializers.close() when either the key or value Deserializer's close() method raised a non-interrupt exception. The first exception captured by Utils.closeQuietly is propagated; InterruptException is rethrown as-is. It indicates user-supplied (or misconfigured) deserializer cleanup code failed during consumer shutdown.

Source

Thrown at clients/src/main/java/org/apache/kafka/clients/consumer/internals/Deserializers.java:87

        return keyDeserializerPlugin.get();
    }

    public Deserializer<V> valueDeserializer() {
        return valueDeserializerPlugin.get();
    }

    @Override
    public void close() {
        AtomicReference<Throwable> firstException = new AtomicReference<>();
        Utils.closeQuietly(keyDeserializerPlugin, "key deserializer", firstException);
        Utils.closeQuietly(valueDeserializerPlugin, "value deserializer", firstException);
        Throwable exception = firstException.get();

        if (exception != null) {
            if (exception instanceof InterruptException) {
                throw (InterruptException) exception;
            }
            throw new KafkaException("Failed to close deserializers", exception);
        }
    }

    @Override
    public String toString() {
        return "Deserializers{" +
                "keyDeserializer=" + keyDeserializerPlugin.get() +
                ", valueDeserializer=" + valueDeserializerPlugin.get() +
                '}';
    }
}

View on GitHub (pinned to c31c9215e1)

Solutions

Inspect the wrapped cause in the stack trace to identify which deserializer (key vs value) and what resource failed.
Make your custom Deserializer.close() idempotent and swallow or log non-fatal cleanup exceptions.
Ensure schema-registry / external clients used by the deserializer are not closed before the consumer closes.
Upgrade the deserializer dependency to match the kafka-clients version on the classpath.

Example fix

// before
public class MyJsonDeserializer implements Deserializer<MyType> {
    private ObjectMapper mapper = new ObjectMapper();
    @Override public void close() {
        mapper.getFactory().close(); // throws IOException -> wrapped as KafkaException
    }
}

// after
@Override public void close() {
    try { mapper.getFactory().close(); }
    catch (IOException e) { /* log, do not propagate */ }
}

Defensive patterns

Strategy: try-catch

Validate before calling

// You cannot fully pre-validate a deserializer's close() — it may throw for I/O reasons.
// But you CAN ensure your Deserializer implementation is non-throwing by design:
public class SafeJsonDeserializer implements Deserializer<MyType> {
    @Override public void close() {
        // Deserializers.close() (line 87) wraps the FIRST exception from key or value
        // deserializer into KafkaException("Failed to close deserializers", cause).
        // Keep this method side-effect free and non-throwing.
    }
}

Try / catch

try {
    consumer.close(); // -> Deserializers.close() -> may throw KafkaException
} catch (org.apache.kafka.common.KafkaException e) {
    if (e.getMessage() != null && e.getMessage().contains("Failed to close deserializers")) {
        // One of your key/value Deserializer.close() implementations threw.
        // The consumer itself is already closed; this is a resource-cleanup warning.
        log.warn("Deserializer cleanup failed (consumer still closed): {}", e.getCause());
        // Do not rethrow during shutdown — you would mask the real close error.
    } else {
        throw e;
    }
}

Prevention

Make every custom Deserializer.close() idempotent and non-throwing; release buffers in try/finally internally.
Always close the consumer in a try-with-resources or try/finally, so a throwing deserializer does not skip other cleanup.
Log, do not rethrow, during shutdown — the KafkaException at line 87 is a cleanup warning, not a data error.
Inspect the cause (e.getCause()) to find which deserializer (key vs value) actually failed.

When it happens

Trigger: KafkaConsumer.close() invokes Deserializers.close(), which calls close() on the configured key.deserializer / value.deserializer classes (ConsumerUtils line 87). Any exception other than InterruptException thrown by those Deserializer.close() implementations is wrapped here.

Common situations: Custom Deserializer that holds a network connection, file handle, or native resource and fails to close it; Avro/Protobuf/JSON deserializers whose schema registry client is closed elsewhere first; deserializer close() that performs I/O without handling IOException; dependency version mismatch where the deserializer class is missing a no-arg close.

Related errors

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/1e248eeb5a64157c.json. Report an issue: GitHub.