apache/kafka · error · KafkaException

Unexpected error fetching metadata for topic ${topic}

Error message

Unexpected error fetching metadata for topic ${topic}

What it means

Thrown as a generic KafkaException by TopicMetadataFetcher.getTopicMetadata for any non-retriable, non-INVALID_TOPIC, non-UNKNOWN_TOPIC_OR_PARTITION error returned by the broker in MetadataResponse.errors(). It is the catch-all branch when none of the recognized error codes match, wrapping the underlying error.exception() as the cause. Treat the wrapped cause as authoritative for diagnosis.

Source

Thrown at clients/src/main/java/org/apache/kafka/clients/consumer/internals/TopicMetadataFetcher.java:136

                    // if there were errors, we need to check whether they were fatal or whether
                    // we should just retry

                    log.debug("Topic metadata fetch included errors: {}", errors);

                    for (Map.Entry<String, Errors> errorEntry : errors.entrySet()) {
                        String topic = errorEntry.getKey();
                        Errors error = errorEntry.getValue();

                        if (error == Errors.INVALID_TOPIC_EXCEPTION)
                            throw new InvalidTopicException("Topic '" + topic + "' is invalid");
                        else if (error == Errors.UNKNOWN_TOPIC_OR_PARTITION)
                            // if a requested topic is unknown, we just continue and let it be absent
                            // in the returned map
                            continue;
                        else if (error.exception() instanceof RetriableException)
                            shouldRetry = true;
                        else
                            throw new KafkaException("Unexpected error fetching metadata for topic " + topic,
                                    error.exception());
                    }
                }

                if (!shouldRetry) {
                    HashMap<String, List<PartitionInfo>> topicsPartitionInfos = new HashMap<>();
                    for (String topic : cluster.topics())
                        topicsPartitionInfos.put(topic, cluster.partitionsForTopic(topic));
                    return topicsPartitionInfos;
                }
            }

            timer.sleep(retryBackoff.backoff(attempts++));
        } while (timer.notExpired());

        throw new TimeoutException("Timeout expired while fetching topic metadata");
    }

View on GitHub (pinned to c31c9215e1)

Solutions

Inspect the wrapped cause (exception.getCause()) — the underlying org.apache.kafka.common.protocol.Errors code is the real signal.
Align client and broker versions to avoid unrecognized error codes (check BrokerProtocolCompatibility / apiVersions response).
If the cause is authorization-related, follow the TopicAuthorizationException remediation (grant ACLs).
Enable DEBUG on org.apache.kafka.clients.consumer.internals.TopicMetadataFetcher to log the full error map ('Topic metadata fetch included errors: ...').

Example fix

// before
try {
    consumer.partitionsFor(topic);
} catch (KafkaException e) {
    log.error("metadata failed", e); // cause is hidden
}

// after
try {
    consumer.partitionsFor(topic);
} catch (KafkaException e) {
    Throwable cause = e.getCause() != null ? e.getCause() : e;
    log.error("metadata failed; broker error code: {}", cause.getClass().getSimpleName(), cause);
}

Defensive patterns

Strategy: try-catch

Validate before calling

// No reliable client-side precheck for arbitrary broker-side metadata errors.
// Best mitigation: keep client lib version compatible with broker version, and preflight reachability:
try (java.net.Socket s = new java.net.Socket()) {
    s.connect(new java.net.InetSocketAddress(bootstrapHost, bootstrapPort), 2000);
}

Type guard

import org.apache.kafka.common.KafkaException;

/** True iff a throwable is the 'Unexpected error fetching metadata' wrapper with a classifiable cause. */
static boolean isUnexpectedMetadataError(Throwable t) {
    if (!(t instanceof KafkaException)) return false;
    String m = t.getMessage();
    return m != null && m.startsWith("Unexpected error fetching metadata for topic");
}

Try / catch

try {
    consumer.partitionsFor(topic);
} catch (org.apache.kafka.common.KafkaException e) {
    Throwable cause = e.getCause();
    log.error("Unexpected metadata error for {}; cause={}:{}", topic,
        cause == null ? null : cause.getClass().getName(),
        cause == null ? null : cause.getMessage(), e);
    // classify: if cause is retriable (e.g. LeaderNotSelected) -> backoff retry; else -> escalate
    if (cause instanceof org.apache.kafka.common.errors.RetriableException) { /* retry */ }
    else throw e;
}

Prevention

Log and inspect getCause() — the wrapper hides the real broker error.
Keep the kafka-clients version within the supported matrix for your broker version.
Surface the underlying cause to monitoring rather than the generic wrapper message.
Add retry/backoff only when the cause is a RetriableException; otherwise escalate.

When it happens

Trigger: Broker returns an unexpected Errors code in the per-topic error map during a MetadataResponse (e.g. TOPIC_AUTHORIZATION_FAILED not surfaced via unauthorizedTopics, CLUSTER_AUTHORIZATION_FAILED, INVALID_RECORD, or a future/unknown error code from a newer broker). Also possible if the error map contains a code the client does not branch on.

Common situations: Client/broker version skew where a newer broker emits an error code the older client does not handle explicitly; non-standard broker or proxy translating errors; rare authorization paths that don't populate unauthorizedTopics; corrupted/translated metadata responses from a sidecar.

Related errors

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/7334cc589596c6e8.json. Report an issue: GitHub.