apache/kafka · error · TimeoutException

Timeout expired while fetching topic metadata

Error message

Timeout expired while fetching topic metadata

What it means

Thrown as TimeoutException by TopicMetadataFetcher.getTopicMetadata when the retry loop exhausts the supplied Timer (driven by the caller's request.timeout.ms / the method's deadline) before a successful, error-free MetadataResponse is received. Retriable errors keep the loop spinning with retryBackoff sleeps; once timer.notExpired() returns false, the call fails. This is purely a wall-clock exhaustion signal, not a protocol error.

Source

Thrown at clients/src/main/java/org/apache/kafka/clients/consumer/internals/TopicMetadataFetcher.java:152

                            shouldRetry = true;
                        else
                            throw new KafkaException("Unexpected error fetching metadata for topic " + topic,
                                    error.exception());
                    }
                }

                if (!shouldRetry) {
                    HashMap<String, List<PartitionInfo>> topicsPartitionInfos = new HashMap<>();
                    for (String topic : cluster.topics())
                        topicsPartitionInfos.put(topic, cluster.partitionsForTopic(topic));
                    return topicsPartitionInfos;
                }
            }

            timer.sleep(retryBackoff.backoff(attempts++));
        } while (timer.notExpired());

        throw new TimeoutException("Timeout expired while fetching topic metadata");
    }

    /**
     * Send Metadata Request to the least loaded node in Kafka cluster asynchronously
     * @return A future that indicates result of sent metadata request
     */
    private RequestFuture<ClientResponse> sendMetadataRequest(MetadataRequest.Builder request) {
        final Node node = client.leastLoadedNode();
        if (node == null)
            return RequestFuture.noBrokersAvailable();
        else
            return client.send(node, request);
    }

}

View on GitHub (pinned to c31c9215e1)

Solutions

1. Validate connectivity: telnet/bootstrap or nc -zv <broker-host> <port> from the client host; correct bootstrap.servers.
2. Raise request.timeout.ms (e.g. 60000) and lower retry.backoff.ms so the loop can complete more attempts within the deadline.
3. Check broker liveness via kafka-topics --bootstrap-server ... --list from the same host; if it also times out, the issue is network/cluster-side.
4. During cluster events (restart, reassignment), wait for controller election / ISRs to stabilize before starting consumers.

Example fix

// before
props.put(ConsumerConfig.REQUEST_TIMEOUT_MS_CONFIG, "10000");
props.put(ConsumerConfig.RETRY_BACKOFF_MS_CONFIG, "100");

// after
props.put(ConsumerConfig.REQUEST_TIMEOUT_MS_CONFIG, "60000");
props.put(ConsumerConfig.RETRY_BACKOFF_MS_CONFIG, "500");

// Verify broker reachability and metadata service before subscribing.
try (java.net.Socket s = new java.net.Socket()) {
    s.connect(new java.net.InetSocketAddress(bootstrapHost, bootstrapPort), 2000);
}
// Tune timeouts so the deadline fits your network RTT:
props.put(ConsumerConfig.REQUEST_TIMEOUT_MS_CONFIG, 30000);
props.put(ConsumerConfig.RETRY_BACKOFF_MS_CONFIG, 500);

import org.apache.kafka.common.errors.TimeoutException;

/** True iff a timeout is specifically the 'fetching topic metadata' timeout (vs. other Kafka timeouts). */
static boolean isMetadataFetchTimeout(Throwable t) {
    return t instanceof TimeoutException
        && t.getMessage() != null
        && t.getMessage().contains("fetching topic metadata");
}

long deadline = System.currentTimeMillis() + 60_000;
while (System.currentTimeMillis() < deadline) {
    try {
        return consumer.partitionsFor(topic);
    } catch (org.apache.kafka.common.errors.TimeoutException e) {
        if (!isMetadataFetchTimeout(e)) throw e;
        // exponential backoff with jitter before retry
        sleepBackoff();
    }
}
throw new org.apache.kafka.common.errors.TimeoutException("metadata fetch gave up after retries");

When it happens

Trigger: Calling consumer.partitionsFor(topic) or listTopics() when every broker is unreachable, when a broker is reachable but keeps returning retriable errors (LEADER_NOT_AVAILABLE during controller handover, NETWORK_EXCEPTION), or when retry.backoff.ms × number-of-retries exceeds request.timeout.ms. Also fires during cluster startup, partition reassignment, or network partition.

Common situations: Misconfigured bootstrap.servers (typo, wrong port, DNS points to a decommissioned broker); broker still starting up (controller not elected); firewall/security group dropping the connection; request.timeout.ms too low relative to retry.backoff.ms; client pointed at a wrong cluster/VPC; long GC pause on broker extending every fetch.

Related errors

• Failed to get offsets by times in {}ms
• Operation timed out before completion
• Failed to get offsets by times in {}ms
• Timeout of {}ms expired before the position for partition {}
• Timeout of {}ms expired before the last committed offset for

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/732cecb0383317bf.json. Report an issue: GitHub.