{"id":"732cecb0383317bf","repo":"apache/kafka","slug":"timeout-expired-while-fetching-topic-metadata","errorCode":null,"errorMessage":"Timeout expired while fetching topic metadata","messagePattern":"Timeout expired while fetching topic metadata","errorType":"exception","errorClass":"TimeoutException","httpStatus":null,"severity":"error","filePath":"clients/src/main/java/org/apache/kafka/clients/consumer/internals/TopicMetadataFetcher.java","lineNumber":152,"sourceCode":"                            shouldRetry = true;\n                        else\n                            throw new KafkaException(\"Unexpected error fetching metadata for topic \" + topic,\n                                    error.exception());\n                    }\n                }\n\n                if (!shouldRetry) {\n                    HashMap<String, List<PartitionInfo>> topicsPartitionInfos = new HashMap<>();\n                    for (String topic : cluster.topics())\n                        topicsPartitionInfos.put(topic, cluster.partitionsForTopic(topic));\n                    return topicsPartitionInfos;\n                }\n            }\n\n            timer.sleep(retryBackoff.backoff(attempts++));\n        } while (timer.notExpired());\n\n        throw new TimeoutException(\"Timeout expired while fetching topic metadata\");\n    }\n\n    /**\n     * Send Metadata Request to the least loaded node in Kafka cluster asynchronously\n     * @return A future that indicates result of sent metadata request\n     */\n    private RequestFuture<ClientResponse> sendMetadataRequest(MetadataRequest.Builder request) {\n        final Node node = client.leastLoadedNode();\n        if (node == null)\n            return RequestFuture.noBrokersAvailable();\n        else\n            return client.send(node, request);\n    }\n\n}\n","sourceCodeStart":134,"sourceCodeEnd":168,"githubUrl":"https://github.com/apache/kafka/blob/c31c9215e131f8c17e79f8901b48c13ee6aa8e7a/clients/src/main/java/org/apache/kafka/clients/consumer/internals/TopicMetadataFetcher.java#L134-L168","documentation":"Thrown as TimeoutException by TopicMetadataFetcher.getTopicMetadata when the retry loop exhausts the supplied Timer (driven by the caller's request.timeout.ms / the method's deadline) before a successful, error-free MetadataResponse is received. Retriable errors keep the loop spinning with retryBackoff sleeps; once timer.notExpired() returns false, the call fails. This is purely a wall-clock exhaustion signal, not a protocol error.","triggerScenarios":"Calling consumer.partitionsFor(topic) or listTopics() when every broker is unreachable, when a broker is reachable but keeps returning retriable errors (LEADER_NOT_AVAILABLE during controller handover, NETWORK_EXCEPTION), or when retry.backoff.ms × number-of-retries exceeds request.timeout.ms. Also fires during cluster startup, partition reassignment, or network partition.","commonSituations":"Misconfigured bootstrap.servers (typo, wrong port, DNS points to a decommissioned broker); broker still starting up (controller not elected); firewall/security group dropping the connection; request.timeout.ms too low relative to retry.backoff.ms; client pointed at a wrong cluster/VPC; long GC pause on broker extending every fetch.","solutions":["Validate connectivity: telnet/bootstrap or nc -zv <broker-host> <port> from the client host; correct bootstrap.servers.","Raise request.timeout.ms (e.g. 60000) and lower retry.backoff.ms so the loop can complete more attempts within the deadline.","Check broker liveness via kafka-topics --bootstrap-server ... --list from the same host; if it also times out, the issue is network/cluster-side.","During cluster events (restart, reassignment), wait for controller election / ISRs to stabilize before starting consumers."],"exampleFix":"// before\nprops.put(ConsumerConfig.REQUEST_TIMEOUT_MS_CONFIG, \"10000\");\nprops.put(ConsumerConfig.RETRY_BACKOFF_MS_CONFIG, \"100\");\n\n// after\nprops.put(ConsumerConfig.REQUEST_TIMEOUT_MS_CONFIG, \"60000\");\nprops.put(ConsumerConfig.RETRY_BACKOFF_MS_CONFIG, \"500\");","handlingStrategy":"retry","validationCode":"// Verify broker reachability and metadata service before subscribing.\ntry (java.net.Socket s = new java.net.Socket()) {\n    s.connect(new java.net.InetSocketAddress(bootstrapHost, bootstrapPort), 2000);\n}\n// Tune timeouts so the deadline fits your network RTT:\nprops.put(ConsumerConfig.REQUEST_TIMEOUT_MS_CONFIG, 30000);\nprops.put(ConsumerConfig.RETRY_BACKOFF_MS_CONFIG, 500);","typeGuard":"import org.apache.kafka.common.errors.TimeoutException;\n\n/** True iff a timeout is specifically the 'fetching topic metadata' timeout (vs. other Kafka timeouts). */\nstatic boolean isMetadataFetchTimeout(Throwable t) {\n    return t instanceof TimeoutException\n        && t.getMessage() != null\n        && t.getMessage().contains(\"fetching topic metadata\");\n}","tryCatchPattern":"long deadline = System.currentTimeMillis() + 60_000;\nwhile (System.currentTimeMillis() < deadline) {\n    try {\n        return consumer.partitionsFor(topic);\n    } catch (org.apache.kafka.common.errors.TimeoutException e) {\n        if (!isMetadataFetchTimeout(e)) throw e;\n        // exponential backoff with jitter before retry\n        sleepBackoff();\n    }\n}\nthrow new org.apache.kafka.common.errors.TimeoutException(\"metadata fetch gave up after retries\");","preventionTips":["Confirm bootstrap servers are reachable and DNS resolves before starting the consumer.","Raise request.timeout.ms and retry.backoff.ms to match your network RTT and broker load.\n","Distinguish this transient timeout from fatal errors — retry with backoff, don't crash.","Monitor broker availability and controller health; a down controller often surfaces here."],"tags":["kafka","consumer","metadata","timeout","network","unreachable-broker"],"analyzedSha":"c31c9215e131f8c17e79f8901b48c13ee6aa8e7a","analyzedAt":"2026-08-03T12:34:05.770Z","schemaVersion":2}