prestodb/presto · error · PrestoException

GENERIC_INTERNAL_ERROR

GENERIC_INTERNAL_ERROR

Error message

Leader election in progress for Kafka topic '%s' partition %s

What it means

While generating splits, KafkaSplitManager inspects each partition's leader via consumer.partitionsFor; if PartitionInfo.leader() is null, a leader election is in progress and the connector throws GENERIC_INTERNAL_ERROR. Presto needs the leader's host/port to build a split pinned to that broker, and cannot proceed until a leader exists.

Source

Thrown at presto-kafka/src/main/java/com/facebook/presto/kafka/KafkaSplitManager.java:101

            ConnectorTransactionHandle transaction,
            ConnectorSession session,
            ConnectorTableLayoutHandle layout,
            SplitSchedulingContext splitSchedulingContext)
    {
        KafkaTableHandle kafkaTableHandle = convertLayout(layout).getTable();
        try {
            String topic = kafkaTableHandle.getTopicName();
            KafkaTableLayoutHandle layoutHandle = (KafkaTableLayoutHandle) layout;
            HostAddress node = KafkaClusterMetadataHelper.selectRandom(clusterMetadataSupplier.getNodes(layoutHandle.getTable().getSchemaName()));

            KafkaConsumer<ByteBuffer, ByteBuffer> consumer = consumerManager.createConsumer(Thread.currentThread().getName(), node);
            List<PartitionInfo> partitions = consumer.partitionsFor(topic);
            ImmutableList.Builder<ConnectorSplit> splits = ImmutableList.builder();

            for (PartitionInfo partition : partitions) {
                Node leader = partition.leader();
                if (leader == null) {
                    throw new PrestoException(GENERIC_INTERNAL_ERROR, format("Leader election in progress for Kafka topic '%s' partition %s", topic, partition.partition()));
                }

                HostAddress partitionLeader = HostAddress.fromParts(leader.host(), leader.port());
                long startTimestamp = layoutHandle.getStartOffsetTimestamp();
                long endTimestamp = layoutHandle.getEndOffsetTimestamp();

                if (startTimestamp > endTimestamp) {
                    throw new IllegalArgumentException(String.format("Invalid Kafka Offset start/end pair: %s - %s", startTimestamp, endTimestamp));
                }

                TopicPartition topicPartition = new TopicPartition(partition.topic(), partition.partition());
                consumer.assign(ImmutableList.of(topicPartition));

                long beginningOffset = (startTimestamp == 0) ?
                        consumer.beginningOffsets(ImmutableList.of(topicPartition)).values().iterator().next() :
                        findOffsetsByTimestamp(consumer, topicPartition, startTimestamp);
                long endOffset = (endTimestamp == 0) ?
                        consumer.endOffsets(ImmutableList.of(topicPartition)).values().iterator().next() :

View on GitHub (pinned to 55bb57d202)

Solutions

1. Wait for leader election to finish and re-run the query (usually seconds)
2. Check partition health: kafka-topics.sh --describe --topic <t> and confirm every partition has a Leader
3. Restore the down broker that hosts the leader replica
4. Increase replication factor so a single broker outage does not orphan partitions

// preflight: every partition has a leader
kafka-topics.sh --bootstrap-server broker:9092 --describe --topic <topic> \
  | awk '$0 !~ /Leader: [^n]/ {print "no leader:", $0}'

null

try { runQuery(); } catch (PrestoException e) {
  if (e.getErrorCode() == GENERIC_INTERNAL_ERROR.toErrorCode() &&
      e.getMessage().contains("Leader election in progress")) {
    Thread.sleep(5_000); retryQuery();
  }
}

When it happens

Trigger: getSplits calls consumer.partitionsFor(topic) and iterates partitions; a partition whose replicas are all down or that is mid-leader-election returns leader()==null (e.g. right after broker restart or under-replicated topic).

Common situations: Querying a topic immediately after a broker crash/failover; topic created with replication factor 1 and that broker is down; topic auto-creation race where partitions have no leader yet.

Related errors

• KAFKA_SPLIT_ERROR
• unexpected internal column '%s'
• KAFKA_SCHEMA_ERROR
• Invalid Kafka Offset start/end pair: %s - %s
• KAFKA_SPLIT_ERROR

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/8522d2f85dc37b06. Report an issue: GitHub.