apache/druid · error · IllegalStateException

got null sequence number for partition

Error message

got null sequence number for partition[%s] when fetching from kafka!

What it means

KafkaIndexTaskRunner.possiblyResetOffsetsOrWait() calls recordSupplier.getEarliestSequenceNumber() to verify a partition still exists; a null return means Kafka reported no offsets for the partition, so the runner throws ISE rather than proceeding with invalid offsets.

Solutions

Verify the topic and its partitions still exist (kafka-topics --describe)
Recreate the topic or restore the deleted partition
Reset the task/datasource offsets to valid values (reset supervisor offsets)
Check broker retention and min.insync settings

Example fix

// before
topic deleted while supervisor running
// after
kafka-topics.sh --create --topic mytopic ... ; then kafka-indexing-service reset supervisor offsets

Defensive patterns

Strategy: retry

Validate before calling

// before/at task config time, confirm partitions exist
try (AdminClient admin = AdminClient.create(consumerProps)) {
  Set<String> topics = admin.listTopics().names().get(30, TimeUnit.SECONDS);
  if (!topics.containsAll(configuredTopics)) throw new IllegalStateException("topic missing");
}

Try / catch

try {
  records = taskRunner.getRecords(partition, ...);
} catch (IllegalStateException e) {
  if (e.getMessage().contains("got null sequence number")) {
    // topic/partition deleted: recreate topic or reset offsets, then resume
  } else { throw e; }
}

Prevention

Disable topic auto-deletion in environments running Druid tasks
Monitor for topic deletion (Kafka alerts)
Use offset reset policies and supervisor reset APIs
Verify retention doesn't fully expire partitions

When it happens

Trigger: During getRecords, when the task (re)reads offsets and getEarliestSequenceNumber returns null — typically the topic/partition was deleted, or the broker has no data for the partition.

Common situations: Topic deleted or recreated while a task is running, partition removal, retention deleting all records, misconfigured multi-topic patterns.

Understand the failure class

Background: Record Not Found Errors: "not found", RecordNotFound, and "was not found" — what they mean and how to fix them — this error's family across 28 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/bacf805aa000e0e0. Report an issue: GitHub.

Appendix: source

Thrown at extensions-core/kafka-indexing-service/src/main/java/org/apache/druid/indexing/kafka/KafkaIndexTaskRunner.java:136

      TaskToolbox taskToolbox
  ) throws InterruptedException, IOException
  {
    final String stream = task.getIOConfig().getStartSequenceNumbers().getStream();
    final boolean isMultiTopic = task.getIOConfig().isMultiTopic();
    final Map<TopicPartition, Long> resetPartitions = new HashMap<>();
    boolean doReset = false;
    if (task.getTuningConfig().isResetOffsetAutomatically()) {
      for (Map.Entry<TopicPartition, Long> outOfRangePartition : outOfRangePartitions.entrySet()) {
        final TopicPartition topicPartition = outOfRangePartition.getKey();
        final long nextOffset = outOfRangePartition.getValue();
        // seek to the beginning to get the least available offset
        StreamPartition<KafkaTopicPartition> streamPartition = StreamPartition.of(
            stream,
            new KafkaTopicPartition(isMultiTopic, topicPartition.topic(), topicPartition.partition())
        );
        final Long leastAvailableOffset = recordSupplier.getEarliestSequenceNumber(streamPartition);
        if (leastAvailableOffset == null) {
          throw new ISE(
              "got null sequence number for partition[%s] when fetching from kafka!",
              topicPartition.partition()
          );
        }
        // reset the seek
        recordSupplier.seek(streamPartition, nextOffset);
        // Reset consumer offset if resetOffsetAutomatically is set to true
        // and the current message offset in the kafka partition is more than the
        // next message offset that we are trying to fetch
        if (leastAvailableOffset > nextOffset) {
          doReset = true;
          resetPartitions.put(topicPartition, nextOffset);
        }
      }
    }

    if (doReset) {
      sendResetRequestAndWait(CollectionUtils.mapKeys(resetPartitions, topicPartition -> StreamPartition.of(

View on GitHub (pinned to 9b90983fd2)