apache/druid · error · IllegalStateException

unable to fetch sequence number for partition

Error message

unable to fetch sequence number for partition[%s] from stream

What it means

The supervisor asked the stream client for an offset for a partition (earliest or latest, depending on useEarliestSequenceNumber) and the client returned null — no offset could be fetched. It throws ISE because a sequence number is mandatory to position the reader for that partition.

Solutions

  1. Verify the stream and partition still exist (Kinesis console / kafka-topics --describe) and recreate or re-point the supervisor spec if the stream was deleted.
  2. Reset the supervisor via the reset API so metadata drops references to dead partitions.
  3. Check network connectivity and IAM permissions (kinesis:GetShardIterator / DescribeStream) if failures are transient.
  4. If partitions legitimately expired, enable partition-expiration handling (Kinesis) or reset so the supervisor stops tracking them.

Example fix

// before: supervisor references partitions of a deleted stream
"topic": "old-stream-name"
// after: update spec to the recreated stream and reset
"topic": "new-stream-name" + curl -X POST .../druid/indexer/v1/supervisor/<id>/reset
Defensive patterns

Strategy: validation

Validate before calling

// confirm all referenced partitions exist before starting
for (String p : savedPartitions) {
    if (getOffsetFromStreamForPartition(p, true) == null) resetSupervisor();
}

Type guard

if (offset == null) { log.warn("Partition {} missing from stream", partition); return null; }

Try / catch

try { fetchSequence(); } catch (ISE e) { if (e.getMessage().contains("unable to fetch sequence number")) { verifyStreamAndPartitions(); } }

Prevention

When it happens

Trigger: getSequenceNumberFromStreamForPartition calls getOffsetFromStreamForPartition(partition, useEarliestSequenceNumber) and receives null — the partition does not exist in the stream, the stream is unreachable/deleted, or the Kinesis shard has no readable records.

Common situations: Partition in saved metadata no longer exists in the stream (shard closed/expired); stream deleted or renamed while supervisor still references it; transient AWS/network failures making the describe/GetShardIterator call fail; IAM permissions preventing shard iteration.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/d47fb58556a65c87. Report an issue: GitHub.

Appendix: source

Thrown at indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/supervisor/SeekableStreamSupervisor.java:4660

              partition,
              startOffset
          );
          return makeSequenceNumber(startOffset, false);
        }
      }

      boolean useEarliestSequenceNumber = ioConfig.isUseEarliestSequenceNumber();
      if (subsequentlyDiscoveredPartitions.contains(partition)) {
        log.info(
            "Overriding useEarliestSequenceNumber and starting from beginning of newly discovered partition [%s] (which is probably from a split or merge)",
            partition
        );
        useEarliestSequenceNumber = true;
      }

      sequence = getOffsetFromStreamForPartition(partition, useEarliestSequenceNumber);
      if (sequence == null) {
        throw new ISE("unable to fetch sequence number for partition[%s] from stream", partition);
      }
      log.debug("Getting sequence number [%s] for partition [%s]", sequence, partition);
      return makeSequenceNumber(sequence, false);
    }
  }

  public Map<PartitionIdType, SequenceOffsetType> getOffsetsFromMetadataStorage()
  {
    final DataSourceMetadata dataSourceMetadata = retrieveDataSourceMetadata();
    if (dataSourceMetadata instanceof SeekableStreamDataSourceMetadata
        && checkSourceMetadataMatch(dataSourceMetadata)) {
      @SuppressWarnings("unchecked")
      SeekableStreamSequenceNumbers<PartitionIdType, SequenceOffsetType> partitions = ((SeekableStreamDataSourceMetadata) dataSourceMetadata)
          .getSeekableStreamSequenceNumbers();
      if (partitions != null) {
        if (!ioConfig.getStream().equals(partitions.getStream())) {
          log.warn(
              "Topic/stream in metadata storage [%s] doesn't match spec topic/stream [%s], ignoring stored sequences",

View on GitHub (pinned to 9b90983fd2)