apache/druid · error · IllegalStateException

Automatic offset reset is disabled, but there are…

Error message

Automatic offset reset is disabled, but there are partitions with unavailable sequence numbers [%s].

What it means

Fatal validation in KinesisIndexTaskRunner.possiblyResetDataSourceMetadata: the stored starting sequence numbers for some partitions are no longer available in the stream (records aged out), but the task's tuning config has resetOffsetAutomatically disabled, so the task cannot recover and refuses to continue rather than silently skipping or duplicating data.

Solutions

  1. Enable resetOffsetAutomatically in the task/supervisor tuning config to let the task reset expired sequences.
  2. Manually reset offsets for the listed partitions using the supervisor reset API.
  3. Increase Kinesis stream retention so sequence numbers do not expire between restarts.
Defensive patterns

Strategy: validation

When it happens

Trigger: Thrown at extensions-core/kinesis-indexing-service/src/main/java/org/apache/druid/indexing/kinesis/KinesisIndexTaskRunner.java:153 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/aae977eacbd37904. Report an issue: GitHub.

Appendix: source

Thrown at extensions-core/kinesis-indexing-service/src/main/java/org/apache/druid/indexing/kinesis/KinesisIndexTaskRunner.java:153

        }
        if (task.getTuningConfig().isResetOffsetAutomatically()) {
          log.info(
              "Attempting to reset offsets for [%d] partitions with ids[%s].",
              partitionToSequenceResetMap.size(),
              partitionToSequenceResetMap.keySet()
          );
          try {
            sendResetRequestAndWait(partitionToSequenceResetMap, toolbox);
          }
          catch (IOException e) {
            throw new ISE(
                e,
                "Exception while attempting to automatically reset sequences for partitions[%s]",
                partitionToSequenceResetMap.keySet()
            );
          }
        } else {
          throw new ISE(
              "Automatic offset reset is disabled, but there are partitions with unavailable sequence numbers [%s].",
              partitionToSequenceResetMap
          );
        }
      }
    }
  }

  @Override
  protected boolean isEndOffsetExclusive()
  {
    return false;
  }

  @Override
  protected boolean isEndOfShard(String seqNum)
  {
    return KinesisSequenceNumber.END_OF_SHARD_MARKER.equals(seqNum);

View on GitHub (pinned to 9b90983fd2)