What causes "OffsetOutOfRangeException with message [%s]" in apache/druid?

Let the task run: it waits for offsets to appear or resets them automatically per resetOffsetAutomatically.

OffsetOutOfRangeException with message

Error message

OffsetOutOfRangeException with message [%s]

What it means

When the Kafka consumer's seeked-to offset does not exist in the topic-partition (OffsetOutOfRangeException), KafkaIndexTaskRunner.getRecords() logs a warning and waits for the data to appear or resets offsets (possiblyResetOffsetsOrWait), returning an empty record list for this poll. This is expected behavior, not a fatal error: the requested offset may simply not have been written yet.

Solutions

Let the task run: it waits for offsets to appear or resets them automatically per resetOffsetAutomatically.
Verify topic retention is not deleting data before tasks consume it.
Check producer throughput; ensure upstream is writing to the expected partitions.
If offsets are truly lost, manually update supervisor metadata store offsets and restart affected tasks.

Defensive patterns

Strategy: retry

Validate before calling

// Before starting tasks, verify stored offsets are within the topic's current offset range
ConsumerRecords r = consumer.partitionsFor(topic); // then compare stored offset to beginning()/end() offsets

Try / catch

try { records = runner.getRecords(); } catch (StreamException e) { /* supervisor retries; empty list is normal while waiting for offsets */ }

Prevention

Set retention well above maximum expected task lag
Let the built-in possiblyResetOffsetsOrWait/retry logic handle it
Monitor offsetsIdle time to detect persistent producer outages

When it happens

Trigger: Calling getRecords() while the task's persisted sequence number for a partition is ahead of what the Kafka broker currently has (e.g. offsets expired or data not yet produced).

Common situations: Tasks restarted before the topic has caught up to stored offsets; retention deleting data at/after the stored offset; slow producers against newly spawned tasks reading future offsets.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/3257465bcba29f74. Report an issue: GitHub.

Appendix: source

Thrown at extensions-core/kafka-indexing-service/src/main/java/org/apache/druid/indexing/kafka/KafkaIndexTaskRunner.java:95

  }

  @Nonnull
  @Override
  protected List<OrderedPartitionableRecord<KafkaTopicPartition, Long, KafkaRecordEntity>> getRecords(
      RecordSupplier<KafkaTopicPartition, Long, KafkaRecordEntity> recordSupplier,
      TaskToolbox toolbox
  ) throws Exception
  {
    try {
      return recordSupplier.poll(task.getIOConfig().getPollTimeout());
    }
    catch (OffsetOutOfRangeException e) {
      //
      // Handles OffsetOutOfRangeException, which is thrown if the seeked-to
      // offset is not present in the topic-partition. This can happen if we're asking a task to read from data
      // that has not been written yet (which is totally legitimate). So let's wait for it to show up
      //
      log.warn("OffsetOutOfRangeException with message [%s]", e.getMessage());
      possiblyResetOffsetsOrWait(e.offsetOutOfRangePartitions(), recordSupplier, toolbox);
      return Collections.emptyList();
    }
  }

  @Override
  protected SeekableStreamEndSequenceNumbers<KafkaTopicPartition, Long> deserializePartitionsFromMetadata(
      ObjectMapper mapper,
      Object object
  )
  {
    return mapper.convertValue(object, mapper.getTypeFactory().constructParametrizedType(
        SeekableStreamEndSequenceNumbers.class,
        SeekableStreamEndSequenceNumbers.class,
        KafkaTopicPartition.class,
        Long.class
    ));
  }

View on GitHub (pinned to 9b90983fd2)