{"record":{"id":"3257465bcba29f74","repo":"apache/druid","slug":"offsetoutofrangeexception-with-message-s","errorCode":null,"errorMessage":"OffsetOutOfRangeException with message [%s]","messagePattern":"OffsetOutOfRangeException with message \\[(.+?)\\]","errorType":"console","errorClass":null,"httpStatus":null,"severity":"info","filePath":"extensions-core/kafka-indexing-service/src/main/java/org/apache/druid/indexing/kafka/KafkaIndexTaskRunner.java","lineNumber":95,"sourceCode":"  }\n\n  @Nonnull\n  @Override\n  protected List<OrderedPartitionableRecord<KafkaTopicPartition, Long, KafkaRecordEntity>> getRecords(\n      RecordSupplier<KafkaTopicPartition, Long, KafkaRecordEntity> recordSupplier,\n      TaskToolbox toolbox\n  ) throws Exception\n  {\n    try {\n      return recordSupplier.poll(task.getIOConfig().getPollTimeout());\n    }\n    catch (OffsetOutOfRangeException e) {\n      //\n      // Handles OffsetOutOfRangeException, which is thrown if the seeked-to\n      // offset is not present in the topic-partition. This can happen if we're asking a task to read from data\n      // that has not been written yet (which is totally legitimate). So let's wait for it to show up\n      //\n      log.warn(\"OffsetOutOfRangeException with message [%s]\", e.getMessage());\n      possiblyResetOffsetsOrWait(e.offsetOutOfRangePartitions(), recordSupplier, toolbox);\n      return Collections.emptyList();\n    }\n  }\n\n  @Override\n  protected SeekableStreamEndSequenceNumbers<KafkaTopicPartition, Long> deserializePartitionsFromMetadata(\n      ObjectMapper mapper,\n      Object object\n  )\n  {\n    return mapper.convertValue(object, mapper.getTypeFactory().constructParametrizedType(\n        SeekableStreamEndSequenceNumbers.class,\n        SeekableStreamEndSequenceNumbers.class,\n        KafkaTopicPartition.class,\n        Long.class\n    ));\n  }","sourceCodeStart":77,"sourceCodeEnd":113,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/extensions-core/kafka-indexing-service/src/main/java/org/apache/druid/indexing/kafka/KafkaIndexTaskRunner.java#L77-L113","documentation":"When the Kafka consumer's seeked-to offset does not exist in the topic-partition (OffsetOutOfRangeException), KafkaIndexTaskRunner.getRecords() logs a warning and waits for the data to appear or resets offsets (possiblyResetOffsetsOrWait), returning an empty record list for this poll. This is expected behavior, not a fatal error: the requested offset may simply not have been written yet.","triggerScenarios":"Calling getRecords() while the task's persisted sequence number for a partition is ahead of what the Kafka broker currently has (e.g. offsets expired or data not yet produced).","commonSituations":"Tasks restarted before the topic has caught up to stored offsets; retention deleting data at/after the stored offset; slow producers against newly spawned tasks reading future offsets.","solutions":["Let the task run: it waits for offsets to appear or resets them automatically per resetOffsetAutomatically.","Verify topic retention is not deleting data before tasks consume it.","Check producer throughput; ensure upstream is writing to the expected partitions.","If offsets are truly lost, manually update supervisor metadata store offsets and restart affected tasks."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// Before starting tasks, verify stored offsets are within the topic's current offset range\nConsumerRecords r = consumer.partitionsFor(topic); // then compare stored offset to beginning()/end() offsets","typeGuard":null,"tryCatchPattern":"try { records = runner.getRecords(); } catch (StreamException e) { /* supervisor retries; empty list is normal while waiting for offsets */ }","preventionTips":["Set retention well above maximum expected task lag","Let the built-in possiblyResetOffsetsOrWait/retry logic handle it","Monitor offsetsIdle time to detect persistent producer outages"],"tags":["kafka","offsets","streaming","retriable"],"backgroundTag":"index-out-of-range","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}