{"id":"68893e0bc69c8ef3","repo":"apache/kafka","slug":"fetch-position-is-out-of-range-for-partition","errorCode":null,"errorMessage":"Fetch position {} is out of range for partition {}","messagePattern":"Fetch position (.+?) is out of range for partition (.+?)","errorType":"exception","errorClass":"OffsetOutOfRangeException","httpStatus":null,"severity":"error","filePath":"clients/src/main/java/org/apache/kafka/clients/consumer/internals/FetchCollector.java","lineNumber":359,"sourceCode":"        } else if (error == Errors.OFFSET_OUT_OF_RANGE) {\n            Optional<Integer> clearedReplicaId = subscriptions.clearPreferredReadReplica(tp);\n\n            if (clearedReplicaId.isEmpty()) {\n                // If there's no preferred replica to clear, we're fetching from the leader so handle this error normally\n                SubscriptionState.FetchPosition position = subscriptions.positionOrNull(tp);\n\n                if (position == null || fetchOffset != position.offset) {\n                    log.debug(\"Discarding stale fetch response for partition {} since the fetched offset {} \" +\n                            \"does not match the current offset {} or the partition has been unassigned\", tp, fetchOffset, position);\n                } else {\n                    String errorMessage = \"Fetch position \" + position + \" is out of range for partition \" + tp;\n\n                    if (subscriptions.hasDefaultOffsetResetPolicy()) {\n                        log.info(\"{}, resetting offset\", errorMessage);\n                        subscriptions.requestOffsetResetIfPartitionAssigned(tp);\n                    } else {\n                        log.info(\"{}, raising error to the application since no reset policy is configured\", errorMessage);\n                        throw new OffsetOutOfRangeException(errorMessage,\n                                Collections.singletonMap(tp, position.offset));\n                    }\n                }\n            } else {\n                log.debug(\"Unset the preferred read replica {} for partition {} since we got {} when fetching {}\",\n                        clearedReplicaId.get(), tp, error, fetchOffset);\n            }\n        } else if (error == Errors.TOPIC_AUTHORIZATION_FAILED) {\n            //we log the actual partition and not just the topic to help with ACL propagation issues in large clusters\n            log.warn(\"Not authorized to read from partition {}.\", tp);\n            throw new TopicAuthorizationException(Collections.singleton(tp.topic()));\n        } else if (error == Errors.UNKNOWN_LEADER_EPOCH) {\n            log.debug(\"Received unknown leader epoch error in fetch for partition {}\", tp);\n        } else if (error == Errors.UNKNOWN_SERVER_ERROR) {\n            log.warn(\"Unknown server error while fetching offset {} for topic-partition {}\",\n                    fetchOffset, tp);\n        } else if (error == Errors.CORRUPT_MESSAGE) {\n            throw new KafkaException(\"Encountered corrupt message when fetching offset \"","sourceCodeStart":341,"sourceCodeEnd":377,"githubUrl":"https://github.com/apache/kafka/blob/c31c9215e131f8c17e79f8901b48c13ee6aa8e7a/clients/src/main/java/org/apache/kafka/clients/consumer/internals/FetchCollector.java#L341-L377","documentation":"OffsetOutOfRangeException thrown at FetchCollector.java:359 when the broker returns OFFSET_OUT_OF_RANGE for the leader fetch (no preferred replica to clear) and the consumer has no default offset-reset policy (auto.offset.reset=none). The fetch position is older than the log start offset or ahead of the high watermark, and the client cannot recover automatically, so it surfaces the error to the application.","triggerScenarios":"Triggered in handleInitializeErrors when error == Errors.OFFSET_OUT_OF_RANGE, preferredReadReplica is empty, the fetch offset matches the current position, and subscriptions.hasDefaultOffsetResetPolicy() is false. Common after the retained data older than the committed offset has been log-compacted or deleted, or the consumer committed an offset past the partition end.","commonSituations":"auto.offset.reset=none with committed offsets outside the current log range; retention deleted segments below the committed offset; consumer group reused against a topic whose data was cleared; manual seek() to a position beyond the log end; compacted topic where the committed offset points to a gap.","solutions":["Set auto.offset.reset to earliest or latest so the consumer can auto-reset when the position is out of range.","Seek the affected partition to a valid offset: earliest(), latest(), or seekToBeginning/seekToEnd before the next poll.","Increase retention (log.retention.hours / retention.bytes) on the broker so committed offsets remain valid.","Reset the consumer group offsets via kafka-consumer-groups.sh --reset-offsets after data loss/recreation."],"exampleFix":"// before\nprops.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, \"none\");\n\n// after\nprops.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, \"earliest\");\n// or, in the rebalance listener, reset explicitly:\nconsumer.seekToBeginning(Collections.singleton(tp));","handlingStrategy":"try-catch","validationCode":"// Configure an auto offset reset policy so the client handles out-of-range internally\n// instead of throwing OffsetOutOfRangeException at FetchCollector.java:359.\nProperties p = new Properties();\np.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, \"earliest\"); // or \"latest\", \"none\"\n// If you set \"none\" intentionally (to detect data loss), validate before poll:\nMap<TopicPartition, Long> endOffsets = consumer.endOffsets(assigned);\nMap<TopicPartition, Long> beginning = consumer.beginningOffsets(assigned);\nfor (TopicPartition tp : assigned) {\n    long pos = consumer.position(tp);\n    if (pos < beginning.get(tp) || pos > endOffsets.get(tp)) {\n        // position is off-log; seek to a safe offset BEFORE poll()\n        consumer.seek(tp, OffsetResetStrategy.EARLIEST.equals(policy) ? beginning.get(tp) : endOffsets.get(tp));\n    }\n}","typeGuard":null,"tryCatchPattern":"try {\n    records = consumer.poll(Duration.ofMillis(500));\n} catch (org.apache.kafka.common.errors.OffsetOutOfRangeException e) {\n    // FetchCollector.java:359: fetch offset off the log AND auto.offset.reset=none\n    // (or no default reset policy). Recover explicitly.\n    Map<TopicPartition, Long> outOfRange = e.offsetOutOfRangePartitions();\n    Map<TopicPartition, Long> beginnings = consumer.beginningOffsets(outOfRange.keySet());\n    for (Map.Entry<TopicPartition, Long> entry : outOfRange.entrySet()) {\n        TopicPartition tp = entry.getKey();\n        long safe = Math.max(beginnings.get(tp), entry.getValue()); // or seek to end\n        log.warn(\"Offset {} out of range for {}; seeking to {}\", entry.getValue(), tp, safe);\n        consumer.seek(tp, safe);\n    }\n}","preventionTips":["Set auto.offset.reset explicitly (earliest/latest) unless you deliberately want failures surfaced — 'none' or unset throws this.","If you commit offsets to external storage, validate them against beginningOffsets/endOffsets after a consumer restart or a long pause.","Resetting consumer group offsets (or log retention deleting them) while a consumer holds a stale committed offset is the most common trigger; coordinate retention vs. consumer liveness.","After catching, seek to a known-safe offset and continue; do not retry the same poll blindly — it will throw again."],"tags":["consumer","offset","configuration","retention"],"analyzedSha":"c31c9215e131f8c17e79f8901b48c13ee6aa8e7a","analyzedAt":"2026-08-03T12:34:05.770Z","schemaVersion":2}