{"record":{"id":"ec160a6d1f5154e5","repo":"apache/druid","slug":"can-t-find-overwritingsegmentmeta-for-interval-s","errorCode":null,"errorMessage":"Can't find overwritingSegmentMeta for interval[%s]","messagePattern":"Can't find overwritingSegmentMeta for interval\\[(.+?)\\]","errorType":"exception","errorClass":"ISE","httpStatus":null,"severity":"error","filePath":"indexing-service/src/main/java/org/apache/druid/indexing/common/task/OverlordCoordinatingSegmentAllocator.java","lineNumber":123,"sourceCode":"  {\n    return internalAllocator.allocate(row, sequenceName, previousSegmentId, skipSegmentLineageCheck);\n  }\n\n  private static PartialShardSpec createPartialShardSpec(\n      AbstractTask.IngestionMode ingestionMode,\n      PartitionsSpec partitionsSpec,\n      TaskLockHelper taskLockHelper,\n      Interval interval\n  )\n  {\n    if (partitionsSpec.getType() == SecondaryPartitionType.LINEAR) {\n      if (taskLockHelper.isUseSegmentLock()) {\n        if (taskLockHelper.hasOverwritingRootGenerationPartition(interval) && (ingestionMode\n                                                                               != AbstractTask.IngestionMode.APPEND)) {\n          final OverwritingRootGenerationPartitions overwritingRootGenerationPartitions = taskLockHelper\n              .getOverwritingRootGenerationPartition(interval);\n          if (overwritingRootGenerationPartitions == null) {\n            throw new ISE(\"Can't find overwritingSegmentMeta for interval[%s]\", interval);\n          }\n          return new NumberedOverwritePartialShardSpec(\n              overwritingRootGenerationPartitions.getStartRootPartitionId(),\n              overwritingRootGenerationPartitions.getEndRootPartitionId(),\n              overwritingRootGenerationPartitions.getMinorVersionForNewSegments()\n          );\n        }\n      }\n      return NumberedPartialShardSpec.instance();\n    } else {\n      throw new ISE(\n          \"%s is not supported for partitionsSpec[%s]\",\n          OverlordCoordinatingSegmentAllocator.class.getName(),\n          partitionsSpec.getClass().getName()\n      );\n    }\n  }\n","sourceCodeStart":105,"sourceCodeEnd":141,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/indexing-service/src/main/java/org/apache/druid/indexing/common/task/OverlordCoordinatingSegmentAllocator.java#L105-L141","documentation":"When using segment locking during replace ingestion, OverlordCoordinatingSegmentAllocator needs the OverwritingRootGenerationPartitions metadata for a row's interval to build a NumberedOverwritePartialShardSpec. The taskLockHelper claims to have overwriting root-generation partitions for that interval, but the lookup returns null — an inconsistent state — so the allocator throws ISE.","triggerScenarios":"partialShardSpec is called for an interval where hasOverwritingRootGenerationPartition(interval) is true but getOverwritingRootGenerationPartition(interval) returns null, typically when the overwriting metadata was cleared/never registered (e.g. non-APPEND replace task racing with helper updates).","commonSituations":"Replace-type ingestion tasks with useSegmentLock enabled whose lock helper state wasn't populated for all intervals; concurrent tasks modifying the interval map; task restarted/resumed losing in-memory helper state.","solutions":["Verify the task type is a proper replace (not append) and that ingestion state registered overwriting partitions for every interval in the spec","Re-submit the task; if it recurs, disable segment locking (segment lock -> time chunk lock) so the overwrite metadata isn't required","Check taskLockHelper initialization/registration code for the failing interval and ensure IngestionMode isn't misconfigured"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"if (taskLockHelper.hasOverwritingRootGenerationPartition(interval)\n    && taskLockHelper.getOverwritingRootGenerationPartition(interval) == null) { throw new IllegalStateException(\"overwrite metadata missing for \" + interval); }","typeGuard":null,"tryCatchPattern":"try { shardSpec = allocator.partialShardSpec(row, seq, prevId, false); } catch (ISE e) { if (e.getMessage().contains(\"Can't find overwritingSegmentMeta\")) { switchToTimeChunkLockAndResubmit(); } else { throw e; } }","preventionTips":["Ensure replace tasks register overwriting partitions for every spec interval","Avoid concurrent tasks mutating the same interval while replacing","Fall back to time-chunk locking when segment-lock replace metadata is unstable"],"tags":["druid","segment-allocation","locking","replace"],"backgroundTag":"internal-invariant-violation","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}