{"record":{"id":"daa5f46791f60d35","repo":"apache/druid","slug":"can-t-find-pushedsegments-for-segment-s","errorCode":null,"errorMessage":"Can't find pushedSegments for segment[%s]","messagePattern":"Can't find pushedSegments for segment\\[(.+?)\\]","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"server/src/main/java/org/apache/druid/segment/realtime/appenderator/BatchAppenderatorDriver.java","lineNumber":176,"sourceCode":"          \"Pushed segments[%s] are different from the requested ones[%s]\",\n          pushedSegmentIdToSegmentMap.keySet(),\n          requestedSegmentIdsForSequences\n      );\n    }\n\n    synchronized (segments) {\n      for (String sequenceName : sequenceNames) {\n        final SegmentsForSequence segmentsForSequence = segments.get(sequenceName);\n        if (segmentsForSequence == null) {\n          throw new ISE(\"Can't find segmentsForSequence for sequence[%s]\", sequenceName);\n        }\n\n        segmentsForSequence.getAllSegmentsOfInterval().forEach(segmentsOfInterval -> {\n          final SegmentWithState appendingSegment = segmentsOfInterval.getAppendingSegment();\n          if (appendingSegment != null) {\n            final DataSegment pushedSegment = pushedSegmentIdToSegmentMap.get(appendingSegment.getSegmentIdentifier());\n            if (pushedSegment == null) {\n              throw new ISE(\"Can't find pushedSegments for segment[%s]\", appendingSegment.getSegmentIdentifier());\n            }\n\n            segmentsOfInterval.finishAppendingToCurrentActiveSegment(\n                segmentWithState -> segmentWithState.pushAndDrop(pushedSegment)\n            );\n          }\n        });\n      }\n    }\n\n    return segmentsAndCommitMetadata;\n  }\n\n  /**\n   * Publish all segments.\n   *\n   * @param segmentsToBeOverwritten segments which can be overwritten by new segments published by the given publisher\n   * @param publisher               segment publisher","sourceCodeStart":158,"sourceCodeEnd":194,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/server/src/main/java/org/apache/druid/segment/realtime/appenderator/BatchAppenderatorDriver.java#L158-L194","documentation":"Thrown by BatchAppenderatorDriver.pushAndClear when a segment that is still in the 'appending' state for a sequence has no corresponding entry in the map of pushed segments. Every active appending segment must have been pushed before the driver finalizes it; a miss means the push did not cover all active segments.","triggerScenarios":"During pushAndClear, an appending SegmentWithState exists whose SegmentIdWithShardSpec is absent from pushedSegmentIdToSegmentMap — i.e. the Appenderator pushed fewer segments than there are active appending segments.","commonSituations":"Partial push failures where some segments were dropped before push; mismatch between the segment identifiers registered in the driver and those produced by the Appenderator; shard spec generation changes mid-task; custom Appenderator bugs.","solutions":["Compare the logged pushed segment set against the appending segment identifiers to find which segment was never pushed","Check deep-storage push logs for failures on the missing segment and address the storage error","Retry the batch task; the transactional model reruns pushes cleanly","If reproducible, inspect the Appenderator/segment allocation logic for identifier mismatches (e.g. changed shard specs)"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// confirm all appending segments have been pushed before finalizing\nSet<SegmentIdWithShardSpec> pending = driver.getActiveAppendingSegments();\nSet<SegmentIdWithShardSpec> pushed = lastPushResult.keySet();\nif (!pushed.containsAll(pending)) {\n  throw new IllegalStateException(\"Unpushed appending segments: \" + Sets.difference(pending, pushed));\n}","typeGuard":null,"tryCatchPattern":"try {\n  driver.pushAllAndClear(publisher, commitFn);\n} catch (ISE e) {\n  if (e.getMessage().startsWith(\"Can't find pushedSegments for segment\")) {\n    // a push failed silently; abort and retry the whole batch\n    throw new TaskAbortedException(e);\n  }\n  throw e;\n}","preventionTips":["Check every Appenderator.push return value for null/missing segments and fail fast at push time","Verify deep-storage availability before the push phase","Avoid dropping segments before pushAndClear completes","Watch for shard-spec generation changes that alter segment identifiers mid-task"],"tags":["java","druid","ingestion","internal-state"],"backgroundTag":"entity-not-found","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}