apache/druid · error · IllegalStateException

Stream ingestion task unexpectedly attempted to overwrite se

Error message

Stream ingestion task unexpectedly attempted to overwrite segments: %s

What it means

SequenceMetadata.publishAnnotatedSegments asserts an invariant: stream ingestion tasks must not overwrite existing segments, so the internal mustBeNullOrEmptyOverwriteVersions/mustBeNullOrEmptyOverwriteSegments marker must always be empty (or null) at publish time. If it contains segments, a code path has violated the streaming contract that segment versions/publishing are strictly append-only, and the runner throws an IllegalStateException listing the offending segment identifiers.

Source

Thrown at indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/SequenceMetadata.java:360

        TaskToolbox toolbox,
        boolean useTransaction
    )
    {
      this.runner = runner;
      this.toolbox = toolbox;
      this.useTransaction = useTransaction;
    }

    @Override
    public SegmentPublishResult publishAnnotatedSegments(
        @Nullable Set<DataSegment> mustBeNullOrEmptyOverwriteSegments,
        Set<DataSegment> segmentsToPush,
        @Nullable Object commitMetadata,
        SegmentSchemaMapping segmentSchemaMapping
    ) throws IOException
    {
      if (mustBeNullOrEmptyOverwriteSegments != null && !mustBeNullOrEmptyOverwriteSegments.isEmpty()) {
        throw new ISE(
            "Stream ingestion task unexpectedly attempted to overwrite segments: %s",
            SegmentUtils.commaSeparatedIdentifiers(mustBeNullOrEmptyOverwriteSegments)
        );
      }
      final Map<?, ?> commitMetaMap = (Map<?, ?>) Preconditions.checkNotNull(commitMetadata, "commitMetadata");
      final SeekableStreamEndSequenceNumbers<PartitionIdType, SequenceOffsetType> finalPartitions =
          runner.deserializePartitionsFromMetadata(
              toolbox.getJsonMapper(),
              commitMetaMap.get(SeekableStreamIndexTaskRunner.METADATA_PUBLISH_PARTITIONS)
          );

      // Sanity check, we should only be publishing things that match our desired end state.
      if (!getEndOffsets().equals(finalPartitions.getPartitionSequenceNumberMap())) {
        throw new ISE(
            "Driver for sequence[%s] attempted to publish invalid metadata[%s].",
            SequenceMetadata.this.toString(),
            commitMetadata
        );

View on GitHub (pinned to 9b90983fd2)

Solutions

Inspect the listed segment identifiers and find which spec/tuning options caused overwrite segments; remove any overwrite-tombstone or non-append configuration from the streaming task.
Reset the supervisor and restart the affected tasks so fresh SequenceMetadata is built without stale overwrite markers.
Upgrade Druid if this reproduces on standard specs — it indicates a bug in the runner/extension publishing path.
Ensure no custom code/extensions mutate the segments set passed to the publish transaction.

Example fix

// before: streaming spec triggering overwrite paths
"tuningConfig": {"type": "kafka", "appendable": false, ...}
// after: keep streaming tasks append-only
"tuningConfig": {"type": "kafka", "appendable": true, ...}

Defensive patterns

Strategy: validation

Validate before calling

// streaming tasks must be append-only; check spec before submission
assert spec.getTuningConfig().isAppendable() : "streaming tasks cannot overwrite segments";

Try / catch

try { runTask(); } catch (ISE e) { if (e.getMessage().contains("unexpectedly attempted to overwrite segments")) { escalateAsDruidBug(e); } else { throw e; } }

Prevention

Keep streaming task specs appendable (no overwrite tombstones)
Avoid custom code that mutates the segment set before publish
Upgrade Druid if standard specs trigger this invariant violation

When it happens

Trigger: Calling publishAnnotatedSegments (from SeekableStreamIndexTaskRunner's publish path) while mustBeNullOrEmptyOverwriteSegments is non-empty — i.e. the task computed overwrite segments it should never produce: e.g. a streaming task configured with a non-null overwrite tombstone, corrupted metadata, or a bug/extension modifying the segment set before publishing.

Common situations: Streaming tasks whose specs or tuning produce overwrite versions due to misused segment-granularity/append-to-existing settings; data corruption in task metadata after crash/replay; custom extensions or code changes feeding extra segments into the publish transaction.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/5990851ba6241148. Report an issue: GitHub.