apache/druid · error · ISE

Specified segments in the spec are different from the…

Error message

Specified segments in the spec are different from the current used segments. Possibly new segments would have been added or some segments have been unpublished.

What it means

CompactionTask's checkSegments compares the segments listed in the compaction spec's inputSpec against the segments currently published in metadata storage. If they differ (segments added or unpublished since the spec was written), this ISE is thrown to prevent compacting against stale input. The library throws it as a concurrency/staleness guard.

Solutions

1. Re-generate the compaction spec from current segment metadata and resubmit
2. Wait for ingestion to quiesce, then compact; or use segment-granularity intervals that are no longer being written
3. Kill or let retention rules remove obsolete segments first, then rebuild the spec
4. If using automatic compaction, let the coordinator recompute the latest segments instead of pinning inputSpec segments

Example fix

// before
"inputSpec": { "type": "segments", "segments": ["ds_2019-01-01T00:00:00.000Z_2019-01-02T00:00:00.000Z_2020-01-01T00:00:00.000Z"] } // stale
// after
"inputSpec": { "type": "interval", "interval": "2019-01-01/2019-01-02" } // resolved against current segments at run time

List<String> current = coordinator.getSegmentIds(ds, interval);
List<String> pinned = spec.getInputSpec().getSegments();
if (!new HashSet<>(current).equals(new HashSet<>(pinned))) {
  spec.refreshInputSpec(current); // rebuild before submitting
}

try {
  submitCompaction(spec);
} catch (IllegalStateException e) {
  if (e.getMessage().contains("Specified segments in the spec are different")) {
    spec = rebuildSpecFromCurrentMetadata(); // regenerate and retry once
    submitCompaction(spec);
  } else throw e;
}

When it happens

Trigger: Submitting a compaction spec with an explicit inputSpec listing segment IDs, but between spec creation and task execution new segments were published (real-time tasks) or some listed segments were killed/unpublished (replacement or retention rules).

Common situations: Compacting a datasource that is actively ingesting; a concurrent replace/kill task changed the segment set; re-submitting an old auto-compaction spec after the underlying data changed.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• All atomicUpdateGroup must be compacted together. Expected…
• Attempt to add row to swapped-out sink for segment
• Can't compact segments of non-consecutive rootPartition…
• Conflicting segment granularities found
• Could not assign role

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/7cd8deeac9e6dc70. Report an issue: GitHub.

Appendix: source

Thrown at indexing-service/src/main/java/org/apache/druid/indexing/common/task/CompactionTask.java:1405

    {
      return new ArrayList<>(
          actionClient.submit(
              new RetrieveUsedSegmentsAction(
                  dataSource,
                  ImmutableList.of(interval),
                  EnumSet.of(SegmentDetail.LOAD_SPEC)
              )
          )
      );
    }

    void checkSegments(LockGranularity lockGranularityInUse, List<DataSegment> latestSegments)
    {
      if (latestSegments.isEmpty()) {
        throw new ISE("No segments found for compaction. Please check that datasource name and interval are correct.");
      }
      if (!inputSpec.validateSegments(lockGranularityInUse, latestSegments)) {
        throw new ISE(
            "Specified segments in the spec are different from the current used segments. "
            + "Possibly new segments would have been added or some segments have been unpublished."
        );
      }
    }
  }

  public static class Builder
  {
    private final String dataSource;
    private final SegmentCacheManagerFactory segmentCacheManagerFactory;

    @Nullable
    private String id;
    private CompactionIOConfig ioConfig;
    @Nullable
    private DimensionsSpec dimensionsSpec;
    @Nullable

View on GitHub (pinned to 9b90983fd2)