apache/druid · critical · IllegalStateException

Failed to insert upgrade segments in DB: %s

Error message

Failed to insert upgrade segments in DB: %s

What it means

During segment upgrade/insertion, IndexerSQLMetadataStorageCoordinator batch-inserts upgraded segment rows into the segments table. After the batch it checks for failed inserts and throws IllegalStateException listing the segment identifiers that could not be written. This almost always means one or more segments already exist in the database with conflicting content.

Source

Thrown at server/src/main/java/org/apache/druid/metadata/IndexerSQLMetadataStorageCoordinator.java:2228

        DataSegment segment = entry.getKey();
        ReplaceTaskLock lock = entry.getValue();
        batch.add()
             .bind("task_id", lock.getSupervisorTaskId())
             .bind("segment_id", segment.getId().toString())
             .bind("lock_version", lock.getVersion());
      }
      final int[] affectedAppendRows = batch.execute();

      final List<DataSegment> failedInserts = new ArrayList<>();
      for (int i = 0; i < partition.size(); ++i) {
        if (affectedAppendRows[i] != 1) {
          failedInserts.add(partition.get(i).getKey());
        } else {
          inserted++;
        }
      }
      if (!failedInserts.isEmpty()) {
        throw new ISE(
            "Failed to insert upgrade segments in DB: %s",
            SegmentUtils.commaSeparatedIdentifiers(failedInserts)
        );
      }
    }
    return inserted;
  }

  private List<DataSegmentPlus> retrieveSegmentsById(
      String dataSource,
      SegmentMetadataReadTransaction transaction,
      Set<String> segmentIds
  )
  {
    if (segmentIds.isEmpty()) {
      return Collections.emptyList();
    }

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Identify the failing segments from the comma-separated list in the message and check for existing rows in druid_segments with those IDs.
2. Delete or reconcile the stale/conflicting segment rows before retrying the upgrade.
3. Make the upgrade idempotent: skip segments whose existing row is already correct instead of re-inserting.
4. Check for overlapping ingestion tasks that may have created duplicate segment identifiers.

Example fix

// before
coordinator.insertSegments(upgradedSegments); // throws listing failures
// after
List<DataSegment> toInsert = upgradedSegments.stream()
    .filter(s -> !segmentsTableContains(s.getId()))
    .collect(Collectors.toList());
coordinator.insertSegments(toInsert);

List<String> existing = getExistingSegmentIds(candidateSegments);
List<DataSegment> insertable = candidateSegments.stream()
    .filter(s -> !existing.contains(s.getId().toString()))
    .collect(Collectors.toList());

try {
  coordinator.insertSegments(upgradedSegments);
} catch (IllegalStateException e) {
  log.error(e, "Segment upgrade inserts failed");
  // parse failed identifiers from message, reconcile DB rows, retry
}

When it happens

Trigger: Running metadata-store upgrade/repair tooling (e.g. kill/restore or segment schema upgrade flows) where some upgraded segments already have rows in druid_segments, causing INSERT failures collected in failedInserts.

Common situations: Re-running an upgrade job after a partial previous run; duplicate segment identifiers produced by re-ingestion; database unique-key violations on segment id due to conflicting payloads.

Related errors

• Error! More than one matching entry[%d] found for [%s]?!
• null/empty intervals
• announceHistoricalSegments failed with null metadata, should
• Segments to drop must all be part of the same datasource
• Couldn't generate AWS token.

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/d4f1e1ce061e7511. Report an issue: GitHub.