apache/druid · error · IllegalStateException

Sink must not be null for identifier when persisting hydrant

Error message

Sink must not be null for identifier when persisting hydrant[%s]

What it means

When persisting a hydrant, BatchAppenderator looks up the SinkMetadata for the identifier to learn the hydrant count carried in sink metadata. A null entry means persist was requested for an identifier that this appenderator does not currently track as an active sink, so it cannot determine the persist directory numbering — this ISE is thrown.

Solutions

Only call persist for identifiers returned from getCurrentSinks()/actively added segments in this appenderator instance
Ensure persist calls complete before drop()/close() is invoked — serialize persist and cleanup operations
Restart the batch job if state is already inconsistent; sinksMetadata cannot be resurrected after drop
Verify no two BatchAppenderatorDriver instances share one appenderator

Defensive patterns

Strategy: validation

Validate before calling

if (appenderator.getCurrentSinks().stream()
    .noneMatch(sink -> sink.getId().equals(identifier))) {
  throw new IllegalStateException("Identifier not active: " + identifier);
}

Type guard

boolean isActiveSink(Appenderator a, SegmentIdWithShardSpec id) {
  return a.getCurrentSinks().stream().anyMatch(s -> s.getId().equals(id));
}

Try / catch

try {
  appenderator.persist(identifier);
} catch (ISE e) {
  if (e.getMessage().contains("Sink must not be null")) {
    log.warn("Persist raced with drop/cleanup for %s", identifier);
  }
  throw e;
}

Prevention

Serialize persist calls with drop()/close() — never overlap them
Only persist identifiers obtained from getCurrentSinks() in this appenderator instance
Do not share one Appenderator between multiple drivers

When it happens

Trigger: Calling appenderator.persist(identifier...) (or the flush path reaching it) for an identifier that was never added, or that was already dropped/cleared via drop/ clearing of sinksMetadata — e.g. persist racing with a drop, or persisting after the batch job's finishTransaction/cleanup.

Common situations: Custom callers driving Appenderator APIs directly and persisting after shutdown/drop; identifiers from a previous job incarnation; race between persist and cleanup in orchestration code.

Understand the failure class

Background: "Not found" and "does not exist" errors: why "Task not found", "No such folder", and "Can't find" fire when a lookup comes back empty — this error's family across 14 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/56674ffa8fa411ee. Report an issue: GitHub.

Appendix: source

Thrown at server/src/main/java/org/apache/druid/segment/realtime/appenderator/BatchAppenderator.java:1189

      throw new ISE(
          "Segment[%s] hydrant[%s] already swapped. This cannot happen.",
          identifier,
          indexToPersist
      );
    }

    log.debug("Segment[%s], persisting Hydrant[%s]", identifier, indexToPersist);

    try {
      final long startTime = System.nanoTime();
      int numRows = indexToPersist.getIndex().numRows();

      // since the sink may have been persisted before it may have lost its
      // hydrant count, we remember that value in the sinks' metadata, so we have
      // to pull it from there....
      SinkMetadata sm = sinksMetadata.get(identifier);
      if (sm == null) {
        throw new ISE("Sink must not be null for identifier when persisting hydrant[%s]", identifier);
      }
      final File persistDir = createPersistDirIfNeeded(identifier);
      indexMerger.persist(
          indexToPersist.getIndex(),
          identifier.getInterval(),
          new File(persistDir, String.valueOf(sm.getNumHydrants())),
          tuningConfig.getIndexSpecForIntermediatePersists(),
          tuningConfig.getSegmentWriteOutMediumFactory()
      );
      sm.setPersistedFileDir(persistDir);

      log.info(
          "Persisted in-memory data for segment[%s] spill[%s] to disk in [%,d] ms (%,d rows).",
          indexToPersist.getSegmentId(),
          sm.getNumHydrants(),
          (System.nanoTime() - startTime) / 1000000,
          numRows
      );

View on GitHub (pinned to 9b90983fd2)