apache/druid · error · SegmentNotWritableException

Attempt to add row to swapped-out sink for segment[%s].

Error message

Attempt to add row to swapped-out sink for segment[%s].

What it means

StreamAppenderator.add throws SegmentNotWritableException when the target sink has been swapped out (persisted and replaced by an immutable hydrant), making it impossible to add more rows in memory to that segment. sinkRowsInMemoryAfterAdd < 0 is the sentinel indicating the sink's writable portion is gone.

Source

Thrown at server/src/main/java/org/apache/druid/segment/realtime/appenderator/StreamAppenderator.java:338

    }

    final Sink sink = getOrCreateSink(identifier);
    metrics.reportMessageMaxTimestamp(row.getTimestampFromEpoch());
    final int sinkRowsInMemoryBeforeAdd = sink.getNumRowsInMemory();
    final int sinkRowsInMemoryAfterAdd;
    final long bytesInMemoryBeforeAdd = sink.getBytesInMemory();
    final long bytesInMemoryAfterAdd;
    final IncrementalIndexAddResult addResult;

    addResult = sink.add(row);
    sinkRowsInMemoryAfterAdd = addResult.getRowCount();
    bytesInMemoryAfterAdd = addResult.getBytesInMemory();

    final long currTs = System.currentTimeMillis();
    metrics.reportMessageGap(currTs - row.getTimestampFromEpoch());

    if (sinkRowsInMemoryAfterAdd < 0) {
      throw new SegmentNotWritableException("Attempt to add row to swapped-out sink for segment[%s].", identifier);
    }

    if (addResult.isRowAdded()) {
      rowIngestionMeters.incrementProcessed();
    } else if (addResult.hasParseException()) {
      parseExceptionHandler.handle(addResult.getParseException());
    }

    final int numAddedRows = sinkRowsInMemoryAfterAdd - sinkRowsInMemoryBeforeAdd;
    rowsCurrentlyInMemory.addAndGet(numAddedRows);
    bytesCurrentlyInMemory.addAndGet(bytesInMemoryAfterAdd - bytesInMemoryBeforeAdd);
    totalRows.addAndGet(numAddedRows);

    boolean isPersistRequired = false;
    boolean persist = false;
    List<String> persistReasons = new ArrayList<>();

    if (!sink.canAppendRow()) {

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Re-route the row to a newly allocated segment identifier (allocate a new pending segment for the event timestamp).
2. Check for clock/timestamp skew causing events to target old, already-persisted segments.
3. Upgrade/verify indexer versions — races between persist and add were fixed in later releases.

Example fix

// before
appenderator.add(oldIdentifier, row, supplier, false); // sink already swapped

// after
SegmentIdWithShardSpec id = allocateNewPendingSegment(row.getTimestamp());
appenderator.add(id, row, supplier, false);

// skip add when the sink was handed off
if (appenderator.getSinks().get(identifier) == null || !isSinkWritable(identifier)) { allocateNewSegmentAndRetry(row); return; }

try { appenderator.add(id, row, supplier, true) } catch (SegmentNotWritableException e) { SegmentIdWithShardSpec fresh = allocator.newSegment(row.getTimestamp()); appenderator.add(fresh, row, supplier, true); }

When it happens

Trigger: Adding a row to a segment whose sink was already swapped out mid-persist — a race between background persist/metadata publish and row arrival, or adding to a segment identifier after its sink was handed off.

Common situations: Kafka/Kinesis tasks during segment handoff where late-arriving records target a segment that was just persisted; replaying old offsets after a publish completed.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• Failed to get locks for intervals[%s]
• Column capacity exceeded
• Column capacity exceeded
• Column capacity exceeded
• ColumnCapacityExceededException

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/2461737de242553c. Report an issue: GitHub.