apache/druid · critical · QueryException

Error while persisting

Error message

Error while persisting

What it means

StreamAppenderator caches the first persist failure in the persistError field; throwPersistErrorIfExists re-throws it as a RuntimeException whenever add/clear/persistAll is subsequently called. This prevents the ingester from continuing to write rows after a background persist has failed.

Solutions

1. Inspect the persistError cause in the stack trace to find the underlying persist failure (usually an IOException on disk).
2. Free disk space or fix permissions on the intermediate persist directory and restart/retry the task.
3. Clean corrupt intermediate segment files and re-run the ingestion task.

Example fix

// before
// task fails repeatedly: Error while persisting: No space left on device
// after
// increase disk space on the task node or raise druid.worker capacity, then resubmit the task
// and monitor druid/served segments free disk space

// before submitting tasks, verify free disk space on the node
if (new File(persistDir).getUsableSpace() < minRequiredBytes) { failFast("insufficient disk for persist"); }

try { appenderator.add(id, row, supplier, allowPersist) } catch (RuntimeException e) { if (isPersistError(e)) { alertAndFailTask(e); } else throw e; }

When it happens

Trigger: A background persist of a Sink fails (e.g. disk I/O error writing intermediate hydrant segments) setting persistError; then any subsequent call to add(), clear(), or persistAll() rethrows it.

Common situations: Disk full or permission errors on druid.segmentCacheInfo / intermediate persist path; corrupt intermediate segments on the peon/task node causing persist thread failure during ingestion.

Related errors

• Attempt to add row to swapped-out sink for segment
• Bloom filter aggregators are query-time only
• Can't find pushedSegments for segment
• Can't find segmentsForSequence for sequence
• Cannot simultaneously replace and append to existing…

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/61deee0d77071adf. Report an issue: GitHub.

Appendix: source

Thrown at server/src/main/java/org/apache/druid/segment/realtime/appenderator/StreamAppenderator.java:300

  {
    return schema.getDataSource();
  }

  @Override
  public Object startJob()
  {
    lockBasePersistDirectory();
    final Object retVal = bootstrapSinksFromDisk();
    initializeExecutors();
    resetNextFlush();
    sinkSchemaAnnouncer.start();
    return retVal;
  }

  private void throwPersistErrorIfExists()
  {
    if (persistError != null) {
      throw new RE(persistError, "Error while persisting");
    }
  }

  @Override
  public AppenderatorAddResult add(
      final SegmentIdWithShardSpec identifier,
      final InputRow row,
      @Nullable final Supplier<Committer> committerSupplier,
      final boolean allowIncrementalPersists
  ) throws SegmentNotWritableException
  {
    throwPersistErrorIfExists();

    if (!identifier.getDataSource().equals(schema.getDataSource())) {
      throw new IAE(
          "Expected dataSource[%s] but was asked to insert row for dataSource[%s]?!",
          schema.getDataSource(),
          identifier.getDataSource()

View on GitHub (pinned to 9b90983fd2)