{"record":{"id":"61deee0d77071adf","repo":"apache/druid","slug":"error-while-persisting","errorCode":null,"errorMessage":"Error while persisting","messagePattern":"Error while persisting","errorType":"exception","errorClass":"QueryException","httpStatus":null,"severity":"critical","filePath":"server/src/main/java/org/apache/druid/segment/realtime/appenderator/StreamAppenderator.java","lineNumber":300,"sourceCode":"  {\n    return schema.getDataSource();\n  }\n\n  @Override\n  public Object startJob()\n  {\n    lockBasePersistDirectory();\n    final Object retVal = bootstrapSinksFromDisk();\n    initializeExecutors();\n    resetNextFlush();\n    sinkSchemaAnnouncer.start();\n    return retVal;\n  }\n\n  private void throwPersistErrorIfExists()\n  {\n    if (persistError != null) {\n      throw new RE(persistError, \"Error while persisting\");\n    }\n  }\n\n  @Override\n  public AppenderatorAddResult add(\n      final SegmentIdWithShardSpec identifier,\n      final InputRow row,\n      @Nullable final Supplier<Committer> committerSupplier,\n      final boolean allowIncrementalPersists\n  ) throws SegmentNotWritableException\n  {\n    throwPersistErrorIfExists();\n\n    if (!identifier.getDataSource().equals(schema.getDataSource())) {\n      throw new IAE(\n          \"Expected dataSource[%s] but was asked to insert row for dataSource[%s]?!\",\n          schema.getDataSource(),\n          identifier.getDataSource()","sourceCodeStart":282,"sourceCodeEnd":318,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/server/src/main/java/org/apache/druid/segment/realtime/appenderator/StreamAppenderator.java#L282-L318","documentation":"StreamAppenderator caches the first persist failure in the persistError field; throwPersistErrorIfExists re-throws it as a RuntimeException whenever add/clear/persistAll is subsequently called. This prevents the ingester from continuing to write rows after a background persist has failed.","triggerScenarios":"A background persist of a Sink fails (e.g. disk I/O error writing intermediate hydrant segments) setting persistError; then any subsequent call to add(), clear(), or persistAll() rethrows it.","commonSituations":"Disk full or permission errors on druid.segmentCacheInfo / intermediate persist path; corrupt intermediate segments on the peon/task node causing persist thread failure during ingestion.","solutions":["Inspect the persistError cause in the stack trace to find the underlying persist failure (usually an IOException on disk).","Free disk space or fix permissions on the intermediate persist directory and restart/retry the task.","Clean corrupt intermediate segment files and re-run the ingestion task."],"exampleFix":"// before\n// task fails repeatedly: Error while persisting: No space left on device\n// after\n// increase disk space on the task node or raise druid.worker capacity, then resubmit the task\n// and monitor druid/served segments free disk space","handlingStrategy":"retry","validationCode":"// before submitting tasks, verify free disk space on the node\nif (new File(persistDir).getUsableSpace() < minRequiredBytes) { failFast(\"insufficient disk for persist\"); }","typeGuard":null,"tryCatchPattern":"try { appenderator.add(id, row, supplier, allowPersist) } catch (RuntimeException e) { if (isPersistError(e)) { alertAndFailTask(e); } else throw e; }","preventionTips":["Monitor free disk space on indexer nodes","Alert on Druid persist warnings in task logs","Use dedicated volumes for intermediate persist paths"],"tags":["druid","persistence","io","ingestion"],"backgroundTag":"database-write-failed","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}