{"record":{"id":"8962631473b2bb21","repo":"apache/druid","slug":"problem-reading-persisted-sinks-in-path-s","errorCode":null,"errorMessage":"Problem reading persisted sinks in path[%s]","messagePattern":"Problem reading persisted sinks in path\\[(.+?)\\]","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"server/src/main/java/org/apache/druid/segment/realtime/appenderator/BatchAppenderator.java","lineNumber":1050,"sourceCode":"      if (!identifierFile.isFile()) {\n        // No identifier in this sinkDir; it must not actually be a sink directory. Skip it.\n        continue;\n      }\n      retVal.add(sinkDir);\n    }\n\n    return retVal;\n  }\n\n  private Sink getSinkForIdentifierPath(SegmentIdWithShardSpec identifier, File identifierPath)\n      throws IOException\n  {\n    // To avoid reading and listing of \"merged\" dir and other special files\n    final File[] sinkFiles = identifierPath.listFiles(\n        (dir, fileName) -> !(Ints.tryParse(fileName) == null)\n    );\n    if (sinkFiles == null) {\n      throw new ISE(\"Problem reading persisted sinks in path[%s]\", identifierPath);\n    }\n\n    Arrays.sort(\n        sinkFiles,\n        (o1, o2) -> Ints.compare(Integer.parseInt(o1.getName()), Integer.parseInt(o2.getName()))\n    );\n\n    List<FireHydrant> hydrants = new ArrayList<>();\n    for (File hydrantDir : sinkFiles) {\n      final int hydrantNumber = Integer.parseInt(hydrantDir.getName());\n\n      log.debug(\"Loading previously persisted partial segment at [%s]\", hydrantDir);\n      if (hydrantNumber != hydrants.size()) {\n        throw new ISE(\"Missing hydrant [%,d] in identifier [%s].\", hydrants.size(), identifier);\n      }\n\n      hydrants.add(\n          new FireHydrant(","sourceCodeStart":1032,"sourceCodeEnd":1068,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/server/src/main/java/org/apache/druid/segment/realtime/appenderator/BatchAppenderator.java#L1032-L1068","documentation":"When a BatchAppenderator is created, it scans the identifier's persist directory for previously persisted sinks, filtering to numeric-named subdirectories (hydrant numbers). File.listFiles() returning null means the path is not a readable directory (or an I/O error occurred), so Druid throws this ISE instead of silently starting with no data.","triggerScenarios":"Constructing a BatchAppenderator whose segment identifier directory exists but cannot be listed — e.g. permission problems on the base persist dir, the path being a regular file instead of a directory, or underlying storage errors (NFS/object mount) during listFiles.","commonSituations":"Corrupted or hand-edited deep-storage/persist layout; restored data with wrong ownership/permissions; base-suffix persist path conflicts between tasks; mounting changes after restart.","solutions":["Check that the path in the message exists, is a directory, and is readable/writable by the Druid process user","Fix file ownership/permissions on the persist directory (druid.segment.cacheInfoDir / intermediate persist dirs)","Remove or repair the corrupted identifier directory if the data can be re-ingested","Check for storage-layer (NFS/HDFS/mount) errors in preceding logs"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"File dir = new File(identifierPath);\nif (!dir.isDirectory() || !dir.canRead()) {\n  throw new IllegalStateException(\"Persist dir not readable: \" + dir);\n}","typeGuard":"boolean isReadableDirectory(File f) {\n  return f != null && f.isDirectory() && f.canRead();\n}","tryCatchPattern":"try {\n  appenderator.startJob();\n} catch (ISE e) {\n  if (e.getMessage().contains(\"Problem reading persisted sinks\")) {\n    repairOrCleanPersistDir(path);\n  }\n  throw e;\n}","preventionTips":["Ensure the Druid process user owns the persist directories","Never replace a directory path with a file in persist layout","Watch for storage-mount (NFS/HDFS) errors in host logs"],"tags":["filesystem","persist","batch-ingestion"],"backgroundTag":"directory-not-found","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}