{"record":{"id":"3f8a5ea2d514640b","repo":"apache/druid","slug":"cannot-delete-all-segment-files-since-druid-storag","errorCode":null,"errorMessage":"Cannot delete all segment files since druid.storage.storageDirectory is not set.","messagePattern":"Cannot delete all segment files since druid\\.storage\\.storageDirectory is not set\\.","errorType":"exception","errorClass":"ISE","httpStatus":null,"severity":"error","filePath":"extensions-core/hdfs-storage/src/main/java/org/apache/druid/storage/hdfs/HdfsDataSegmentKiller.java","lineNumber":133,"sourceCode":"        }\n\n        // descriptor.json is a file to store segment metadata in deep storage. This file is deprecated and not stored\n        // anymore, but we still delete them if exists.\n        fs.delete(descriptorPath, false);\n\n        removeEmptyParentDirectories(fs, segmentPath, segmentParts.length > 1 ? 2 : 3);\n      }\n    }\n    catch (IOException e) {\n      throw new SegmentLoadingException(e, \"Unable to kill segment\");\n    }\n  }\n\n  @Override\n  public void killAll() throws IOException\n  {\n    if (storageDirectory == null) {\n      throw new ISE(\"Cannot delete all segment files since druid.storage.storageDirectory is not set.\");\n    }\n\n    log.info(\"Deleting all segment files from hdfs dir [%s].\", storageDirectory.toUri().toString());\n    final FileSystem fs = storageDirectory.getFileSystem(config);\n    fs.delete(storageDirectory, true);\n  }\n\n  private void removeEmptyParentDirectories(final FileSystem fs, final Path segmentPath, final int depth)\n  {\n    Path path = segmentPath;\n    try {\n      for (int i = 1; i <= depth; i++) {\n        path = path.getParent();\n        if (fs.listStatus(path).length != 0 || !fs.delete(path, false)) {\n          break;\n        }\n      }\n    }","sourceCodeStart":115,"sourceCodeEnd":151,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/extensions-core/hdfs-storage/src/main/java/org/apache/druid/storage/hdfs/HdfsDataSegmentKiller.java#L115-L151","documentation":"HdfsDataSegmentKiller.killAll() refuses to run when the HdfsDataSegmentPusherConfig has no druid.storage.storageDirectory configured (the field is null because the constructor saw an empty/null value). Since killAll would otherwise recursively delete the entire deep-storage segment root, Druid fails fast with IllegalStateException rather than deleting an unknown/incorrect location. This is a missing-required-configuration guard, thrown before any HDFS call is made.","triggerScenarios":"Calling killAll() (e.g. via the coordinator's full datasource deletion flow) on a cluster where druid.storage.storageDirectory is absent or empty in HdfsDataSegmentPusherConfig.","commonSituations":"Operator removed or never set druid.storage.storageDirectory in the runtime properties while druid.storage.type=hdfs; killAll invoked from a service whose config injection uses a default HdfsDataSegmentPusherConfig; migrating from local storage to HDFS deep storage without updating the pusher config.","solutions":["Set druid.storage.storageDirectory to the HDFS base directory (e.g. druid.storage.storageDirectory=/druid/segments) in the runtime.properties of the services that kill segments, then restart.","Confirm the property key spelling and that it is non-empty (Strings.isNullOrEmpty check) on the node executing killAll.","If you do not intend to ever use killAll on this cluster, avoid invoking it and delete individual segments instead.","Verify the injected HdfsDataSegmentPusherConfig actually loads from the intended config source (no shadowing default config)."],"exampleFix":"// before (runtime.properties)\n# druid.storage.storageDirectory=   (missing/empty)\n// after\ndruid.storage.type=hdfs\ndruid.storage.storageDirectory=/druid/segments","handlingStrategy":"validation","validationCode":"String dir = props.getProperty(\"druid.storage.storageDirectory\");\nif (dir == null || dir.isEmpty()) {\n  throw new IllegalStateException(\"Set druid.storage.storageDirectory before calling killAll\");\n}","typeGuard":null,"tryCatchPattern":"try { killer.killAll(); }\ncatch (ISE e) { log.error(\"Configure druid.storage.storageDirectory to use killAll\"); }","preventionTips":["Always set druid.storage.storageDirectory whenever druid.storage.type=hdfs.","Validate runtime properties at service startup (fail fast on empty deep-storage config).","Keep a single shared config source so all services see the same storage directory."],"tags":["hdfs","configuration","kill-all","missing-config"],"backgroundTag":"missing-required-config-field","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}