apache/druid · error · ISE

Cannot delete all segment files since druid.storage.storageD

Error message

Cannot delete all segment files since druid.storage.storageDirectory is not set.

What it means

HdfsDataSegmentKiller.killAll() refuses to run when the HdfsDataSegmentPusherConfig has no druid.storage.storageDirectory configured (the field is null because the constructor saw an empty/null value). Since killAll would otherwise recursively delete the entire deep-storage segment root, Druid fails fast with IllegalStateException rather than deleting an unknown/incorrect location. This is a missing-required-configuration guard, thrown before any HDFS call is made.

Source

Thrown at extensions-core/hdfs-storage/src/main/java/org/apache/druid/storage/hdfs/HdfsDataSegmentKiller.java:133

        }

        // descriptor.json is a file to store segment metadata in deep storage. This file is deprecated and not stored
        // anymore, but we still delete them if exists.
        fs.delete(descriptorPath, false);

        removeEmptyParentDirectories(fs, segmentPath, segmentParts.length > 1 ? 2 : 3);
      }
    }
    catch (IOException e) {
      throw new SegmentLoadingException(e, "Unable to kill segment");
    }
  }

  @Override
  public void killAll() throws IOException
  {
    if (storageDirectory == null) {
      throw new ISE("Cannot delete all segment files since druid.storage.storageDirectory is not set.");
    }

    log.info("Deleting all segment files from hdfs dir [%s].", storageDirectory.toUri().toString());
    final FileSystem fs = storageDirectory.getFileSystem(config);
    fs.delete(storageDirectory, true);
  }

  private void removeEmptyParentDirectories(final FileSystem fs, final Path segmentPath, final int depth)
  {
    Path path = segmentPath;
    try {
      for (int i = 1; i <= depth; i++) {
        path = path.getParent();
        if (fs.listStatus(path).length != 0 || !fs.delete(path, false)) {
          break;
        }
      }
    }

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Set druid.storage.storageDirectory to the HDFS base directory (e.g. druid.storage.storageDirectory=/druid/segments) in the runtime.properties of the services that kill segments, then restart.
2. Confirm the property key spelling and that it is non-empty (Strings.isNullOrEmpty check) on the node executing killAll.
3. If you do not intend to ever use killAll on this cluster, avoid invoking it and delete individual segments instead.
4. Verify the injected HdfsDataSegmentPusherConfig actually loads from the intended config source (no shadowing default config).

Example fix

// before (runtime.properties)
# druid.storage.storageDirectory=   (missing/empty)
// after
druid.storage.type=hdfs
druid.storage.storageDirectory=/druid/segments

String dir = props.getProperty("druid.storage.storageDirectory");
if (dir == null || dir.isEmpty()) {
  throw new IllegalStateException("Set druid.storage.storageDirectory before calling killAll");
}

try { killer.killAll(); }
catch (ISE e) { log.error("Configure druid.storage.storageDirectory to use killAll"); }

When it happens

Trigger: Calling killAll() (e.g. via the coordinator's full datasource deletion flow) on a cluster where druid.storage.storageDirectory is absent or empty in HdfsDataSegmentPusherConfig.

Common situations: Operator removed or never set druid.storage.storageDirectory in the runtime properties while druid.storage.type=hdfs; killAll invoked from a service whose config injection uses a default HdfsDataSegmentPusherConfig; migrating from local storage to HDFS deep storage without updating the pusher config.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

• taskLogDir [%s] must be a directory.
• Skipping deep storage directory kill: storage directory not
• The gRPC query server requires either a Basic or Anonymous a
• Metric [%s] not whitelisted.
• Can't load TrustStore. Truststore path or password is not se

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/3f8a5ea2d514640b. Report an issue: GitHub.