apache/druid · warning

Skipping deep storage directory kill: relative path must not

Error message

Skipping deep storage directory kill: relative path must not be absolute, got [%s]

What it means

constructHdfsDeletePath rejects relative paths that start with '/' because an absolute path would cause the killer to delete outside (or at the root of) the configured deep-storage directory. It logs this warning and returns null, skipping the delete. This is a safety guard against misconfigured segment loadSpec paths.

Source

Thrown at extensions-core/hdfs-storage/src/main/java/org/apache/druid/storage/hdfs/HdfsDataSegmentKiller.java:187

    log.info("Deleting deep storage directory[%s]", dirToDelete);
    if (!fs.delete(dirToDelete, true)) {
      throw new IOException("Failed to delete deep storage directory[" + dirToDelete + "].");
    }
  }

  /**
   * Construct a path to delete from HDFS. Returns null if the path is invalid.
   * Replicates how {@link HdfsDataSegmentPusher#pushToPath} handles ':', by replacing that with '_'.
   */
  @Nullable
  private Path constructHdfsDeletePath(String relativePath)
  {
    if (Strings.isNullOrEmpty(relativePath)) {
      log.warn("Skipping deep storage directory kill: relative path is empty");
      return null;
    }
    if (relativePath.charAt(0) == '/') {
      log.warn("Skipping deep storage directory kill: relative path must not be absolute, got [%s]", relativePath);
      return null;
    }
    if (relativePath.indexOf('\\') >= 0) {
      log.warn("Skipping deep storage directory kill: backslash not allowed in path [%s]", relativePath);
      return null;
    }
    for (String segment : StringUtils.splitPreserveAllTokens(relativePath, '/')) {
      if (segment.isEmpty() || "..".equals(segment)) {
        log.warn("Skipping deep storage directory kill: invalid path[%s]", relativePath);
        return null;
      }
    }

    if (storageDirectory == null) {
      log.warn("Skipping deep storage directory kill: storage directory not configured");
      return null;
    }

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Fix the segment's loadSpec path in the metadata store to be relative to druid.storage.storageDirectory
2. Verify druid.storage.storageDirectory config is correct and that segments were pushed by the same config
3. Do not work around by editing the killer; the absolute path would otherwise delete arbitrary HDFS data

Example fix

// before (metadata loadSpec)
"path": "/druid/segments/datasource/2019-01-01T00:00:00.000Z_2019-01-02T00:00:00.000Z/2020-01-01T00:00:00.000Z/0/index.zip"
// after
"path": "datasource/2019-01-01T00:00:00.000Z_2019-01-02T00:00:00.000Z/2020-01-01T00:00:00.000Z/0/index.zip"

if (segmentPath.startsWith("/")) {
  throw new IllegalArgumentException("segment relative path must not be absolute: " + segmentPath);
}

static boolean isRelativePath(String p) { return p != null && !p.startsWith("/"); }

When it happens

Trigger: A DataSegment's loadSpec key/path begins with '/', e.g. a manually crafted segment or one pushed by a pusher configured with an absolute storage path.

Common situations: Mixing deep storage configs (base directory changed from relative to absolute), hand-migrated metadata rows, or segments copied between clusters with different HDFS layouts.

Understand the failure class

Background: Path traversal blocked: "path escapes the workspace" and "outside site root" errors when a path will not stay inside its allowed directory — this error's family across 26 libraries.

Related errors

• Skipping deep storage directory kill: relative path is empty
• Skipping deep storage directory kill: backslash not allowed
• Skipping deep storage directory kill: invalid path[%s]
• Segment path [%s] does not exist
• Skipping deep storage directory kill: storage directory not

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/a02db7070e9399f4. Report an issue: GitHub.