apache/druid · info

Segment path [%s] does not exist

Error message

Segment path [%s] does not exist

What it means

HdfsDataSegmentKiller.kill() removes segment files from HDFS deep storage. Before deleting, it checks whether the segment path exists; if not, it logs this warning and returns silently instead of throwing. It is informational: the segment is considered already killed.

Source

Thrown at extensions-core/hdfs-storage/src/main/java/org/apache/druid/storage/hdfs/HdfsDataSegmentKiller.java:82

  {
    return new Path(String.valueOf(segment.getLoadSpec().get(PATH_KEY)));
  }

  @Override
  public void kill(DataSegment segment) throws SegmentLoadingException
  {
    final Path segmentPath = getPath(segment);
    log.info("Killing segment[%s] mapped to path[%s]", segment.getId(), segmentPath);

    try (final FileSystem fs = segmentPath.getFileSystem(config)) {
      final String filename = segmentPath.getName();
      final CompressionUtils.Format compressionFormat = CompressionUtils.Format.fromFileName(filename);
      if (compressionFormat != CompressionUtils.Format.ZIP && compressionFormat != CompressionUtils.Format.LZ4) {
        throw new SegmentLoadingException("Unknown file type[%s]", segmentPath);
      } else {

        if (!fs.exists(segmentPath)) {
          log.warn("Segment path [%s] does not exist", segmentPath);
          return;
        }

        // There are 3 supported path formats for each segment compression format:
        //    - hdfs://nn1/hdfs_base_directory/data_source_name/interval/version/shardNum/index.zip
        //    - hdfs://nn1/hdfs_base_directory/data_source_name/interval/version/shardNum_index.zip
        //    - hdfs://nn1/hdfs_base_directory/data_source_name/interval/version/shardNum_UUID_index.zip
        // The same formats with an index.lz4 suffix are also supported.
        final String[] segmentParts = filename.split("_");

        Path descriptorPath = new Path(segmentPath.getParent(), "descriptor.json");
        if (segmentParts.length > 1) {
          Preconditions.checkState(segmentParts.length <= 3 &&
                                   StringUtils.isNumeric(segmentParts[0]) &&
                                   ("index" + compressionFormat.getSuffix()).equals(
                                       segmentParts[segmentParts.length - 1]
                                   ),
                                   "Unexpected segmentPath format [%s]", segmentPath

View on GitHub (pinned to 9b90983fd2)

Solutions

1. No action required if the segment was intentionally deleted; treat as success
2. If the segment should exist, verify the path with `hdfs dfs -ls` against druid.storage.storageDirectory and the segment's dataSource/interval/version/partition
3. Check druid.storage.* config for base-directory changes that would make computed paths wrong

// caller-side check before kill
Path p = new Path(storageDir, segmentRelativePath);
if (!fs.exists(p)) { /* already gone; skip kill */ }

try {
  killer.kill(segment);
} catch (SegmentLoadingException e) {
  if (!fs.exists(path)) { /* benign: already deleted */ } else { throw e; }
}

When it happens

Trigger: Calling kill(DataSegment) for a segment whose index.zip / descriptor files were already deleted (or never pushed) at the computed HDFS path.

Common situations: Retrying a failed kill, running the Druid kill task twice, manually cleaning HDFS before the kill task runs, or segment paths changed after a deep-storage migration.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

• Skipping deep storage directory kill: relative path is empty
• Skipping deep storage directory kill: relative path must not
• Skipping deep storage directory kill: backslash not allowed
• Skipping deep storage directory kill: invalid path[%s]
• Skipping deep storage directory kill: storage directory not

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/4537f5331c1e4feb. Report an issue: GitHub.