apache/druid · error · SegmentLoadingException

Unknown file type[%s]

Error message

Unknown file type[%s]

What it means

HdfsDataSegmentKiller.kill deletes a segment's files on HDFS. It inspects the segment file name via CompressionUtils.Format.fromFileName and only knows how to kill ZIP and LZ4-compressed segments; anything else (including non-compressed or unrecognized names) raises this SegmentLoadingException. The killer intentionally refuses formats it cannot safely remove.

Source

Thrown at extensions-core/hdfs-storage/src/main/java/org/apache/druid/storage/hdfs/HdfsDataSegmentKiller.java:78

    }
  }

  private static Path getPath(DataSegment segment)
  {
    return new Path(String.valueOf(segment.getLoadSpec().get(PATH_KEY)));
  }

  @Override
  public void kill(DataSegment segment) throws SegmentLoadingException
  {
    final Path segmentPath = getPath(segment);
    log.info("Killing segment[%s] mapped to path[%s]", segment.getId(), segmentPath);

    try (final FileSystem fs = segmentPath.getFileSystem(config)) {
      final String filename = segmentPath.getName();
      final CompressionUtils.Format compressionFormat = CompressionUtils.Format.fromFileName(filename);
      if (compressionFormat != CompressionUtils.Format.ZIP && compressionFormat != CompressionUtils.Format.LZ4) {
        throw new SegmentLoadingException("Unknown file type[%s]", segmentPath);
      } else {

        if (!fs.exists(segmentPath)) {
          log.warn("Segment path [%s] does not exist", segmentPath);
          return;
        }

        // There are 3 supported path formats for each segment compression format:
        //    - hdfs://nn1/hdfs_base_directory/data_source_name/interval/version/shardNum/index.zip
        //    - hdfs://nn1/hdfs_base_directory/data_source_name/interval/version/shardNum_index.zip
        //    - hdfs://nn1/hdfs_base_directory/data_source_name/interval/version/shardNum_UUID_index.zip
        // The same formats with an index.lz4 suffix are also supported.
        final String[] segmentParts = filename.split("_");

        Path descriptorPath = new Path(segmentPath.getParent(), "descriptor.json");
        if (segmentParts.length > 1) {
          Preconditions.checkState(segmentParts.length <= 3 &&
                                   StringUtils.isNumeric(segmentParts[0]) &&

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Inspect the actual file at the segment path (hdfs dfs -ls) to see its real naming/format.
2. If it is a plain directory or unsupported layout, delete it manually with hdfs dfs -rm -r after confirming no coordinator use.
3. Verify the segment's loadSpec/payload in the metadata store matches the on-disk file; fix stale entries.
4. Keep only ZIP/LZ4 segments under HDFS deep storage, or re-ingest/re-compact segments into the supported layout.
5. Catch SegmentLoadingException in tooling and skip such segments rather than aborting a kill batch.

Example fix

// before
killer.kill(segment); // throws for non-zip/lz4
// after
try {
  killer.kill(segment);
} catch (SegmentLoadingException e) {
  log.warn("Skipping segment %s: %s", segment.getId(), e.getMessage());
  FileSystem fs = segmentPath.getFileSystem(config);
  fs.delete(segmentPath, true); // manual removal if verified
}

CompressionUtils.Format f = CompressionUtils.Format.fromFileName(new Path(path).getName());
boolean killable = f == CompressionUtils.Format.ZIP || f == CompressionUtils.Format.LZ4;

try { killer.kill(segment); }
catch (SegmentLoadingException e) {
  log.warn("Unsupported segment layout, manual cleanup needed: %s", e.getMessage());
}

When it happens

Trigger: Calling kill() on a segment whose path filename does not resolve to .zip or .lz4 — e.g. a directory-style or uncompressed segment, a renamed file, or a custom deep-storage layout.

Common situations: Segments written by very old Druid versions or migrated from other deep storage layouts; manually renamed/moved segment files on HDFS; metadata store entries pointing at non-standard paths; users invoking kill API on unsupported segment shapes.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

• Unable to kill segment, failed to delete [%s]
• Do not know how to handle file type at [%s]
• Segment path [%s] does not exist
• Skipping deep storage directory kill: relative path is empty
• Skipping deep storage directory kill: relative path must not

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/d8d17d43d1280888. Report an issue: GitHub.