apache/druid · error · SegmentLoadingException

Do not know how to handle file type at [%s]

Error message

Do not know how to handle file type at [%s]

What it means

When the deep-storage path is a single file (not a directory) and its name does not match a supported compression format (ZIP, LZ4, or gzip per CompressionUtils), getSegmentFiles throws SegmentLoadingException("Do not know how to handle file type at [%s]"). Druid can only pull plain directories, index.zip/index.gz-zip archives, .lz4 files, or .gz files from HDFS; anything else is rejected by extension.

Source

Thrown at extensions-core/hdfs-storage/src/main/java/org/apache/druid/storage/hdfs/HdfsDataSegmentPuller.java:289

            {
              @Override
              public InputStream openStream() throws IOException
              {
                return getInputStream(path);
              }
            },
            outFile
        );

        log.info(
            "Gunzipped %d bytes from [%s] to [%s]",
            result.size(),
            path.toString(),
            outFile.getAbsolutePath()
        );
        return result;
      } else {
        throw new SegmentLoadingException("Do not know how to handle file type at [%s]", path.toString());
      }
    }
    catch (IOException e) {
      throw new SegmentLoadingException(e, "Error loading [%s]", path.toString());
    }
  }

  private void emitMetrics(CompressionUtils.Format format, long size, long duration)
  {
    if (emitter == null) {
      return;
    }
    ServiceMetricEvent.Builder metricBuilder = ServiceMetricEvent.builder();
    metricBuilder.setDimension("format", format);
    emitter.emit(metricBuilder.setMetric("hdfs/pull/size", size));
    emitter.emit(metricBuilder.setMetric("hdfs/pull/duration", duration));
  }

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Inspect the loadSpec path and the actual HDFS file name (`hdfs dfs -ls`); confirm it ends in .zip, .lz4, or .gz.
2. Re-push or re-ingest the segment so it is stored in the supported compressed format (index.zip).
3. If the path points at descriptor.json or another metadata file, correct the loadSpec to the segment archive path.
4. For custom deep-storage tooling, mimic HdfsDataSegmentPusher's naming (shardNum[_uuid]_index.zip) so old versions of the puller recognize it.

Example fix

// before: loadSpec path = hdfs://nn/druid/segments/ds/2020-01-01T00:00:00.000Z_2020-01-02T00:00:00.000Z/2020-01-01T00:00:00.000Z/v8/0/index  (no extension)
// after: store segment as index.zip and set loadSpec path accordingly
//   hdfs://nn/druid/segments/ds/2020-01-01T00:00:00.000Z_2020-01-02T00:00:00.000Z/2020-01-01T00:00:00.000Z/v8/0/index.zip

String name = new Path(loadSpecPath).getName();
CompressionUtils.Format f = CompressionUtils.Format.fromFileName(name);
boolean ok = f == CompressionUtils.Format.ZIP || f == CompressionUtils.Format.LZ4 || CompressionUtils.isGz(name);
if (!ok) { failFast("Unsupported segment artifact: " + name); }

try { puller.getSegmentFiles(path, outDir); }
catch (SegmentLoadingException e) {
  if (e.getMessage().contains("Do not know how to handle file type")) { rePushSegmentAsZip(segment); }
}

When it happens

Trigger: A loadSpec path pointing at a file whose name lacks a recognized suffix (.zip/.lz4/.gz) — e.g. raw uncompressed index files, descriptor.json itself, partially uploaded files without final names, or custom-pushed segment artifacts.

Common situations: Segments pushed by very old Druid versions or custom pushers storing uncompressed index files; loadSpec 'path' accidentally pointing to descriptor.json or a metadata file instead of the segment archive; manual copies into deep storage that renamed the segment file.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• Unknown file type[%s]
• Error loading [%s]
• Do not know how to handle source [%s]
• e.getMessage()
• '%s' must be a string or an array of strings

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/f248e1da5b7e3fb1. Report an issue: GitHub.