{"record":{"id":"f248e1da5b7e3fb1","repo":"apache/druid","slug":"do-not-know-how-to-handle-file-type-at-s","errorCode":null,"errorMessage":"Do not know how to handle file type at [%s]","messagePattern":"Do not know how to handle file type at \\[(.+?)\\]","errorType":"exception","errorClass":"SegmentLoadingException","httpStatus":null,"severity":"error","filePath":"extensions-core/hdfs-storage/src/main/java/org/apache/druid/storage/hdfs/HdfsDataSegmentPuller.java","lineNumber":289,"sourceCode":"            {\n              @Override\n              public InputStream openStream() throws IOException\n              {\n                return getInputStream(path);\n              }\n            },\n            outFile\n        );\n\n        log.info(\n            \"Gunzipped %d bytes from [%s] to [%s]\",\n            result.size(),\n            path.toString(),\n            outFile.getAbsolutePath()\n        );\n        return result;\n      } else {\n        throw new SegmentLoadingException(\"Do not know how to handle file type at [%s]\", path.toString());\n      }\n    }\n    catch (IOException e) {\n      throw new SegmentLoadingException(e, \"Error loading [%s]\", path.toString());\n    }\n  }\n\n  private void emitMetrics(CompressionUtils.Format format, long size, long duration)\n  {\n    if (emitter == null) {\n      return;\n    }\n    ServiceMetricEvent.Builder metricBuilder = ServiceMetricEvent.builder();\n    metricBuilder.setDimension(\"format\", format);\n    emitter.emit(metricBuilder.setMetric(\"hdfs/pull/size\", size));\n    emitter.emit(metricBuilder.setMetric(\"hdfs/pull/duration\", duration));\n  }\n","sourceCodeStart":271,"sourceCodeEnd":307,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/extensions-core/hdfs-storage/src/main/java/org/apache/druid/storage/hdfs/HdfsDataSegmentPuller.java#L271-L307","documentation":"When the deep-storage path is a single file (not a directory) and its name does not match a supported compression format (ZIP, LZ4, or gzip per CompressionUtils), getSegmentFiles throws SegmentLoadingException(\"Do not know how to handle file type at [%s]\"). Druid can only pull plain directories, index.zip/index.gz-zip archives, .lz4 files, or .gz files from HDFS; anything else is rejected by extension.","triggerScenarios":"A loadSpec path pointing at a file whose name lacks a recognized suffix (.zip/.lz4/.gz) — e.g. raw uncompressed index files, descriptor.json itself, partially uploaded files without final names, or custom-pushed segment artifacts.","commonSituations":"Segments pushed by very old Druid versions or custom pushers storing uncompressed index files; loadSpec 'path' accidentally pointing to descriptor.json or a metadata file instead of the segment archive; manual copies into deep storage that renamed the segment file.","solutions":["Inspect the loadSpec path and the actual HDFS file name (`hdfs dfs -ls`); confirm it ends in .zip, .lz4, or .gz.","Re-push or re-ingest the segment so it is stored in the supported compressed format (index.zip).","If the path points at descriptor.json or another metadata file, correct the loadSpec to the segment archive path.","For custom deep-storage tooling, mimic HdfsDataSegmentPusher's naming (shardNum[_uuid]_index.zip) so old versions of the puller recognize it."],"exampleFix":"// before: loadSpec path = hdfs://nn/druid/segments/ds/2020-01-01T00:00:00.000Z_2020-01-02T00:00:00.000Z/2020-01-01T00:00:00.000Z/v8/0/index  (no extension)\n// after: store segment as index.zip and set loadSpec path accordingly\n//   hdfs://nn/druid/segments/ds/2020-01-01T00:00:00.000Z_2020-01-02T00:00:00.000Z/2020-01-01T00:00:00.000Z/v8/0/index.zip","handlingStrategy":"validation","validationCode":"String name = new Path(loadSpecPath).getName();\nCompressionUtils.Format f = CompressionUtils.Format.fromFileName(name);\nboolean ok = f == CompressionUtils.Format.ZIP || f == CompressionUtils.Format.LZ4 || CompressionUtils.isGz(name);\nif (!ok) { failFast(\"Unsupported segment artifact: \" + name); }","typeGuard":null,"tryCatchPattern":"try { puller.getSegmentFiles(path, outDir); }\ncatch (SegmentLoadingException e) {\n  if (e.getMessage().contains(\"Do not know how to handle file type\")) { rePushSegmentAsZip(segment); }\n}","preventionTips":["Only push segments through HdfsDataSegmentPusher so files get the index.zip/index.lz4 naming.","Point loadSpec at the segment archive, never at descriptor.json or raw index files.","When copying segments manually, preserve the original compressed file names."],"tags":["hdfs","segment-loading","unsupported-format","file-extension"],"backgroundTag":"unsupported-operation","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}