apache/druid · error · SegmentLoadingException

Do not know how to handle source [%s]

Error message

Do not know how to handle source [%s]

What it means

LocalDataSegmentPuller.getSegmentFiles throws SegmentLoadingException when the local segment file's name is none of the recognized archive types (tgz/tar.gz, zip, gz). The puller dispatches on the file extension and cannot extract an unknown-format source. It indicates a segment file on local deep storage with an unexpected name or format.

Source

Thrown at server/src/main/java/org/apache/druid/segment/loading/LocalDataSegmentPuller.java:192

      catch (IOException e) {
        throw new SegmentLoadingException(e, "Unable to unzip file [%s]", sourceFile.getAbsolutePath());
      }
    } else if (CompressionUtils.isGz(sourceFile.getName())) {
      final File outFile = new File(dir, CompressionUtils.getGzBaseName(sourceFile.getName()));
      final FileUtils.FileCopyResult result = CompressionUtils.gunzip(
          Files.asByteSource(sourceFile),
          outFile,
          shouldRetryPredicate()
      );
      log.info(
          "Gunzipped %d bytes from [%s] to [%s]",
          result.size(),
          sourceFile.getAbsolutePath(),
          outFile.getAbsolutePath()
      );
      return result;
    } else {
      throw new SegmentLoadingException("Do not know how to handle source [%s]", sourceFile.getAbsolutePath());
    }
  }


  @Override
  public InputStream getInputStream(URI uri) throws IOException
  {
    return buildFileObject(uri).openInputStream();
  }

  /**
   * Returns the "version" (aka last modified timestamp) of the URI of interest
   *
   * @param uri The URI to check the last modified timestamp
   *
   * @return The last modified timestamp in ms of the URI in String format
   */
  @Override

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Point the segment loadSpec 'path' at the actual archive file (e.g. index.zip), not a directory or plain file.
2. Confirm the segment was pushed with druid.storage.type=local so naming matches what the puller expects.
3. Re-ingest or re-push the segment to regenerate a standard archive layout.
4. If the segment really is uncompressed, place the files in the expected directory structure and re-push with the local pusher.

Example fix

// before: loadSpec pointing at a directory
{"type":"local","path":"/ds/druid/segments/wikipedia/2015-01-01T00_..."}
// after: loadSpec pointing at the zip
{"type":"local","path":"/ds/druid/segments/wikipedia/2015-01-01T00_.../index.zip"}

String path = (String) segment.getLoadSpec().get("path");
String name = new File(path).getName();
boolean recognized = name.endsWith(".zip") || name.endsWith(".gz") || name.endsWith(".tar.gz") || name.endsWith(".tgz");
if (!recognized) throw new IllegalStateException("puller cannot handle segment source: " + name);

boolean isRecognizedSegmentSource(Map<String,Object> loadSpec) {
  Object p = loadSpec.get("path");
  if (!(p instanceof String)) return false;
  String n = new File((String) p).getName();
  return n.endsWith(".zip") || n.endsWith(".tar.gz") || n.endsWith(".tgz") || n.endsWith(".gz");
}

try {
  puller.getSegmentFiles(segment, outDir);
} catch (SegmentLoadingException e) {
  log.error(e, "unsupported segment format for %s; re-push required", segment.getId());
}

When it happens

Trigger: getSegmentFiles called with a sourceFile whose name does not match CompressionUtils.isTarGz, isZip, or isGz — e.g. loadSpec path pointing at a directory, an uncompressed file, or a misnamed archive.

Common situations: Hand-crafted loadSpec paths in segments.json; segments produced by a nonstandard pusher; files renamed manually; loadSpec pointing at the directory instead of index.zip; migration from other storage formats.

Related errors

• Do not know how to handle file type at [%s]
• Unable to load from local directory [%s]
• Asked to load path[%s], but it doesn't exist.
• e.getMessage()
• Unknown file type[%s]

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/0209a11b82f7a95b. Report an issue: GitHub.