apache/druid · error · SegmentLoadingException

Unable to load from local directory

Error message

Unable to load from local directory [%s]

What it means

LocalDataSegmentPuller.getSegmentFiles throws SegmentLoadingException when it fails to extract a locally stored segment that is a .tgz/.tar.gz archive. The underlying IOException (missing file, permissions, corrupt archive, full disk) is wrapped with the source file path. This is the local-filesystem deep-storage loader, so it means the segment archive on disk could not be expanded into the target directory.

Solutions

1. Verify the source file exists and is readable by the Druid process user (ls -l on the path in the message).
2. Test archive integrity with `tar -tzf <file>`; re-ingest or re-replicate the segment if it is corrupt.
3. Check disk space and write permission on the output directory.
4. Remove/re-download the bad segment copy: disable the segment and let Druid re-replicate from another replica.

Example fix

// before (manual copy skipped nested files)
cp -r /mnt/old-deepstorage/druid/segments/wikipedia/2015/ /mnt/deepstorage/druid/segments/wikipedia/2015/
// after (copy full segment dir, verify archive)
rsync -a /mnt/old-deepstorage/druid/segments/wikipedia/2015/01/ /mnt/deepstorage/druid/segments/wikipedia/2015/01/
tar -tzf /mnt/deepstorage/druid/segments/wikipedia/2015/01/2015-01-01T00:00:00.000Z_2015-01-02T00:00:00.000Z_day/0/index.zip

final File f = new File((String) segment.getLoadSpec().get("path"));
if (!f.isFile() || !f.canRead()) throw new IllegalStateException("segment source unreadable: " + f);
if (!f.getName().endsWith(".tar.gz") && !f.getName().endsWith(".tgz")) throw new IllegalStateException("not a tgz archive: " + f);

boolean isTgzSegment(Map<String,Object> loadSpec) {
  Object p = loadSpec.get("path");
  return p instanceof String && ((String) p).matches(".*\\.(tar\\.gz|tgz)$");
}

try {
  puller.getSegmentFiles(segment, outDir);
} catch (SegmentLoadingException e) {
  log.error(e, "failed to extract local segment %s; marking replica bad", segment.getId());
  // let Druid re-replicate from another replica
}

When it happens

Trigger: DataSegment whose loadSpec type is 'local' and whose file name ends in .tgz/.tar.gz, when CompressionUtils.tarGz to the output dir throws IOException during getSegmentFiles.

Common situations: Manually copied or trimmed druid segment directories on historical nodes; segments corrupted by incomplete writes or disk-full; files moved without their archive; read-permission loss after user change; corrupting archives by editing deep-storage files by hand.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• Asked to load path[ ], but it doesn't exist.
• Cannot list directory
• Do not know how to handle source
• Failed to rename [ ] to [ ]
• Unable to unzip file

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/ea71aabd9f8bbb74. Report an issue: GitHub.

Appendix: source

Thrown at server/src/main/java/org/apache/druid/segment/loading/LocalDataSegmentPuller.java:156

            continue;
          }

          final File newFile = new File(dir, oldFile.getName());
          final FileUtils.LinkOrCopyResult linkOrCopyResult = FileUtils.linkOrCopy(oldFile, newFile);
          link = link && linkOrCopyResult == FileUtils.LinkOrCopyResult.LINK;
          result.addFile(newFile);
        }
        log.info(
            "%s %d bytes from [%s] to [%s]",
            link ? "Linked" : "Copied",
            result.size(),
            sourceFile.getAbsolutePath(),
            dir.getAbsolutePath()
        );
        return result;
      }
      catch (IOException e) {
        throw new SegmentLoadingException(e, "Unable to load from local directory [%s]", sourceFile.getAbsolutePath());
      }
    } else if (CompressionUtils.isZip(sourceFile.getName())) {
      try {
        final FileUtils.FileCopyResult result = CompressionUtils.unzip(
            Files.asByteSource(sourceFile),
            dir,
            shouldRetryPredicate(),
            false
        );
        log.info(
            "Unzipped %d bytes from [%s] to [%s]",
            result.size(),
            sourceFile.getAbsolutePath(),
            dir.getAbsolutePath()
        );
        return result;
      }
      catch (IOException e) {

View on GitHub (pinned to 9b90983fd2)