What causes "Unable to load from local directory [%s]" in apache/druid?

Verify the source file exists and is readable by the Druid process user (ls -l on the path in the message).

apache/druid · error · SegmentLoadingException

Unable to load from local directory [%s]

Error message

Unable to load from local directory [%s]

What it means

LocalDataSegmentPuller.getSegmentFiles throws SegmentLoadingException when it fails to extract a locally stored segment that is a .tgz/.tar.gz archive. The underlying IOException (missing file, permissions, corrupt archive, full disk) is wrapped with the source file path. This is the local-filesystem deep-storage loader, so it means the segment archive on disk could not be expanded into the target directory.

Source

Thrown at server/src/main/java/org/apache/druid/segment/loading/LocalDataSegmentPuller.java:156

            continue;
          }

          final File newFile = new File(dir, oldFile.getName());
          final FileUtils.LinkOrCopyResult linkOrCopyResult = FileUtils.linkOrCopy(oldFile, newFile);
          link = link && linkOrCopyResult == FileUtils.LinkOrCopyResult.LINK;
          result.addFile(newFile);
        }
        log.info(
            "%s %d bytes from [%s] to [%s]",
            link ? "Linked" : "Copied",
            result.size(),
            sourceFile.getAbsolutePath(),
            dir.getAbsolutePath()
        );
        return result;
      }
      catch (IOException e) {
        throw new SegmentLoadingException(e, "Unable to load from local directory [%s]", sourceFile.getAbsolutePath());
      }
    } else if (CompressionUtils.isZip(sourceFile.getName())) {
      try {
        final FileUtils.FileCopyResult result = CompressionUtils.unzip(
            Files.asByteSource(sourceFile),
            dir,
            shouldRetryPredicate(),
            false
        );
        log.info(
            "Unzipped %d bytes from [%s] to [%s]",
            result.size(),
            sourceFile.getAbsolutePath(),
            dir.getAbsolutePath()
        );
        return result;
      }
      catch (IOException e) {

View on GitHub (pinned to 9b90983fd2)

Solutions

Verify the source file exists and is readable by the Druid process user (ls -l on the path in the message).
Test archive integrity with `tar -tzf <file>`; re-ingest or re-replicate the segment if it is corrupt.
Check disk space and write permission on the output directory.
Remove/re-download the bad segment copy: disable the segment and let Druid re-replicate from another replica.

Example fix

// before (manual copy skipped nested files)
cp -r /mnt/old-deepstorage/druid/segments/wikipedia/2015/ /mnt/deepstorage/druid/segments/wikipedia/2015/
// after (copy full segment dir, verify archive)
rsync -a /mnt/old-deepstorage/druid/segments/wikipedia/2015/01/ /mnt/deepstorage/druid/segments/wikipedia/2015/01/
tar -tzf /mnt/deepstorage/druid/segments/wikipedia/2015/01/2015-01-01T00:00:00.000Z_2015-01-02T00:00:00.000Z_day/0/index.zip

Defensive patterns

Strategy: try-catch

Validate before calling

final File f = new File((String) segment.getLoadSpec().get("path"));
if (!f.isFile() || !f.canRead()) throw new IllegalStateException("segment source unreadable: " + f);
if (!f.getName().endsWith(".tar.gz") && !f.getName().endsWith(".tgz")) throw new IllegalStateException("not a tgz archive: " + f);

Type guard

boolean isTgzSegment(Map<String,Object> loadSpec) {
  Object p = loadSpec.get("path");
  return p instanceof String && ((String) p).matches(".*\\.(tar\\.gz|tgz)$");
}

Try / catch

try {
  puller.getSegmentFiles(segment, outDir);
} catch (SegmentLoadingException e) {
  log.error(e, "failed to extract local segment %s; marking replica bad", segment.getId());
  // let Druid re-replicate from another replica
}

Prevention

Never modify or partially copy files in Druid deep storage by hand; use rsync -a and verify with tar -tzf.
Monitor disk space on deep-storage and segment-cache volumes.
Keep at least 2 segment replicas so a corrupt copy can be replaced.
Run deep storage under a stable service user with consistent permissions.

When it happens

Trigger: DataSegment whose loadSpec type is 'local' and whose file name ends in .tgz/.tar.gz, when CompressionUtils.tarGz to the output dir throws IOException during getSegmentFiles.

Common situations: Manually copied or trimmed druid segment directories on historical nodes; segments corrupted by incomplete writes or disk-full; files moved without their archive; read-permission loss after user change; corrupting archives by editing deep-storage files by hand.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/ea71aabd9f8bbb74. Report an issue: GitHub.