apache/druid · error · IOException

Failed to rename [%s] to [%s]

Error message

Failed to rename [%s] to [%s]

What it means

LocalDataSegmentPusher.pushZip throws IOException when File.renameTo fails to move the freshly created temporary index.zip into its final location under the output directory. Java renameTo fails silently (returns false) on cross-filesystem moves or when the target exists/is locked, and the pusher surfaces that as this error.

Source

Thrown at server/src/main/java/org/apache/druid/segment/loading/LocalDataSegmentPusher.java:117

  }

  private DataSegment pushZip(final File inDir, final File outDir, final DataSegment baseSegment) throws IOException
  {
    final File tmpSegmentDir = new File(config.getStorageDirectory(), makeIntermediateDir());
    final File tmpIndexFile = new File(tmpSegmentDir, INDEX_ZIP_FILENAME);

    log.debug("Creating intermediate directory[%s] for segment[%s].", tmpSegmentDir.toString(), baseSegment.getId());
    FileUtils.mkdirp(tmpSegmentDir);

    try {
      log.debug("Compressing files from[%s] to [%s]", inDir, tmpIndexFile);
      final long size = CompressionUtils.zip(inDir, tmpIndexFile, true);

      FileUtils.mkdirp(outDir);
      final File indexFileTarget = new File(outDir, tmpIndexFile.getName());

      if (!tmpIndexFile.renameTo(indexFileTarget)) {
        throw new IOE("Failed to rename [%s] to [%s]", tmpIndexFile, indexFileTarget);
      }

      return baseSegment.withLoadSpec(makeLoadSpec(new File(outDir, INDEX_ZIP_FILENAME).toURI()))
                        .withSize(size);
    }
    finally {
      FileUtils.deleteDirectory(tmpSegmentDir);
    }
  }

  private DataSegment pushNoZip(final File inDir, final File outDir, final DataSegment baseSegment) throws IOException
  {
    final File tmpSegmentDir = new File(config.getStorageDirectory(), makeIntermediateDir());
    FileUtils.mkdirp(tmpSegmentDir);

    try {
      final File[] files = inDir.listFiles();
      if (files == null) {

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Put the local base storage directory on the same filesystem as the temp directory (or point java.io.tmpdir at the same volume).
2. Check write permission on the output directory for the Druid process user.
3. Check for a stale target file at indexFileTarget and remove it.
4. Free disk space / check filesystem errors (dmesg) on the target volume.

Example fix

// before: storage on data volume, tmp on root volume -> cross-device rename fails
# druid.segment.pusher.local.baseDir=/data/druid/segments
// after: same filesystem
# druid.segment.pusher.local.baseDir=/data/tmp/../druid/segments  (same mount as java.io.tmpdir=/data/tmp)

File outDir = new File(baseDir, interPath);
FileUtils.mkdirp(outDir);
if (!outDir.canWrite()) throw new IllegalStateException("storage dir not writable: " + outDir);
// also ensure same filesystem as java.io.tmpdir:
if (!outDir.getCanonicalPath().split("/")[1].equals(System.getProperty("java.io.tmpdir").split("/")[1])) {
  log.warn("tmp and storage dirs may be on different filesystems; renameTo may fail");
}

boolean sameFilesystem(File a, File b) throws IOException {
  return a.getCanonicalFile().toPath().getRoot().equals(b.getCanonicalFile().toPath().getRoot());
}

try {
  pusher.push(segment, outDir, true);
} catch (IOException e) {
  log.error(e, "segment push failed (rename); check tmp/storage filesystem layout");
  throw e;
}

When it happens

Trigger: pushToPath -> pushZip: after CompressionUtils.zip, tmpIndexFile.renameTo(indexFileTarget) returns false, typically because temp dir and storage dir are on different mount points/filesystems.

Common situations: druid.segment.pusher.local.baseDir on a different volume from java.io.tmpdir; target file already exists with different ownership; NFS/storage dir permissions changed; disk-full preventing directory entry creation.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

• Cannot list directory [%s]
• Failed to rename temp file [%s] and final segment path [%s]
• Unable to load from local directory [%s]
• Unexpected subdirectory [%s]
• RuntimeException wrapping underlying cause

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/cd2ce26921d16d79. Report an issue: GitHub.