apache/druid · error · IOE

Failed to rename temp file [%s] and final segment path [%s]

Error message

Failed to rename temp file [%s] and final segment path [%s] is not present.

What it means

After pushing a segment to HDFS, HdfsDataSegmentPusher copies the built temp file to its final path. If the rename fails AND the final segment path does not already exist, neither failure nor prior success explains the state, so an IOException is thrown. When the final path exists, it is assumed a replica task already pushed the same segment and the error is downgraded to an info log.

Source

Thrown at extensions-core/hdfs-storage/src/main/java/org/apache/druid/storage/hdfs/HdfsDataSegmentPusher.java:240

      catch (IOException ex) {
        log.error(ex, "Failed to delete temp directory[%s]", tmpIndexFile.getParent());
      }
    }

    return dataSegment;
  }

  private void copyFilesWithChecks(final FileSystem fs, final Path from, final Path to) throws IOException
  {
    if (!HadoopFsWrapper.rename(fs, from, to)) {
      if (fs.exists(to)) {
        log.info(
            "Unable to rename temp file [%s] to segment path [%s], it may have already been pushed by a replica task.",
            from,
            to
        );
      } else {
        throw new IOE("Failed to rename temp file [%s] and final segment path [%s] is not present.", from, to);
      }
    }
  }

  @Override
  public Map<String, Object> makeLoadSpec(URI finalIndexZipFilePath)
  {
    return ImmutableMap.of("type", "hdfs", "path", finalIndexZipFilePath.toString());
  }

  /**
   * Due to https://issues.apache.org/jira/browse/HDFS-13 ":" are not allowed in
   * path names. So we format paths differently for HDFS.
   */

  @Override
  public String getStorageDir(DataSegment segment, boolean useUniquePath)
  {

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Check HDFS permissions on the segment storage directory for the Druid process user
2. Retry the push — the failure may be a transient HDFS/NameNode issue
3. Verify the configured druid.storage.storageDirectory exists and is writable; create/fix it with hdfs dfs -mkdir -p and chown
4. Inspect NameNode logs for rename failures around the push time

Example fix

// before
// push fails with IOE 'Failed to rename temp file ...'
hdfs dfs -mkdir -p /druid/segments && hdfs dfs -chown druid:druid /druid/segments
// after
// retry the ingestion task; rename succeeds

// pre-check with hadoop fs shell before ingesting
// hdfs dfs -test -w /druid/segments || hdfs dfs -mkdir -p /druid/segments
testHdfsWritable(storageDir);

null

try {
  pusher.push(segment, file);
} catch (IOException e) {
  if (e.getMessage().contains("Failed to rename temp file")) {
    // check HDFS permissions/NameNode health, then retry
  }
  throw e;
}

When it happens

Trigger: copyFilesWithChecks() (invoked from pushToFilePath) fails FileSystem.rename(tempFile, segmentPath) and a subsequent exists() check on the target path returns false.

Common situations: HDFS permission problems on the segment directory; NameNode issues or transient HDFS unavailability; running as a user without write access to the configured storage directory; concurrent pushing where replicas race but the target is genuinely absent.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

• Failed to access hdfs.
• Failed to rename [%s] to [%s]
• Input stream is null
• Failed to fetch google cloud storage object from bucket [%s]
• Failed to fetch google cloud storage object from bucket [%s]

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/b6cd87d65489acd1. Report an issue: GitHub.