apache/druid · error · org.apache.druid.java.util.common.IOE

Could not create temporary file

Error message

Could not create temporary file [%s] for copying [%s]

What it means

ChunkingStorageConnector's chunk-copy iterator creates a temporary file with File.createNewFile() before streaming each chunk. If the JDK reports the file already exists (createNewFile returns false) it throws this IOE. createNewFile also returns false if the file was concurrently created or the check-and-create atomically fails.

Solutions

1. Clear the temp directory of stale files from previous failed copies
2. Ensure each copy uses a unique temp directory or unique file names (UUID per attempt)
3. Avoid running concurrent copies of the same object into the same temp dir

Example fix

// before
File outFile = new File(sharedTmpDir, "chunk-" + chunkIndex);
// after
File outFile = File.createTempFile("chunk-" + chunkIndex + "-", ".tmp", tmpDir);

File outFile = new File(tmpDir, name);
if (outFile.exists()) {
  outFile = File.createTempFile("chunk-", ".tmp", tmpDir);
}

try { copyChunk(...); } catch (IOE e) { // temp file exists: use unique name or clean tmp dir }

When it happens

Trigger: Copying a cloud-storage object where the computed temp file path (in the given temp dir) already exists — e.g. a previous failed copy left the temp file behind, or two concurrent copies of the same chunk compute the same temp file name in a shared temp directory.

Common situations: Multiple Druid processes or retries sharing one tmp dir (same java.io.tmpdir or configured tmpDir); leftover temp files after a crash; concurrent segment downloads of the same segment to a shared cache directory.

Understand the failure class

Background: "already exists" / EEXIST / FileAlreadyExistsException: what the 'file already exists' error means and how to fix it — this error's family across 37 libraries.

Related errors

• Cannot delete temp file
• Unable to find temp file
• Attempt to add row to swapped-out sink for segment
• Background lookup manager thread could not be cancelled
• can't start.

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/f2340f30c1a40868. Report an issue: GitHub.

Appendix: source

Thrown at processing/src/main/java/org/apache/druid/storage/remote/ChunkingStorageConnector.java:143

          {
            if (!initStream) {
              initStream = true;
              return new NullInputStream();
            }

            File outFile = new File(
                params.getTempDirSupplier().get().getAbsolutePath(),
                UUID.randomUUID().toString()
            );

            long currentReadEndPosition = Math.min(
                currentReadStartPosition.get() + chunkSizeBytes,
                readEnd
            );

            try {
              if (!outFile.createNewFile()) {
                throw new IOE(
                    StringUtils.format(
                        "Could not create temporary file [%s] for copying [%s]",
                        outFile.getAbsolutePath(),
                        params.getCloudStoragePath()
                    )
                );
              }

              FileUtils.copyLarge(
                  () -> new RetryingInputStream<>(
                      params.getObjectSupplier().getObject(currentReadStartPosition.get(), currentReadEndPosition),
                      params.getObjectOpenFunction(),
                      params.getRetryCondition(),
                      params.getMaxRetry()
                  ),
                  outFile,
                  new byte[FETCH_BUFFER_SIZE_BYTES],
                  Predicates.alwaysFalse(),

View on GitHub (pinned to 9b90983fd2)