apache/druid · error · org.apache.druid.java.util.common.RE

Unable to copy [ ] to [ ]

Error message

Unable to copy [%s] to [%s]

What it means

During the chunk copy, the connector writes the downloaded chunk stream into the temp file. Any IOException raised while reading the source (cloud storage) or writing the temp file is wrapped in this RuntimeException 'Unable to copy [source] to [dest]', preserving the original cause.

Solutions

1. Check the wrapped cause (getCause()) to distinguish network vs disk errors and act accordingly
2. Free disk space on the temp volume or point the temp dir to a larger mount
3. Retry the copy; transient network errors usually resolve with backoff

Example fix

// handling
try {
  segment = connector.getSegment(...);
} catch (RuntimeException e) {
  logger.warn(e, "chunk copy failed, will retry");
  // inspect e.getCause(): IOException reading source or writing temp file
}

if (new File(outDir).getFreeSpace() < requiredBytes) {
  throw new IOException("insufficient space in " + outDir);
}

catch (RuntimeException e) {
  IOException cause = (IOException) e.getCause();
  logger.warn(e, "copy failed; retrying with backoff");
}

When it happens

Trigger: nextElement() chunk copy where network read from cloud storage fails mid-stream (connection reset, 4xx/5xx), the temp disk is full, or write permission on the temp file is lost.

Common situations: Transient cloud storage outages during segment deep-storage reads; disk full on the Druid host (/tmp or segment cache volume); proxy/firewall dropping long-lived S3/GCS connections.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

• Cannot delete temp file
• Cannot load resource
• Could not create temporary file
• Could not delete cache file at
• Encountered error while reading the output of stage

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/caf96853ded2b08b. Report an issue: GitHub.

Appendix: source

Thrown at processing/src/main/java/org/apache/druid/storage/remote/ChunkingStorageConnector.java:171

                  () -> new RetryingInputStream<>(
                      params.getObjectSupplier().getObject(currentReadStartPosition.get(), currentReadEndPosition),
                      params.getObjectOpenFunction(),
                      params.getRetryCondition(),
                      params.getMaxRetry()
                  ),
                  outFile,
                  new byte[FETCH_BUFFER_SIZE_BYTES],
                  Predicates.alwaysFalse(),
                  1,
                  StringUtils.format(
                      "Retrying copying of [%s] to [%s]",
                      params.getCloudStoragePath(),
                      outFile.getAbsolutePath()
                  )
              );
            }
            catch (IOException e) {
              throw new RE(e, StringUtils.format("Unable to copy [%s] to [%s]", params.getCloudStoragePath(), outFile));
            }

            try {
              AtomicBoolean fileInputStreamClosed = new AtomicBoolean(false);
              return new FileInputStream(outFile)
              {
                @Override
                public void close() throws IOException
                {
                  // close should be idempotent
                  if (fileInputStreamClosed.get()) {
                    return;
                  }
                  fileInputStreamClosed.set(true);
                  super.close();
                  currentReadStartPosition.set(currentReadEndPosition);
                  if (!outFile.delete()) {
                    throw new RE("Cannot delete temp file [%s]", outFile);

View on GitHub (pinned to 9b90983fd2)