{"record":{"id":"caf96853ded2b08b","repo":"apache/druid","slug":"unable-to-copy-s-to-s","errorCode":null,"errorMessage":"Unable to copy [%s] to [%s]","messagePattern":"Unable to copy \\[(.+?)\\] to \\[(.+?)\\]","errorType":"exception","errorClass":"org.apache.druid.java.util.common.RE","httpStatus":null,"severity":"error","filePath":"processing/src/main/java/org/apache/druid/storage/remote/ChunkingStorageConnector.java","lineNumber":171,"sourceCode":"                  () -> new RetryingInputStream<>(\n                      params.getObjectSupplier().getObject(currentReadStartPosition.get(), currentReadEndPosition),\n                      params.getObjectOpenFunction(),\n                      params.getRetryCondition(),\n                      params.getMaxRetry()\n                  ),\n                  outFile,\n                  new byte[FETCH_BUFFER_SIZE_BYTES],\n                  Predicates.alwaysFalse(),\n                  1,\n                  StringUtils.format(\n                      \"Retrying copying of [%s] to [%s]\",\n                      params.getCloudStoragePath(),\n                      outFile.getAbsolutePath()\n                  )\n              );\n            }\n            catch (IOException e) {\n              throw new RE(e, StringUtils.format(\"Unable to copy [%s] to [%s]\", params.getCloudStoragePath(), outFile));\n            }\n\n            try {\n              AtomicBoolean fileInputStreamClosed = new AtomicBoolean(false);\n              return new FileInputStream(outFile)\n              {\n                @Override\n                public void close() throws IOException\n                {\n                  // close should be idempotent\n                  if (fileInputStreamClosed.get()) {\n                    return;\n                  }\n                  fileInputStreamClosed.set(true);\n                  super.close();\n                  currentReadStartPosition.set(currentReadEndPosition);\n                  if (!outFile.delete()) {\n                    throw new RE(\"Cannot delete temp file [%s]\", outFile);","sourceCodeStart":153,"sourceCodeEnd":189,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/processing/src/main/java/org/apache/druid/storage/remote/ChunkingStorageConnector.java#L153-L189","documentation":"During the chunk copy, the connector writes the downloaded chunk stream into the temp file. Any IOException raised while reading the source (cloud storage) or writing the temp file is wrapped in this RuntimeException 'Unable to copy [source] to [dest]', preserving the original cause.","triggerScenarios":"nextElement() chunk copy where network read from cloud storage fails mid-stream (connection reset, 4xx/5xx), the temp disk is full, or write permission on the temp file is lost.","commonSituations":"Transient cloud storage outages during segment deep-storage reads; disk full on the Druid host (/tmp or segment cache volume); proxy/firewall dropping long-lived S3/GCS connections.","solutions":["Check the wrapped cause (getCause()) to distinguish network vs disk errors and act accordingly","Free disk space on the temp volume or point the temp dir to a larger mount","Retry the copy; transient network errors usually resolve with backoff"],"exampleFix":"// handling\ntry {\n  segment = connector.getSegment(...);\n} catch (RuntimeException e) {\n  logger.warn(e, \"chunk copy failed, will retry\");\n  // inspect e.getCause(): IOException reading source or writing temp file\n}","handlingStrategy":"retry","validationCode":"if (new File(outDir).getFreeSpace() < requiredBytes) {\n  throw new IOException(\"insufficient space in \" + outDir);\n}","typeGuard":null,"tryCatchPattern":"catch (RuntimeException e) {\n  IOException cause = (IOException) e.getCause();\n  logger.warn(e, \"copy failed; retrying with backoff\");\n}","preventionTips":["Monitor disk free space on temp/cache volumes","Configure retry with backoff for cloud-storage reads","Keep connections alive / tune timeouts for large segment copies"],"tags":["io-error","cloud-storage","temp-file"],"backgroundTag":"file-write-failed","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}