{"record":{"id":"4d74d515d8ee526b","repo":"apache/druid","slug":"failed-to-upload-s-to-s-4d74d5","errorCode":null,"errorMessage":"Failed to upload [%s] to [%s]","messagePattern":"Failed to upload \\[(.+?)\\] to \\[(.+?)\\]","errorType":"exception","errorClass":"RuntimeException","httpStatus":500,"severity":"error","filePath":"extensions-core/google-extensions/src/main/java/org/apache/druid/storage/google/GoogleTaskLogs.java","lineNumber":113,"sourceCode":"      InputStreamContent mediaContent = new InputStreamContent(\"text/plain\", fileStream);\n      mediaContent.setLength(logFile.length());\n\n      try {\n        RetryUtils.retry(\n            (RetryUtils.Task<Void>) () -> {\n              storage.insert(config.getBucket(), taskKey, mediaContent, UPLOAD_BUFFER_SIZE);\n              return null;\n            },\n            GoogleUtils::isRetryable,\n            1,\n            5\n        );\n      }\n      catch (IOException e) {\n        throw e;\n      }\n      catch (Exception e) {\n        throw new RE(e, \"Failed to upload [%s] to [%s]\", logFile, taskKey);\n      }\n    }\n  }\n\n  @Override\n  public Optional<InputStream> streamTaskLog(final String taskid, final long offset) throws IOException\n  {\n    final String taskKey = getTaskLogKey(taskid);\n    return streamTaskFile(offset, taskKey);\n  }\n\n  @Override\n  public Optional<InputStream> streamTaskReports(String taskid) throws IOException\n  {\n    final String taskKey = getTaskReportKey(taskid);\n    return streamTaskFile(0, taskKey);\n  }\n","sourceCodeStart":95,"sourceCodeEnd":131,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/extensions-core/google-extensions/src/main/java/org/apache/druid/storage/google/GoogleTaskLogs.java#L95-L131","documentation":"GoogleTaskLogs.pushTaskFile uploads a task log/report/status file to GCS. Any non-IOException failure during upload (per-file, after the configured retry attempts) is rethrown as this RuntimeException naming the local file and the target GCS key. It means the push permanently failed for that file.","triggerScenarios":"Calling pushTaskLog/pushTaskReports/pushTaskStatus when GCS upload fails with something other than IOException — e.g. StorageException from auth failures, bucket absence, quota, or a null/bad config key — after retries are exhausted.","commonSituations":"Service account missing storage.objects.create; bucket does not exist or name typo; disabled GCS APIs on the project; network partitions during a long-running task finish; misconfigured maxAttempts/retry config.","solutions":["Read the cause chain (RE wraps the original exception) to find the real StorageException and its reason.","Verify the service account has roles/storage.objectCreator on the target bucket.","Confirm the bucket exists and google-cloud-storage API is enabled for the project.","Check GoogleTaskLogsConfig: bucket, prefix, and retry counts are sensible.","Because the exception is a RuntimeException, wrap the push call if task completion should tolerate log-upload failure."],"exampleFix":"// before\ntaskLogs.pushTaskLog(taskId, logFile); // may blow up the task\n// after\ntry {\n  taskLogs.pushTaskLog(taskId, logFile);\n} catch (RuntimeException e) {\n  log.error(e, \"Could not push task log to GCS; keeping local copy %s\", logFile);\n}","handlingStrategy":"try-catch","validationCode":"if (!logFile.exists()) {\n  throw new IllegalStateException(\"Nothing to push: \" + logFile);\n}\n// and confirm bucket is set in GoogleTaskLogsConfig","typeGuard":null,"tryCatchPattern":"try { taskLogs.pushTaskLog(taskId, logFile); }\ncatch (RuntimeException e) {\n  log.error(e, \"GCS push failed for %s; local file retained\", logFile);\n}","preventionTips":["Pre-verify the target bucket exists and the service account can create objects.","Keep the wrapped cause (e.getCause()) to diagnose StorageException codes.","Retain local log files until push is confirmed.","Test GCS connectivity/permissions with a canary upload at startup."],"tags":["gcs","upload","task-logs","runtime-exception"],"backgroundTag":"file-write-failed","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}