{"record":{"id":"ae2bff2ed8dc7c1b","repo":"apache/druid","slug":"failed-to-upload-s-to-s","errorCode":null,"errorMessage":"Failed to upload [%s] to [%s]","messagePattern":"Failed to upload \\[(.+?)\\] to \\[(.+?)\\]","errorType":"exception","errorClass":"RuntimeException","httpStatus":500,"severity":"error","filePath":"extensions-core/google-extensions/src/main/java/org/apache/druid/storage/google/GoogleDataSegmentPusher.java","lineNumber":77,"sourceCode":"      throws IOException\n  {\n    log.debug(\"Inserting [%s] to [%s]\", file, path);\n    try {\n      RetryUtils.retry(\n          (RetryUtils.Task<Void>) () -> {\n            storage.insert(config.getBucket(), path, new FileContent(contentType, file), null);\n            return null;\n          },\n          GoogleUtils::isRetryable,\n          1,\n          5\n      );\n    }\n    catch (IOException e) {\n      throw e;\n    }\n    catch (Exception e) {\n      throw new RE(e, \"Failed to upload [%s] to [%s]\", file, path);\n    }\n  }\n\n  @Override\n  public DataSegment push(final File indexFilesDir, final DataSegment segment, final boolean useUniquePath)\n      throws IOException\n  {\n    log.debug(\"Uploading [%s] to Google.\", indexFilesDir);\n    final String storageDir = this.getStorageDir(segment, useUniquePath);\n    return pushToPath(indexFilesDir, segment, storageDir);\n  }\n\n  @Override\n  public DataSegment pushToPath(File indexFilesDir, DataSegment segment, String storageDirSuffix) throws IOException\n  {\n    final int version = SegmentUtils.getVersionFromDir(indexFilesDir);\n    File indexFile = null;\n","sourceCodeStart":59,"sourceCodeEnd":95,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/extensions-core/google-extensions/src/main/java/org/apache/druid/storage/google/GoogleDataSegmentPusher.java#L59-L95","documentation":"GoogleDataSegmentPusher.insert uploads a segment's zipped index file to Google Cloud Storage. Non-IOException errors from the upload (GCS client StorageException for permission/quota/API failures) are wrapped in a RuntimeException naming the source file and destination path; IOExceptions are rethrown as-is per the method contract.","triggerScenarios":"Calling insert (via push during segment publishing) when the GCS blob create/upload raises StorageException (403 insufficient permissions, 404 bucket missing, 429 quota, resumable upload failures) or other RuntimeExceptions from the client.","commonSituations":"Service account lacking storage.objects.create on the target bucket; bucket name typo or deleted bucket; GCS rate limits during heavy segment publish; segment size exceeding GCS limits.","solutions":["Grant the service account storage.objects.create/roles-storage-objectAdmin on the bucket","Verify druid.storage.bucket (and prefix) point to an existing, correct bucket","Retry on transient failures (429/5xx); publishing is retried by the task framework","Check the cause StorageException message for the exact GCS API error code"],"exampleFix":"// before\npusher.push(segmentDir, segment, true); // RE: Failed to upload ...\n// after\n// ensure in GCS IAM: roles/storage.objectCreator for the Druid service account on the bucket\npusher.push(segmentDir, segment, true);","handlingStrategy":"retry","validationCode":"// Pre-flight: verify bucket exists and account has create rights\n// gsutil ls gs://druid-storage-bucket && gcloud storage buckets get-iam-policy gs://druid-storage-bucket","typeGuard":null,"tryCatchPattern":"try { pusher.push(dir, segment, true); } catch (Exception e) { if (isRetryableStorageError(e)) backoffRetry(); else throw e; }","preventionTips":["Grant storage.objects.create on the push bucket","Verify bucket name spelling and region","Back off on 429 rate limits during bulk publishing"],"tags":["google-cloud-storage","segment-push","upload","apache-druid"],"backgroundTag":"gcs-upload-failed","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}