{"record":{"id":"b2d727b372ae1051","repo":"apache/druid","slug":"failed-to-fetch-google-cloud-storage-object-from-b-b2d727","errorCode":null,"errorMessage":"Failed to fetch google cloud storage object from bucket [%s] and path [%s].","messagePattern":"Failed to fetch google cloud storage object from bucket \\[(.+?)\\] and path \\[(.+?)\\]\\.","errorType":"exception","errorClass":"IOException","httpStatus":500,"severity":"error","filePath":"extensions-core/google-extensions/src/main/java/org/apache/druid/storage/google/GoogleStorage.java","lineNumber":213,"sourceCode":"    if (statuses.contains(false)) {\n      log.debug(\n          \"Google cloud storage object(s) to be deleted not found in bucket [%s].\",\n          bucket\n      );\n    }\n  }\n\n  public boolean exists(final String bucket, final String path)\n  {\n    Blob blob = storage.get().get(bucket, path);\n    return blob != null;\n  }\n\n  public long size(final String bucket, final String path) throws IOException\n  {\n    Blob blob = storage.get().get(bucket, path, Storage.BlobGetOption.fields(Storage.BlobField.SIZE));\n    if (blob == null) {\n      throw new IOE(\"Failed to fetch google cloud storage object from bucket [%s] and path [%s].\", bucket, path);\n    }\n    return blob.getSize();\n  }\n\n  /**\n   * Return the etag for an object. This is a value that changes whenever the object's data or metadata changes and is\n   * typically but not always the MD5 hash of the object. Ref:\n   * <a href=\"https://cloud.google.com/storage/docs/hashes-etags#etags\">ETags</a>\n   *\n   * @param bucket\n   * @param path\n   *\n   * @return\n   *\n   * @throws IOException\n   */\n  public String version(final String bucket, final String path) throws IOException\n  {","sourceCodeStart":195,"sourceCodeEnd":231,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/extensions-core/google-extensions/src/main/java/org/apache/druid/storage/google/GoogleStorage.java#L195-L231","documentation":"GoogleStorage.size() fetches only the SIZE field of a GCS blob via the Storage client. When the object does not exist at bucket/path (or is not visible to the credentials), the SDK returns null and Druid converts that into an IOException instead of returning a size. It signals 'object not found', not a transport failure.","triggerScenarios":"Calling GoogleStorage.size(bucket, path) for a key that was never written, was deleted, has a different case/encoding than requested, or is in a bucket the service account cannot read.","commonSituations":"Task log/report lookups before the task pushed its files; stale metadata pointing at segments pruned by retention policies; typos in the configured GCS bucket or prefix; service account lacking storage.objects.get on the bucket.","solutions":["Verify the object exists with the same credentials: gsutil -o Credentials/... ls gs://<bucket>/<path> (or storage.get(bucket, path) != null) before calling size().","Check the configured bucket and key-building logic (prefix, task id substitution ':' -> '_') for mismatches.","Grant the service account roles/storage.objectViewer on the bucket.","Handle null/IOException in callers so a missing object is treated as absent rather than fatal.","If objects were legitimately deleted, refresh the metadata/segment state instead of retrying the lookup."],"exampleFix":"// before\nlong size = druidStorage.size(bucket, path);\n// after\nBlob blob = storage.get(bucket, path, Storage.BlobGetOption.fields(Storage.BlobField.SIZE));\nif (blob == null) {\n  log.warn(\"GCS object gs://%s/%s not found, skipping\", bucket, path);\n  return -1;\n}\nlong size = blob.getSize();","handlingStrategy":"validation","validationCode":"boolean exists = storage.get(bucket, path, Storage.BlobGetOption.fields(Storage.BlobField.SIZE)) != null;\nif (!exists) { /* skip or alert before calling size() */ }","typeGuard":"static boolean gcsObjectExists(Storage storage, String bucket, String path) {\n  return storage.get(bucket, path, Storage.BlobGetOption.fields(Storage.BlobField.SIZE)) != null;\n}","tryCatchPattern":"try { long size = googleStorage.size(bucket, path); }\ncatch (IOException e) { log.warn(\"GCS object gs://%s/%s missing\", bucket, path); }","preventionTips":["Existence-check objects before size/version reads.","Keep bucket/prefix config consistent with the writer's key scheme.","Grant service accounts read access to all target buckets.","Treat missing objects as a normal state in polling code."],"tags":["gcs","io","object-not-found","google-cloud-storage"],"backgroundTag":"resource-not-found","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}