{"record":{"id":"6d67f6f6c063d066","repo":"apache/druid","slug":"failed-to-fetch-google-cloud-storage-object-from-b-6d67f6","errorCode":null,"errorMessage":"Failed to fetch google cloud storage object from bucket [%s] and prefix [%s].","messagePattern":"Failed to fetch google cloud storage object from bucket \\[(.+?)\\] and prefix \\[(.+?)\\]\\.","errorType":"exception","errorClass":"IOException","httpStatus":500,"severity":"error","filePath":"extensions-core/google-extensions/src/main/java/org/apache/druid/storage/google/GoogleStorage.java","lineNumber":271,"sourceCode":"  {\n    List<Storage.BlobListOption> options = new ArrayList<>();\n\n    if (prefix != null) {\n      options.add(Storage.BlobListOption.prefix(prefix));\n    }\n\n    if (pageSize != null) {\n      options.add(Storage.BlobListOption.pageSize(pageSize));\n    }\n\n    if (pageToken != null) {\n      options.add(Storage.BlobListOption.pageToken(pageToken));\n    }\n\n    Page<Blob> blobPage = storage.get().list(bucket, options.toArray(new Storage.BlobListOption[0]));\n\n    if (blobPage == null) {\n      throw new IOE(\"Failed to fetch google cloud storage object from bucket [%s] and prefix [%s].\", bucket, prefix);\n    }\n\n\n    List<GoogleStorageObjectMetadata> googleStorageObjectMetadataList =\n        blobPage.streamValues()\n                .map(blob -> new GoogleStorageObjectMetadata(\n                    blob.getBucket(),\n                    blob.getName(),\n                    blob.getSize(),\n                    blob.getUpdateTimeOffsetDateTime()\n                        .toEpochSecond() * 1000\n                ))\n                .collect(Collectors.toList());\n\n    return new GoogleStorageObjectPage(googleStorageObjectMetadataList, blobPage.getNextPageToken());\n\n  }\n","sourceCodeStart":253,"sourceCodeEnd":289,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/extensions-core/google-extensions/src/main/java/org/apache/druid/storage/google/GoogleStorage.java#L253-L289","documentation":"GoogleStorage.list() performs a paged blob listing for a bucket+prefix. Although GCS list calls normally return an empty page instead of null, Druid defensively throws this IOException when the SDK returns a null Page, so callers never dereference a missing result. It indicates the listing request failed rather than 'no results'.","triggerScenarios":"Calling GoogleStorage.list(bucket, prefix, maxResults) when the Storage client returns a null Page — typically due to a client/transport problem, an invalid bucket argument, or a mocked/failed response in tests.","commonSituations":"Misspelled or renamed bucket in task-log/output config; transient GCS client failures; unit tests (expectListObjectsPageRequest) simulating null pages; network interruptions mid-list.","solutions":["Retry the list call; null pages are usually transient client failures.","Validate the bucket name against the project (gsutil ls or console) before listing.","Catch IOException and fall back to an empty metadata list if 'no objects' is an acceptable outcome.","Upgrade/inspect the google-cloud-storage client version if null pages recur with healthy buckets.","Enable client logging to distinguish auth vs transport causes."],"exampleFix":"// before\nList<GoogleStorageObjectMetadata> objs = googleStorage.list(bucket, prefix, limit);\n// after\nList<GoogleStorageObjectMetadata> objs;\ntry {\n  objs = googleStorage.list(bucket, prefix, limit);\n} catch (IOException e) {\n  log.warn(\"GCS list failed for gs://%s/%s, treating as empty\", bucket, prefix, e);\n  objs = Collections.emptyList();\n}","handlingStrategy":"try-catch","validationCode":"if (bucket == null || bucket.isEmpty() || prefix == null) {\n  throw new IllegalArgumentException(\"bucket and prefix required before GCS list\");\n}","typeGuard":null,"tryCatchPattern":"try { page = googleStorage.list(bucket, prefix, limit); }\ncatch (IOException e) { page = Collections.emptyList(); /* or retry */ }","preventionTips":["Retry transient list failures with backoff.","Validate bucket names against the project at startup.","Monitor GCS client errors; upgrade the google-cloud-storage library if null pages recur.","Use test mocks that mirror SDK behavior (empty page, not null)."],"tags":["gcs","io","list","google-cloud-storage"],"backgroundTag":"empty-api-response","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}