{"record":{"id":"ad0069c50d9f80c1","repo":"apache/iceberg","slug":"failed-to-refresh-storage-credentials-ad0069","errorCode":null,"errorMessage":"Failed to refresh storage credentials","messagePattern":"Failed to refresh storage credentials","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"gcp/src/main/java/org/apache/iceberg/gcp/gcs/GCSFileIO.java","lineNumber":258,"sourceCode":"  private void refreshStorageCredentials() {\n    if (isResourceClosed.get()) {\n      return;\n    }\n\n    try (OAuth2RefreshCredentialsHandler handler =\n        OAuth2RefreshCredentialsHandler.create(properties)) {\n      List<StorageCredential> refreshed =\n          handler.fetchCredentials().credentials().stream()\n              .filter(c -> c.prefix().startsWith(ROOT_STORAGE_PREFIX))\n              .map(c -> StorageCredential.create(c.prefix(), c.config()))\n              .toList();\n\n      if (!refreshed.isEmpty() && !isResourceClosed.get()) {\n        this.storageCredentials = Lists.newArrayList(refreshed);\n        scheduleCredentialRefresh();\n      }\n    } catch (Exception e) {\n      LOG.warn(\"Failed to refresh storage credentials\", e);\n    }\n  }\n\n  private ScheduledExecutorService executorService() {\n    if (executorService == null) {\n      synchronized (GCSFileIO.class) {\n        if (executorService == null) {\n          executorService =\n              ThreadPools.newExitingScheduledPool(\n                  \"iceberg-gcsfileio-tasks\", 1, Duration.ofSeconds(10));\n        }\n      }\n    }\n\n    return executorService;\n  }\n\n  @Override","sourceCodeStart":240,"sourceCodeEnd":276,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/gcp/src/main/java/org/apache/iceberg/gcp/gcs/GCSFileIO.java#L240-L276","documentation":"GCSFileIO.refreshStorageCredentials periodically refreshes delegated/stored storage credentials. When the refresh throws, it logs this warning (with the cause) and keeps the old credentials; the executor is only rescheduled if refresh succeeded and the resource is still open. This can silently lead to expired credentials later.","triggerScenarios":"The scheduled credential refresh task runs while the identity/credential provider fails (expired OAuth token, network error to the token endpoint, IAM permission changes) and throws an Exception.","commonSituations":"Long-running jobs where short-lived tokens expire mid-run; temporary network outage to GCS metadata/token server; misconfigured impersonation/service-account permissions introduced after job start.","solutions":["Inspect the logged exception cause to fix the underlying token/IAM issue","Ensure the workload has rights to the credential/impersonation chain being refreshed","Verify network access to the token endpoint from the job environment","Restart the job if credentials have fully expired, since refresh failures are swallowed"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// pre-check token validity before long jobs\nassert tokenExpiry > Instant.now().plus(Duration.ofHours(1));","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Use workload identity/long-lived credential strategies for long jobs","Monitor logs for repeated refresh warnings","Ensure egress to the token endpoint from job runners","Restart jobs whose credentials fully expired"],"tags":["gcs","credentials","token-refresh","background-task"],"backgroundTag":"oauth-token-exchange-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}