apache/beam · error · IOException

Error trying to delete %s: %s

Error message

Error trying to delete %s: %s

What it means

Thrown by GcsUtilV1's async delete callback when a GCS object deletion fails with any error other than 404 (404s are logged and ignored since the file already doesn't exist). The message embeds the target path and the full GoogleJsonError.

Source

Thrown at sdks/java/extensions/google-cloud-platform-core/src/main/java/org/apache/beam/sdk/extensions/gcp/util/GcsUtilV1.java:1407

  private void enqueueDelete(final GcsPath file, BatchInterface batch) throws IOException {
    Storage.Objects.Delete deleteRequest =
        storageClient.objects().delete(file.getBucket(), file.getObject());
    batch.queue(
        deleteRequest,
        new JsonBatchCallback<Void>() {
          @Override
          public void onSuccess(Void obj, HttpHeaders responseHeaders) {
            LOG.debug("Successfully deleted {}", file);
          }

          @Override
          public void onFailure(GoogleJsonError e, HttpHeaders responseHeaders) throws IOException {
            if (e.getCode() == 404) {
              LOG.info(
                  "Ignoring failed deletion of file {} which already does not exist: {}", file, e);
            } else {
              throw new IOException(String.format("Error trying to delete %s: %s", file, e));
            }
          }
        });
  }

  @VisibleForTesting
  interface BatchInterface {
    <T> void queue(AbstractGoogleJsonClientRequest<T> request, JsonBatchCallback<T> cb)
        throws IOException;

    void execute() throws IOException;

    int size();
  }
}

View on GitHub (pinned to 12126d8942)

Solutions

1. Read the embedded GoogleJsonError code: 403 means grant storage.objects.delete (or storage.admin) to the credential.
2. For 429/5xx, retry the deletion later or reduce deletion concurrency.
3. Verify the object isn't protected by a retention policy, event-based hold, or bucket policy.
4. If the file is expected to possibly not exist, no action needed for 404 — it's already ignored.

Example fix

// before: single service account with read-only access deletes temp files
// after: grant delete permission
//   gcloud projects add-iam-policy-binding PROJECT \
//     --member=serviceAccount:SA --role=roles/storage.objectAdmin

// verify delete permission ahead of teardown
// gcloud storage objects list gs://bucket --format=... requires objectViewer;
// deletion additionally requires roles/storage.objectAdmin or objectCreator+delete

try {
  gcsUtil.remove(ImmutableList.of(path));
} catch (IOException e) {
  if (e.getMessage().startsWith("Error trying to delete")) {
    // inspect embedded GoogleJsonError code: 403 fix IAM, 429/5xx retry
  }
}

When it happens

Trigger: Calling GcsUtilV1.remove (batched object deletion); the GCS JSON API returns a non-404 error for one object, e.g. 403 permission denied, 429 rate limit, or 5xx.

Common situations: Cleaning up temporary/staging files during pipeline teardown with credentials lacking storage.objects.delete permission; hitting GCS rate limits when deleting thousands of objects; object under retention/hold policies.

Understand the failure class

Background: "API error: {status}" and "HTTP 401/403/404/429/5xx" errors: non-2xx HTTP responses explained — this error's family across 27 libraries.

Related errors

• Error completing file copies with retries, sample: from %s t
• Failed closing channel to %s
• Error determining if %s allows dynamic splitting
• File spec %s not found
• Error matching file spec %s: status %s

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/727ca57908ed0754. Report an issue: GitHub.