apache/beam · error · S3ClientError

The specified bucket does not exist

Error message

The specified bucket does not exist

What it means

FakeS3Client.delete raises S3ClientError('The specified bucket does not exist', 404) when the request's bucket was never registered in the fake's known_buckets set. It mirrors a 404 NoSuchBucket from real S3.

Solutions

1. Seed the bucket via fake_client.create_file(bucket, obj, contents) before deleting
2. Fix bucket-name mismatches between setup and the delete request
3. Verify the bucket is in fake_client.known_buckets before deleting
4. For delete_batch, ensure every bucket in the batch is known to the fake

Example fix

// before
fake_client.delete(messages.DeleteRequest('prod-bucket', 'tmp/1'))
// after
fake_client.create_file('prod-bucket', 'tmp/1', b'x')
fake_client.delete(messages.DeleteRequest('prod-bucket', 'tmp/1'))

if request.bucket not in fake_client.known_buckets:
    raise AssertionError(f'bucket {request.bucket} not registered in fake')

When it happens

Trigger: Calling delete() (directly or via delete_batch) with a bucket name that was never passed to create_file or otherwise added to known_buckets in the fake client.

Common situations: Tests deleting from a bucket spelled differently than the one seeded; fixture setup skipped; bucket name loaded from config differing from the test default.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

• Not Found
• Tried to list nonexistent S3 path: s3://
• Basepath %r must be S3 path.
• Checksum operation failed
• Delete operation failed

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/4663733e49a81d0e. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/io/aws/clients/s3/fake_client.py:150

      Args:
        request: (GetRequest) request
      Returns:
        (bytes) The response message.
      """

    file_ = self.get_file(request.bucket, request.object)

    # Replicates S3's behavior, per the spec here:
    # https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.35
    if start < 0 or end <= start:
      return file_.contents

    return file_.contents[start:end]

  def delete(self, request):
    if request.bucket not in self.known_buckets:
      raise messages.S3ClientError('The specified bucket does not exist', 404)

    if (request.bucket, request.object) in self.files:
      self.delete_file(request.bucket, request.object)
    else:
      # S3 doesn't raise an error if you try to delete a nonexistent file from
      # an extant bucket
      return

  def delete_batch(self, request):

    deleted, failed, errors = [], [], []
    for object in request.objects:
      try:
        delete_request = messages.DeleteRequest(request.bucket, object)
        self.delete(delete_request)
        deleted.append(object)
      except messages.S3ClientError as e:
        failed.append(object)

View on GitHub (pinned to 12126d8942)