apache/beam · error · S3ClientError

The specified upload does not exist

Error message

The specified upload does not exist

What it means

FakeS3Client.upload_part raises S3ClientError('The specified upload does not exist', 404) when the request's upload_id is not a key in the fake's multipart_uploads dict. It mirrors a 404 NoSuchUpload from real S3.

Solutions

1. Create the multipart upload first and use the returned upload_id for all parts
2. Ensure no complete_multipart_upload/abort happens before all upload_part calls
3. Keep the upload_id from create_multipart_upload in a variable rather than retyping it
4. Check for swallowed exceptions during upload creation in test setup

Example fix

// before
upload_id = 'hardcoded-id'
fake_client.upload_part(messages.UploadPartRequest(upload_id, 1, b'data'))
// after
upload_id = fake_client.create_multipart_upload('bucket', 'key').upload_id
fake_client.upload_part(messages.UploadPartRequest(upload_id, 1, b'data'))

assert upload_id in fake_client.multipart_uploads, f'upload {upload_id} not created/already completed'

try:
    fake_client.upload_part(req)
except messages.S3ClientError as e:
    if e.code == 404 and 'upload does not exist' in e.message:
        ...  # recreate or fail test with a clear message
    else:
        raise

When it happens

Trigger: Calling upload_part with an upload_id never created by create_multipart_upload, already completed/aborted, or mistyped/reconstructed differently.

Common situations: Test aborts an upload then still sends parts; upload_id stored/retrieved from a different variable; exception during create_multipart_upload swallowed, leaving parts to be uploaded against a stale ID.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

• All parts but the last must be larger than
• One or more of the specified parts could not be found
• Param validation failed on part number
• AWS dependencies are not installed, and no alternative…
• Basepath %r must be S3 path.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/8343f601f496ef43. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/io/aws/clients/s3/fake_client.py:196

    self.add_file(dest_file)

  def create_multipart_upload(self, request):
    # Create hash of bucket and key
    # Store upload_id internally
    upload_id = request.bucket + request.object
    self.multipart_uploads[upload_id] = {}
    return messages.UploadResponse(upload_id)

  def upload_part(self, request):
    # Save off bytes passed to internal data store
    upload_id, part_number = request.upload_id, request.part_number

    if part_number < 0 or not isinstance(part_number, int):
      raise messages.S3ClientError(
          'Param validation failed on part number', 400)

    if upload_id not in self.multipart_uploads:
      raise messages.S3ClientError('The specified upload does not exist', 404)

    self.multipart_uploads[upload_id][part_number] = request.bytes

    etag = '"%s"' % ('x' * 32)
    return messages.UploadPartResponse(etag, part_number)

  def complete_multipart_upload(self, request):
    MIN_PART_SIZE = 5 * 2**10  # 5 KiB

    parts_received = self.multipart_uploads[request.upload_id]

    # Check that we got all the parts that they intended to send
    part_numbers_to_confirm = set(part['PartNumber'] for part in request.parts)

    # Make sure all the expected parts are present
    if part_numbers_to_confirm != set(parts_received.keys()):
      raise messages.S3ClientError(
          'One or more of the specified parts could not be found', 400)

View on GitHub (pinned to 12126d8942)