apache/beam · error · RuntimeError

Failed to build python sdk container image on google cloud…

Error message

Failed to build python sdk container image on google cloud build, please check build log for error.

What it means

The Google Cloud Build container builder polls the build via get_build until it leaves the working/queued states. If the final status is anything other than SUCCESS, RuntimeError is raised telling the user to consult the Cloud Build log, since the actual failure detail lives there.

Solutions

Open the Cloud Build console (or 'gcloud builds log <build-id>') to read the failure reason
Fix the failing build step shown in the log (dependency/network errors)
Increase the Cloud Build timeout if the build timed out
Verify the Cloud Build service account has permission to push to the target registry

Example fix

// before
cloudbuild config with default 10m timeout; heavy build times out
// after
options = {'timeout': '1800s'}  # raise timeout in the Build request

Defensive patterns

Strategy: retry

Validate before calling

from google.api_core import exceptions
client = cloudbuild_v1.services.cloud_build.CloudBuildClient()
# pre-flight: API enabled and service account can push
try:
    client.get_build(cloud_build_types.GetBuildRequest(id='0', project_id=project))
except exceptions.PermissionDenied:
    raise SystemExit('Enable cloudbuild.googleapis.com and grant build rights')

Type guard

def build_succeeded(status) -> bool:
    from google.cloud.devtools import cloudbuild_v1 as cloud_build_types
    return status == cloud_build_types.Build.Status.SUCCESS

Try / catch

try:
    builder.build_container_image(...)
except RuntimeError as e:
    if 'google cloud build' in str(e):
        logger.error('Cloud Build failed; fetch logs: gcloud builds list --filter=%s', build_id)
        raise SystemExit(6)

Prevention

Set a generous Cloud Build timeout for SDK image builds
Pre-warm the image once and reuse it rather than building per-run
Ensure the Cloud Build service account has write access to the target registry
Check gcloud builds log output as part of CI on failure

When it happens

Trigger: _invoke_docker_build_and_push on the Cloud Build path submits a build to Google Cloud Build, polls with _cloudbuild_client.get_build, and the build finishes with status FAILURE, TIMEOUT, CANCELLED, or INTERNAL_ERROR instead of SUCCESS.

Common situations: Dockerfile steps failing inside Cloud Build (pip resolution errors, network egress blocked); build exceeding the Cloud Build timeout; project quota or API not enabled; push step failing due to insufficient IAM permissions on the target GCR/Artifact Registry repo.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/87de5aad4254d33d. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/runners/portability/sdk_container_builder.py:296

    build_id, log_url = self._get_cloud_build_id_and_log_url(build.metadata)
    _LOGGER.info(
        'Building sdk container with Google Cloud Build, this may '
        'take a few minutes, you may check build log at %s' % log_url)

    # block until build finish, if build fails exception will be raised and
    # stops the job submission.
    response = self._cloudbuild_client.get_build(
        request=cloud_build_types.GetBuildRequest(
            id=build_id, project_id=project_id))
    while response.status in [cloud_build_types.Build.Status.QUEUED,
                              cloud_build_types.Build.Status.PENDING,
                              cloud_build_types.Build.Status.WORKING]:
      time.sleep(10)
      response = self._cloudbuild_client.get_build(
          cloud_build_types.GetBuildRequest(id=build_id, project_id=project_id))

    if response.status != cloud_build_types.Build.Status.SUCCESS:
      raise RuntimeError(
          'Failed to build python sdk container image on google cloud build, '
          'please check build log for error.')

    _LOGGER.info(
        "Python SDK container pre-build finished in %.2f seconds" %
        (time.time() - now))
    _LOGGER.info(
        "Python SDK container built and pushed as %s." % container_image_name)

  def _upload_to_gcs(self, local_file_path, gcs_location):
    bucket_name, blob_name = self._get_gcs_bucket_and_name(gcs_location)
    _LOGGER.info('Starting GCS upload to %s...', gcs_location)
    from google.cloud import storage
    from google.cloud.exceptions import Forbidden
    from google.cloud.exceptions import NotFound
    try:
      bucket = self._storage_client.get_bucket(bucket_name)
      blob = bucket.get_blob(blob_name)

View on GitHub (pinned to 12126d8942)