apache/beam · error · RuntimeError

Failed to build sdk container with local docker, stderr

Error message

Failed to build sdk container with local docker, stderr:
 %s.

What it means

The local Docker container builder runs 'docker build . -t <image>' in a temp source dir. When docker exits nonzero (CalledProcessError), the build failed and RuntimeError wraps the captured stderr from the docker build process.

Solutions

Read the stderr in the message to find the failing Dockerfile step
Verify docker works: run 'docker info' and fix daemon/permission issues
Fix the failing build step (dependency install errors, missing context files)
Check disk space and Docker Hub pull limits; retry after resolving

Example fix

// before
subprocess.run(['docker', 'build', '.', '-t', image], check=True)  # fails silently in CI
// after
# ensure daemon reachable, then pin base image and fix step:
FROM python:3.11-slim@sha256:<digest>  # avoids rate limit / tag drift

Defensive patterns

Strategy: validation

Validate before calling

import subprocess
assert subprocess.run(['docker', 'info'], capture_output=True).returncode == 0, 'docker daemon not reachable; start Docker or fix permissions'

Type guard

def docker_ready() -> bool:
    import subprocess
    return subprocess.run(['docker', 'info'], capture_output=True).returncode == 0

Try / catch

try:
    builder.build_container_image(...)
except RuntimeError as e:
    if 'Failed to build sdk container with local docker' in str(e):
        logger.error('Docker build failed:\n%s', e)
        raise SystemExit(4)

Prevention

Verify 'docker info' works as the invoking user before kicking off builds
Pin base image tags/digests to avoid pull failures and rate limits
Keep free disk headroom; prune stale images regularly
Test the Dockerfile build locally before wiring it into the pipeline flow

When it happens

Trigger: _SdkContainerImageLocalBuilder._invoke_docker_build_and_push invokes 'docker build' with check=True and the docker daemon/tool fails (Dockerfile error, base image pull failure, daemon not running).

Common situations: Docker daemon not running/permissions (user not in docker group); base image unavailable or rate-limited on Docker Hub; Dockerfile build steps failing (pip install errors, missing files); disk space exhaustion.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/714eb191114efe9c. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/runners/portability/sdk_container_builder.py:173

    return available_builders[0]


class _SdkContainerImageLocalBuilder(SdkContainerImageBuilder):
  """SdkContainerLocalBuilder builds the sdk container image with local
  docker."""
  @classmethod
  def _builder_key(cls):
    return 'local_docker'

  def _invoke_docker_build_and_push(self, container_image_name):
    try:
      _LOGGER.info("Building sdk container, this may take a few minutes...")
      now = time.time()
      subprocess.run(['docker', 'build', '.', '-t', container_image_name],
                     check=True,
                     cwd=self._temp_src_dir)
    except subprocess.CalledProcessError as err:
      raise RuntimeError(
          'Failed to build sdk container with local docker, '
          'stderr:\n %s.' % err.stderr)
    else:
      _LOGGER.info(
          "Successfully built %s in %.2f seconds" %
          (container_image_name, time.time() - now))

    if self._docker_registry_push_url:
      _LOGGER.info("Pushing prebuilt sdk container...")
      try:
        subprocess.run(['docker', 'push', container_image_name], check=True)
      except subprocess.CalledProcessError as err:
        raise RuntimeError(
            'Failed to push prebuilt sdk container %s, stderr: \n%s' %
            (container_image_name, err.stderr))
      _LOGGER.info(
          "Successfully pushed %s in %.2f seconds" %
          (container_image_name, time.time() - now))

View on GitHub (pinned to 12126d8942)