{"record":{"id":"a4bc3287b7e6745e","repo":"apache/beam","slug":"sdk-docker-container-image-has-to-be-a-non-empty-string","errorCode":null,"errorMessage":"SDK Docker container image has to be a non-empty string","messagePattern":"SDK Docker container image has to be a non-empty string","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/runners/dataflow/internal/apiclient.py","lineNumber":844,"sourceCode":"      docker_payload = proto_utils.parse_Bytes(\n          environment.payload, beam_runner_api_pb2.DockerPayload)\n      overridden = False\n      new_container_image = docker_payload.container_image\n      for pattern, override in sdk_overrides.items():\n        new_container_image = re.sub(pattern, override, new_container_image)\n        if new_container_image != docker_payload.container_image:\n          overridden = True\n\n      # Container of the current (Python) SDK is overridden separately, hence\n      # not updated here.\n      if (is_apache_beam_container(new_container_image) and not overridden and\n          new_container_image != current_sdk_container_image):\n        new_container_image = (\n            DataflowApplicationClient._update_container_image_for_dataflow(\n                docker_payload.container_image))\n\n      if not new_container_image:\n        raise ValueError(\n            'SDK Docker container image has to be a non-empty string')\n\n      new_payload = copy(docker_payload)\n      new_payload.container_image = new_container_image\n      environment.payload = new_payload.SerializeToString()\n\n  def create_job_description(self, job: Job):\n    \"\"\"Creates a job described by the workflow proto.\"\"\"\n    DataflowApplicationClient._apply_sdk_environment_overrides(\n        job.proto_pipeline, self._sdk_image_overrides, job.options)\n\n    # Stage other resources for the SDK harness\n    resources = self._stage_resources(job.proto_pipeline, job.options)\n\n    # Stage proto pipeline.\n    serialized_pipeline = job.proto_pipeline.SerializeToString()\n    pipeline_proto_hash = hashlib.sha256(serialized_pipeline).hexdigest()\n","sourceCodeStart":826,"sourceCodeEnd":862,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py#L826-L862","documentation":"Raised by the Beam Dataflow runner when the SDK harness Docker container image resolves to an empty or falsy string after applying environment overrides. Dataflow workers need a valid container image to launch the SDK harness, so an empty image would produce a broken job. The library throws a ValueError early at pipeline submission to fail fast instead of at worker startup.","triggerScenarios":"Calling Pipeline.run() against DataflowRunner where a DockerPayload environment override exists but the resulting container image is empty — e.g. pipeline option --sdk_container_image set to an empty string, or a custom environment override setting container_image to None/''.","commonSituations":"CI config rendering an empty --sdk_container_image variable; templates with an unset placeholder for the container image; custom _apply_sdk_environment_overrides hooks passing a blank image.","solutions":["Set --sdk_container_image (or the equivalent container_image option) to a non-empty valid image URI, e.g. 'gcr.io/cloud-dataflow/v1beta3/beam_python3.11_sdk:latest'.","If overriding via a custom environment, ensure docker_payload.container_image is populated before the payload is serialized.","Check whether a config/template variable feeding sdk_container_image is interpolated to an empty string and give it a default."],"exampleFix":"// before\npython -m my_pipeline --runner DataflowRunner --sdk_container_image=\"\"\n// after\npython -m my_pipeline --runner DataflowRunner --sdk_container_image=\"gcr.io/cloud-dataflow/v1beta3/beam_python3.11_sdk:latest\"","handlingStrategy":"validation","validationCode":"image = getattr(pipeline_options, 'sdk_container_image', None)\nif not image:\n  raise ValueError('sdk_container_image must be set to a non-empty string before submitting to Dataflow')","typeGuard":"def has_container_image(opts) -> bool:\n    img = getattr(opts, 'sdk_container_image', None)\n    return isinstance(img, str) and img.strip() != ''","tryCatchPattern":"try:\n    result = pipeline.run()\nexcept ValueError as e:\n    if 'container image' in str(e):\n        logging.error('Set --sdk_container_image to a valid image URI: %s', e)\n    raise","preventionTips":["Always pass an explicit --sdk_container_image when using custom environments.","Validate template variables feeding the image option before submission.","Add a pre-submit assert that all pipeline options are non-empty strings."],"tags":["python","apache-beam","dataflow","config"],"backgroundTag":"empty-required-field","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}