apache/beam · error · ValueError

External environment endpoint must be set.

Error message

External environment endpoint must be set.

What it means

ExternalEnvironment.from_options parses environment_config as JSON and requires a 'url' key giving the external environment's gRPC endpoint; if the JSON parses but has no url (or the value is empty), it raises ValueError 'External environment endpoint must be set.'

Solutions

1. Add the endpoint to the JSON: {"url": "host:port"}.
2. Or pass the endpoint directly as environment_config='host:port' instead of JSON.
3. Or set the external_service_address pipeline option so the fallback lookup finds it.

Example fix

// before
--environment_config='{"params": {"docker_image": "x"}}'
// after
--environment_config='{"url": "localhost:50051", "params": {}}'

import json
def has_url(cfg):
    try:
        return bool(json.loads(cfg).get('url'))
    except json.JSONDecodeError:
        return True  # raw endpoint string is fine

try:
    env = ExternalEnvironment.from_options(options)
except ValueError as e:
    if 'endpoint must be set' in str(e): ...

When it happens

Trigger: --environment_config='{"params": {...}}' (JSON without 'url'), or environment_config set to '{"url": null}'. The non-JSON / fallback branches (raw endpoint string or lookup_environment_option) are unaffected.

Common situations: Hand-written JSON config omitting the url key, config templated from another environment type (e.g. process config pasted as external), or url intentionally left blank expecting a default that doesn't exist.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

• Both withDataSourceConfiguration and…
• couldn't decode characteristic for variant
• Either Write Statement or Table must be set.
• Enrichment requests to Vertex AI Feature Store should…
• environment with urn unimplemented

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/62fee679ed4e405e. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/transforms/environments.py:575

      resource_hints,  # type: Mapping[str, bytes]
      context  # type: PipelineContext
  ):
    # type: (...) -> ExternalEnvironment
    return ExternalEnvironment(
        payload.endpoint.url,
        params=payload.params or None,
        capabilities=capabilities,
        artifacts=artifacts,
        resource_hints=resource_hints)

  @classmethod
  def from_options(cls, options):
    # type: (PortableOptions) -> ExternalEnvironment
    if looks_like_json(options.environment_config):
      config = json.loads(options.environment_config)
      url = config.get('url')
      if not url:
        raise ValueError('External environment endpoint must be set.')
      params = config.get('params')
    elif options.environment_config:
      url = options.environment_config
      params = None
    else:
      url = options.lookup_environment_option('external_service_address')
      params = None

    return cls(
        url,
        params=params,
        capabilities=python_sdk_capabilities(),
        artifacts=python_sdk_dependencies(options),
        resource_hints=resource_hints_from_options(options))


def expand_anyof_environments(env_proto):
  if env_proto.urn == common_urns.environments.ANYOF.urn:

View on GitHub (pinned to 12126d8942)