apache/beam · error · ValueError

Invalid process_variables

Error message

Invalid process_variables "%s" (expected assignment in the form "FOO=bar").

What it means

ProcessEnvironment.parse_environment_variables splits each process_variables entry on '=' to build an env dict; entries without an '=' raise ValueError telling the user each entry must look like 'FOO=bar'.

Solutions

1. Ensure every entry contains '=' in NAME=VALUE form (e.g. --process_variables=FOO=bar,BAZ=qux).
2. Quote the argument in your shell so '=' is preserved.
3. For an empty value use NAME= explicitly.

Example fix

// before
--process_variables=MY_VAR
// after
--process_variables=MY_VAR=1

def valid_process_vars(raw):
    return all('=' in v for v in raw.split(','))

try:
    env = ProcessEnvironment.parse_environment_variables(variables)
except ValueError as e:
    logging.error('Bad process_variables entry: %s', e)

When it happens

Trigger: Passing --process_variables=FOO (no '='), or an entry like '=bar' handled by split succeeding but malformed input such as 'FOO' alone; any entry where var.split('=', 1) does not yield two parts.

Common situations: Shell quoting mistakes dropping the =value part, setting a variable without a value, or YAML/CLI config joining arguments incorrectly.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

• Environment option ' ' must be set for process environment.
• environment with urn unimplemented
• External environment endpoint must be set.
• failed to start environment
• must supply dot_file argument

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/580894bd4072d4d9. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/transforms/environments.py:482

    return ProcessEnvironment(
        command=payload.command,
        os=payload.os,
        arch=payload.arch,
        env=payload.env,
        capabilities=capabilities,
        artifacts=artifacts,
        resource_hints=resource_hints,
    )

  @staticmethod
  def parse_environment_variables(variables):
    env = {}
    for var in variables:
      try:
        name, value = var.split('=', 1)
        env[name] = value
      except ValueError:
        raise ValueError(
            'Invalid process_variables "%s" (expected assignment in the '
            'form "FOO=bar").' % var)
    return env

  @classmethod
  def from_options(cls, options):
    # type: (PortableOptions) -> ProcessEnvironment
    if options.environment_config:
      config = json.loads(options.environment_config)
      return cls(
          config.get('command'),
          os=config.get('os', ''),
          arch=config.get('arch', ''),
          env=config.get('env', ''),
          capabilities=python_sdk_capabilities(),
          artifacts=python_sdk_dependencies(options),
          resource_hints=resource_hints_from_options(options),
      )

View on GitHub (pinned to 12126d8942)