apache/beam · error · RuntimeError

External transform wrappers have not been generated and the…

Error message

External transform wrappers have not been generated and the generation script `gen_xlang_wrappers.py` and/or the standard external transforms config could not be found

What it means

setup.py's generate_external_transform_wrappers checks for both the gen_xlang_wrappers.py script and the standard external transforms config YAML. If wrappers haven't been generated and either file is missing, it raises RuntimeError naming whichever pieces could not be found, because it cannot create the wrapper modules needed for cross-language transforms.

Solutions

1. Restore the missing file: git checkout -- sdks/python/gen_xlang_wrappers.py and/or standard_external_transforms.yaml
2. Generate wrappers first by running gen_xlang_wrappers.py directly so the skip branch succeeds
3. Build from a complete Beam source tree or use the released sdist which includes generated wrappers
4. Fix path resolution if sdk_dir is computed incorrectly in your environment

Example fix

# before
$ python setup.py build
RuntimeError: External transform wrappers have not been generated ... could not be found
# after
$ git checkout -- sdks/python/gen_xlang_wrappers.py standard_external_transforms.yaml
$ python sdks/python/gen_xlang_wrappers.py
$ python setup.py build

import os
script = os.path.join(sdk_dir, 'gen_xlang_wrappers.py')
config = os.path.join(os.path.dirname(sdk_dir), 'standard_external_transforms.yaml')
assert os.path.exists(script) and os.path.exists(config), 'xlang wrapper inputs missing'
assert glob.glob(os.path.join(sdk_dir, 'apache_beam', 'transforms', 'xlang_wrapper*')), 'wrappers not generated'

try:
    build()
except RuntimeError as e:
    if 'External transform wrappers have not been generated' in str(e):
        restore_missing_files_or_run_gen_xlang_wrappers()
        build()
    else:
        raise

When it happens

Trigger: Building apache_beam from a source tree missing sdks/python/gen_xlang_wrappers.py or the sibling standard_external_transforms.yaml, with no pre-generated wrapper modules present under apache_beam/transforms/xlang_wrapper (or equivalent).

Common situations: Partial/incomplete checkouts or sdists; files deleted by clean scripts; building a slimmed-down fork where the YAML config was removed; wrong sdk_dir resolution pointing at a nonexistent path.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

• Could not find the provided transforms config source
• Could not generate external transform wrappers due to error
• Could not generate protos due to error
• Could not locate yaml docs source directory
• Cython not found, cython extensions will not be generated…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/b066794175449aa2. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/setup.py:304

        os.path.join(
            os.path.dirname(sdk_dir), 'standard_external_transforms.yaml'))
    # we need both the script and the standard transforms config file.
    # at build time, we don't have access to apache_beam to discover and
    # retrieve external transforms, so the config file has to already exist
    if not script_exists or not config_exists:
      generated_transforms_dir = os.path.join(
          sdk_dir, 'apache_beam', 'transforms', 'xlang')

      # if exists, this directory will have at least its __init__.py file
      if (not os.path.exists(generated_transforms_dir) or
          len(os.listdir(generated_transforms_dir)) <= 1):
        message = 'External transform wrappers have not been generated '
        if not script_exists:
          message += 'and the generation script `gen_xlang_wrappers.py`'
        if not config_exists:
          message += 'and the standard external transforms config'
        message += ' could not be found'
        raise RuntimeError(message)
      else:
        logging.info(
            'Skipping external transform wrapper generation as they '
            'are already generated.')
      return
    subprocess.run([
        sys.executable,
        os.path.join(sdk_dir, 'gen_xlang_wrappers.py'),
        '--cleanup',
        '--transforms-config-source',
        os.path.join(
            os.path.dirname(sdk_dir), 'standard_external_transforms.yaml')
    ],
                   capture_output=True,
                   check=True)
  except subprocess.CalledProcessError as err:
    raise RuntimeError(
        'Could not generate external transform wrappers due to '

View on GitHub (pinned to 12126d8942)