apache/beam · warning

Could not locate yaml docs source directory

Error message

Could not locate yaml docs source directory {docs_src}. Skipping copying tests from docs.

What it means

Beam's copy_tests_from_docs helper warns and does nothing when the YAML docs source directory does not exist. It normally copies yaml*.md docs files into a destination directory for generated doc tests; a missing source dir means those doc-derived tests are not copied.

Solutions

1. Clone/fetch the full Beam repository including the docs/website yaml sources, then re-run.
2. Set or correct the docs source path so it points at the directory containing yaml*.md files.
3. If doc tests are not needed, treat this warning as safe to ignore.

Example fix

// before
$ python setup.py copy_tests_from_docs   # docs dir missing
// after
$ git clone --recurse-submodules https://github.com/apache/beam.git  # or fetch website yaml docs
$ python setup.py copy_tests_from_docs

import os
if not os.path.isdir(docs_src):
    print(f'docs source {docs_src} missing; fetch the beam website yaml docs first')

When it happens

Trigger: Running the copy_tests_from_docs setup target (or the build step that calls it) when the expected docs source directory (e.g. the beam website yaml docs path) is absent from the checkout.

Common situations: Shallow/partial clones that exclude the website docs; running from a Beam source tree without the docs submodule; CI checkouts that skip website content.

Related errors

• Could not find the provided transforms config source
• Could not generate external transform wrappers due to error
• Could not generate protos due to error
• Cython not found, cython extensions will not be generated…
• error occurred

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/e8ec34dfce4648d9. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/setup.py:275

    print(out.stdout)
  except subprocess.CalledProcessError as err:
    raise RuntimeError('Could not generate protos due to error: %s', err.stderr)


def copy_tests_from_docs():
  python_root = os.path.abspath(os.path.dirname(__file__))
  docs_src = os.path.normpath(
      os.path.join(
          python_root, '../../website/www/site/content/en/documentation/sdks'))
  docs_dest = os.path.normpath(
      os.path.join(python_root, 'apache_beam/yaml/docs'))
  if os.path.exists(docs_src):
    shutil.rmtree(docs_dest, ignore_errors=True)
    os.mkdir(docs_dest)
    for path in glob.glob(os.path.join(docs_src, 'yaml*.md')):
      shutil.copy(path, docs_dest)
  else:
    warnings.warn(
        f'Could not locate yaml docs source directory {docs_src}. '
        f'Skipping copying tests from docs.')


def generate_external_transform_wrappers():
  try:
    sdk_dir = os.path.abspath(os.path.dirname(__file__))
    script_exists = os.path.exists(
        os.path.join(sdk_dir, 'gen_xlang_wrappers.py'))
    config_exists = os.path.exists(
        os.path.join(
            os.path.dirname(sdk_dir), 'standard_external_transforms.yaml'))
    # we need both the script and the standard transforms config file.
    # at build time, we don't have access to apache_beam to discover and
    # retrieve external transforms, so the config file has to already exist
    if not script_exists or not config_exists:
      generated_transforms_dir = os.path.join(
          sdk_dir, 'apache_beam', 'transforms', 'xlang')

View on GitHub (pinned to 12126d8942)