apache/beam · error · RuntimeError

Test fixing only supported for file-backed tests. Please…

Error message

Test fixing only supported for file-backed tests. Please use the --test_suite flag.

What it means

Raised by update_tests in apache_beam.yaml.main when test fixing is requested but the tests were supplied inline (e.g. via --yaml_pipeline on the command line) rather than in a file. Auto-fixing rewrites the YAML file on disk, which is only possible when tests are file-backed.

Solutions

Write the YAML pipeline (including tests) to a file and pass it via --test_suite.
Remove the --fix_tests/--create_test flag if you don't need the tests written back to disk.
Pipe the YAML into a temp file first, e.g. `beam_yaml > suite.yaml`, then run test fixing against that file.

Example fix

// before
python -m apache_beam.yaml.main --yaml_pipeline "pipeline: ..." --fix_tests
// after
python -m apache_beam.yaml.main --test_suite my_pipeline.yaml --fix_tests

Defensive patterns

Strategy: validation

Validate before calling

import sys
args = sys.argv
if ('--fix_tests' in args or '--create_test' in args) and '--test_suite' not in args and '--yaml_pipeline_file' not in args:
    raise SystemExit('Test fixing requires a file: pass --test_suite <file.yaml>')

Type guard

def is_file_backed(known_args) -> bool:
    return bool(getattr(known_args, 'test_suite', None) or getattr(known_args, 'yaml_pipeline_file', None))

Try / catch

try:
    update_tests(...)
except RuntimeError as e:
    if 'file-backed tests' in str(e):
        write_yaml_to_tempfile(pipeline_yaml); retry_with_test_suite_flag()
    else:
        raise

Prevention

Always materialize inline YAML to a file before enabling --fix_tests
Use --test_suite for all test-fixing workflows
Document in CI that test fixing requires file-backed input

When it happens

Trigger: Running with test-fixing flags (--fix_tests or --create_test) where neither --test_suite nor --yaml_pipeline_file points at an existing file — i.e. the pipeline YAML came from a string/inline source with no path to write back to.

Common situations: Developers pass the pipeline YAML inline via stdin or --yaml_pipeline while also requesting --fix_tests; CI jobs that generate the pipeline dynamically and then try to fix tests.

Understand the failure class

Background: "--flag is required" and "must specify" CLI errors: how missing-required-flag validation works and how to fix it — this error's family across 20 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/973e31a555e6cac3. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/yaml/main.py:218

  if exit:
    # emulates unittest.main()
    sys.exit(0 if result.wasSuccessful() else 1)
  else:
    if not result.wasSuccessful():
      raise RuntimeError(result)


def update_tests(known_args, pipeline_yaml, pipeline_spec, options, tests):
  if known_args.test_suite:
    path = known_args.test_suite
    if not os.path.exists(path) and known_args.create_test:
      with open(path, 'w') as fout:
        fout.write('tests: []')
  elif known_args.yaml_pipeline_file:
    path = known_args.yaml_pipeline_file
  else:
    raise RuntimeError(
        'Test fixing only supported for file-backed tests. '
        'Please use the --test_suite flag.')
  with open(path) as fin:
    original_yaml = fin.read()
  if path == known_args.yaml_pipeline_file and pipeline_yaml.strip(
  ) != original_yaml.strip():
    raise RuntimeError(
        'In-file test fixing not yet supported for templated pipelines. '
        'Please use the --test_suite flag.')
  updated_spec = yaml.load(original_yaml, Loader=yaml.SafeLoader) or {}

  if known_args.fix_tests:
    updated_spec['tests'] = [test.fixed_test() for test in tests]

  if known_args.create_test:
    if 'tests' not in updated_spec:
      updated_spec['tests'] = []
    updated_spec['tests'].append(

View on GitHub (pinned to 12126d8942)