apache/beam · error · RuntimeError

No tests found. If you haven't added a set of tests yet…

Error message

No tests found. If you haven't added a set of tests yet, you can get started by running your pipeline with the --create_test flag enabled.

What it means

Raised by run_tests in apache_beam.yaml.main when a YAML test suite resolves to an empty or missing 'tests' list. The YAML pipeline spec loaded (or provided inline) contains no 'tests' entries, so there is nothing to run. Beam throws this to distinguish 'your tests failed' from 'you never defined any tests'.

Solutions

1. Add a 'tests:' list with at least one test specification to the YAML file.
2. Run the pipeline once with the --create_test flag to scaffold an empty tests section, then fill in test cases.
3. Verify the key is exactly 'tests' and is a list of test mappings, not a mapping or misspelled key.

Example fix

// before
pipeline:
  type: ReadFromPubsub
  ...
// after
pipeline:
  type: ReadFromPubsub
  ...
tests:
  - name: basic
    elements: [1, 2, 3]
    expected: [2, 4, 6]

import yaml
spec = yaml.safe_load(open(suite_path))
tests = spec.get('tests', []) if isinstance(spec, dict) else []
if not isinstance(tests, list) or not tests:
    raise SystemExit('Test suite has no tests; add a tests: list or use --create_test')

def has_tests(spec) -> bool:
    return isinstance(spec, dict) and isinstance(spec.get('tests'), list) and len(spec['tests']) > 0

try:
    run_tests(...)
except RuntimeError as e:
    if 'No tests found' in str(e):
        scaffold_tests_with_create_test_flag()  # rerun with --create_test
    else:
        raise

When it happens

Trigger: Running `python -m apache_beam.yaml.main --test_suite file.yaml` (or a pipeline with test-parsing args) where the top-level mapping has no 'tests' key, or 'tests: []'. Also triggered when the loaded spec is not a dict and defaults to an empty test list.

Common situations: Developers enable --fix_tests or test mode on a plain YAML pipeline file that has no tests section yet; CI pipelines pointed at a test-suite file that was created as an empty stub ('tests: []'); typos like 'test:' instead of 'tests:'.

Understand the failure class

Background: "must not be empty", "cannot be empty" — required-field validation errors across open-source libraries — this error's family across 41 libraries.

Related errors

• Ambiguous expression type (perhaps missing quoting?)
• Ambiguous expression type (perhaps missing quoting?)
• At most one of --create_test and --fix_tests may be…
• Can only use expressions on a schema'd input.
• CombineFn spec missing type

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/5db6c3218bdba87c. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/yaml/main.py:185

  if known_args.create_test and known_args.fix_tests:
    raise ValueError(
        'At most one of --create_test and --fix_tests may be specified.')
  elif known_args.create_test:
    result = unittest.TestResult()
    tests = []
  else:
    if known_args.test_suite:
      with open(known_args.test_suite) as fin:
        test_suite_holder = yaml.load(
            fin, Loader=yaml_transform.SafeLineLoader) or {}
    else:
      test_suite_holder = pipeline_spec
    test_specs = test_suite_holder.get('tests', [])
    if not isinstance(test_specs, list):
      raise TypeError('tests attribute must be a list of test specifications.')
    elif not test_specs:
      raise RuntimeError(
          'No tests found. '
          "If you haven't added a set of tests yet, you can get started by "
          'running your pipeline with the --create_test flag enabled.')

    tests = [
        yaml_testing.YamlTestCase(
            pipeline_spec, test_spec, options, known_args.fix_tests)
        for test_spec in test_specs
    ]
    suite = unittest.TestSuite(tests)
    result = unittest.TextTestRunner().run(suite)

  if known_args.fix_tests or known_args.create_test:
    update_tests(known_args, pipeline_yaml, pipeline_spec, options, tests)

  if exit:
    # emulates unittest.main()
    sys.exit(0 if result.wasSuccessful() else 1)

View on GitHub (pinned to 12126d8942)