{"record":{"id":"583c78c427ebb987","repo":"apache/beam","slug":"f-non-mocked-source-name-or-type-at-line-yaml-transform","errorCode":null,"errorMessage":"f'Non-mocked source {name_or_type} at line {yaml_transform.SafeLineLoader.get_line(transform)}'","messagePattern":"f'Non-mocked source (.+?) at line (.+?)'","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/yaml/yaml_testing.py","lineNumber":92,"sourceCode":"    pipeline_spec_dict = yaml.load(\n        pipeline_spec, Loader=yaml_utils.SafeLineLoader)\n  else:\n    pipeline_spec_dict = pipeline_spec\n\n  processed_pipeline_spec = _preprocess_for_testing(pipeline_spec_dict)\n\n  transform_spec, recording_ids = inject_test_tranforms(\n      processed_pipeline_spec,\n      test_spec,\n      fix_failures)\n\n  allowed_sources = set(test_spec.get('allowed_sources', []) + ['Create'])\n  for transform in transform_spec['transforms']:\n    name_or_type = transform.get('name', transform['type'])\n    if (not yaml_transform.empty_if_explicitly_empty(transform.get('input', []))\n        and not transform.get('name') in allowed_sources and\n        not transform['type'] in allowed_sources):\n      raise ValueError(\n          f'Non-mocked source {name_or_type} '\n          f'at line {yaml_transform.SafeLineLoader.get_line(transform)}')\n\n  if options is None:\n    options = beam.options.pipeline_options.PipelineOptions(\n        pickle_library='cloudpickle',\n        **yaml_transform.SafeLineLoader.strip_metadata(\n            pipeline_spec_dict.get('options', {})))\n\n  providers = yaml_provider.merge_providers(\n      yaml_provider.parse_providers(\n          '', pipeline_spec_dict.get('providers', [])),\n      {\n          'AssertEqualAndRecord': yaml_provider.as_provider_list(\n              'AssertEqualAndRecord', AssertEqualAndRecord)\n      })\n\n  with beam.Pipeline(options=options) as p:","sourceCodeStart":74,"sourceCodeEnd":110,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/yaml/yaml_testing.py#L74-L110","documentation":"run_test in yaml_testing enforces that all sources in a test pipeline are mocked or explicitly allowed. If a transform reads from a real source (its type or name not in allowed_sources and it has non-empty input handling), a ValueError naming the transform and its YAML line is raised, preventing accidental reads from production systems.","triggerScenarios":"Running a Beam YAML test whose pipeline contains a source transform (e.g. ReadFromBigQuery, Kafka Read) whose type or name is not listed in the test spec's allowed_sources and is not mocked via mock_inputs.","commonSituations":"Adding a new source to a pipeline without updating the test spec, forgetting to mock a database/file read, or a renamed transform no longer matching allowed_sources entries.","solutions":["Add the source's type or transform name to the test spec's allowed_sources list.","Replace the real source with mock_inputs in the test spec.","Remove the unmocked source if the test should not read external data."],"exampleFix":"# before\n- tests:\n  - name: my_test\n    expected_outputs: {...}\n\n# after\n- tests:\n  - name: my_test\n    allowed_sources: [ReadFromBigQuery]\n    expected_outputs: {...}","handlingStrategy":"validation","validationCode":"for t in transform_spec['transforms']:\n    if t.get('type') not in allowed_sources and t.get('name') not in allowed_sources:\n        raise ValueError(f'unmocked source: {t.get(\"name\", t[\"type\"])}')","typeGuard":null,"tryCatchPattern":"try:\n    run_test(test_spec, ...)\nexcept ValueError as e:\n    if 'Non-mocked source' in str(e):\n        add_source_to_allowed_sources_or_mock(e)\n    raise","preventionTips":["Mock every external source with mock_inputs","Keep allowed_sources in sync whenever pipeline sources change","Never add production sources to allowed_sources to make tests pass silently"],"tags":["python","apache-beam","yaml","testing"],"backgroundTag":"unsupported-operation","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T21:17:11.552Z"}