{"record":{"id":"8eb2b8520a5632ec","repo":"apache/beam","slug":"field-file-pattern-not-found-in-input-schema-fields-field","errorCode":null,"errorMessage":"Field '{file_pattern}' not found in input schema fields: {field_names}","messagePattern":"Field '(.+?)' not found in input schema fields: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/yaml/yaml_io.py","lineNumber":890,"sourceCode":"      the file pattern string. If not specified and the input PCollection has\n      exactly one field, that field will be used.\n    empty_match_treatment (str): How to treat empty matches. Possible values are\n      'ALLOW', 'DISALLOW', and 'ALLOW_IF_WILDCARD'. Defaults to 'ALLOW'.\n  \"\"\"\n  from apache_beam.typehints import schemas\n\n  try:\n    field_names = [\n        name for name, _ in schemas.named_fields_from_element_type(\n            pcoll.element_type)\n    ]\n  except Exception:\n    field_names = None\n\n  if field_names:\n    if file_pattern is not None:\n      if file_pattern not in field_names:\n        raise ValueError(\n            f\"Field '{file_pattern}' not found in input schema fields: {field_names}\"\n        )\n      pattern_field = file_pattern\n    elif len(field_names) == 1:\n      pattern_field = field_names[0]\n    else:\n      raise ValueError(\n          f\"Input schema has multiple fields {field_names}. \"\n          f\"Please specify the 'file_pattern' parameter to select which field \"\n          f\"contains the file pattern.\")\n    patterns = pcoll | beam.Map(lambda x: str(getattr(x, pattern_field)))\n  else:\n    patterns = pcoll\n\n  matched = patterns | beam.io.fileio.MatchAll(\n      empty_match_treatment=empty_match_treatment)\n\n  return matched | beam.Map(","sourceCodeStart":872,"sourceCodeEnd":908,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/yaml/yaml_io.py#L872-L908","documentation":"match_all reads file patterns from a schema field of the input. When the user explicitly supplies file_pattern naming a field, that name must exist in the input schema; otherwise this ValueError lists the missing name and the actual fields.","triggerScenarios":"Calling match_all(file_pattern='some_field') where 'some_field' is not among the input PCollection's schema field names (and is being interpreted as a field name, not a glob).","commonSituations":"Confusion between a glob pattern and a field name: file_pattern here selects which schema field holds per-record patterns, not the pattern itself; typos or schema changes also trigger it.","solutions":["Pass the name of the input schema field that contains the file patterns (e.g. file_pattern='path')","Ensure the input schema has that field, or drop the argument so a single-field schema is used automatically","Print pcoll.element_type schema fields to confirm names"],"exampleFix":"// before\nmatch_all(file_pattern='gs://bucket/*.json')  # glob mistaken for field name\n// after\nmatch_all(file_pattern='file_path')  # schema field holding the patterns","handlingStrategy":"validation","validationCode":"field_names = None\ntry:\n    field_names = [n for n, _ in schemas.named_fields_from_element_type(pcoll.element_type)]\nexcept Exception:\n    pass\nif file_pattern is not None and field_names and file_pattern not in field_names:\n    raise ValueError(f'{file_pattern!r} is not a schema field: {field_names}')","typeGuard":null,"tryCatchPattern":"try:\n    pcoll | yaml_io.match_all(file_pattern=fp)\nexcept ValueError as e:\n    if 'not found in input schema fields' in str(e):\n        pcoll = pcoll | beam.Map(lambda r: beam.Row(file_path=r.path))","preventionTips":["Remember file_pattern here names a schema field, not a glob","Ensure the input schema includes a pattern/path field","Check field names against element_type before calling"],"tags":["python","apache-beam","filesystem","schema","field-not-found"],"backgroundTag":"resource-not-found","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}