{"record":{"id":"e7ad951f803758e4","repo":"apache/beam","slug":"input-schema-has-multiple-fields-field-names-please-specify","errorCode":null,"errorMessage":"Input schema has multiple fields {field_names}. Please specify the 'file_pattern' parameter to select which field contains the file pattern.","messagePattern":"Input schema has multiple fields (.+?)\\. Please specify the 'file_pattern' parameter to select which field contains the file pattern\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/yaml/yaml_io.py","lineNumber":897,"sourceCode":"  try:\n    field_names = [\n        name for name, _ in schemas.named_fields_from_element_type(\n            pcoll.element_type)\n    ]\n  except Exception:\n    field_names = None\n\n  if field_names:\n    if file_pattern is not None:\n      if file_pattern not in field_names:\n        raise ValueError(\n            f\"Field '{file_pattern}' not found in input schema fields: {field_names}\"\n        )\n      pattern_field = file_pattern\n    elif len(field_names) == 1:\n      pattern_field = field_names[0]\n    else:\n      raise ValueError(\n          f\"Input schema has multiple fields {field_names}. \"\n          f\"Please specify the 'file_pattern' parameter to select which field \"\n          f\"contains the file pattern.\")\n    patterns = pcoll | beam.Map(lambda x: str(getattr(x, pattern_field)))\n  else:\n    patterns = pcoll\n\n  matched = patterns | beam.io.fileio.MatchAll(\n      empty_match_treatment=empty_match_treatment)\n\n  return matched | beam.Map(\n      lambda x: beam.Row(\n          path=str(x.path), size_in_bytes=int(x.size_in_bytes),\n          last_updated_in_seconds=float(x.last_updated_in_seconds)\n          if x.last_updated_in_seconds is not None else None))\n\n\n_DICOM_SEARCH_OUTPUT_SCHEMA = RowTypeConstraint.from_fields([","sourceCodeStart":879,"sourceCodeEnd":915,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/yaml/yaml_io.py#L879-L915","documentation":"Thrown by match_all in apache_beam/yaml/yaml_io.py when the input PCollection's schema has more than one field and no 'file_pattern' parameter was given, so the transform cannot tell which field holds the file path pattern to match. The library only auto-selects the pattern field when the schema has exactly one field.","triggerScenarios":"Calling match_all (or the YAML MatchAll transform) on a PCollection whose element schema has 2+ fields while omitting the file_pattern parameter.","commonSituations":"YAML pipeline authors pipe rows from a previous transform (e.g. a database read or ParseTransform) into MatchAll without realizing the rows have multiple columns; forgetting the file_pattern after changing an upstream schema to add extra fields.","solutions":["Set the file_pattern parameter to the name of the schema field containing the file path pattern.","Check pcoll.element_type._fields to see the available field names and pick the right one.","If only the path matters upstream, project the PCollection to a single-field schema before match_all."],"exampleFix":"// before\nout = yaml_match_all.transform(pcoll, {})  # pcoll schema: (path, size)\n// after\nout = yaml_match_all.transform(pcoll, {'file_pattern': 'path'})","handlingStrategy":"validation","validationCode":"fields = list(getattr(pcoll.element_type, '_fields', []))\nassert fields, 'input has no schema'\nif len(fields) > 1 and not options.get('file_pattern'):\n    raise ValueError(f\"must pass file_pattern; fields={fields}\")","typeGuard":"def has_single_schema_field(pcoll):\n    return len(getattr(pcoll.element_type, '_fields', [])) == 1","tryCatchPattern":null,"preventionTips":["Always pass file_pattern explicitly when the input schema could grow fields","Log pcoll.element_type._fields during pipeline development","Project to a single-field PCollection before MatchAll"],"tags":["apache-beam","yaml","configuration","schema"],"backgroundTag":"missing-required-argument","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}