{"record":{"id":"04e9572f221a1a0e","repo":"apache/beam","slug":"transform-s-was-configured-with-unknown-fields-s-valid","errorCode":null,"errorMessage":"Transform '%s' was configured with unknown fields: %s. Valid fields: %s","messagePattern":"Transform '(.+?)' was configured with unknown fields: (.+?)\\. Valid fields: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/transforms/external.py","lineNumber":267,"sourceCode":"      elif type_info == 'array_type':\n        return [\n            dict_to_row_recursive(field_type.array_type.element_type, value)\n            for value in py_value\n        ]\n      elif type_info == 'map_type':\n        return {\n            key: dict_to_row_recursive(field_type.map_type.value_type, value)\n            for key, value in py_value.items()\n        }\n      else:\n        return py_value\n\n    def dict_to_row(schema_proto, py_value):\n      row_type = named_tuple_from_schema(schema_proto)\n      if isinstance(py_value, dict):\n        extra = set(py_value.keys()) - set(row_type._fields)\n        if extra:\n          raise ValueError(\n              f\"Transform '{self.identifier()}' was configured with unknown \"\n              f\"fields: {extra}. Valid fields: {set(row_type._fields)}\")\n        return row_type(\n            *[\n                dict_to_row_recursive(\n                    field.type, py_value.get(field.name, None))\n                for field in schema_proto.fields\n            ])\n      else:\n        return row_type(py_value)\n\n    return external_transforms_pb2.SchemaTransformPayload(\n        identifier=self._identifier,\n        configuration_schema=self._schema_proto,\n        configuration_row=RowCoder(self._schema_proto).encode(\n            dict_to_row(self._schema_proto, self._kwargs)))\n\n","sourceCodeStart":249,"sourceCodeEnd":285,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/transforms/external.py#L249-L285","documentation":"When converting a Python dict configuration into a schema Row for a cross-language transform, Beam validates that every key matches a field of the transform's declared schema. Unknown keys (which may be misspellings or from a changed schema) raise this ValueError listing the extra keys and the valid field names, prefixed with the transform's identifier.","triggerScenarios":"Passing a nested dict to a SchemaTransform build() (or dict_to_row) containing keys not present in the transform's schema, e.g. after renaming a field in a newer provider version.","commonSituations":"Upgrading a Beam expansion service whose transform schema changed; typos in config keys; hand-edited YAML/JSON configs converted to kwargs.","solutions":["Remove or rename the offending keys to match the valid fields listed in the error message.","Check the transform's documentation for current field names after upgrading Beam or the expansion service.","Validate config dicts against the schema (via named_tuple_from_schema) before calling build()."],"exampleFix":"# before\nbuilder.build(jar_path='x.jar', csv_file='data.csv')  # 'csv_file' unknown\n# after\nbuilder.build(jar_path='x.jar', file_pattern='data.csv')  # matches schema field","handlingStrategy":"validation","validationCode":"from apache_beam.typehints.schemas import named_tuple_from_schema\nrow_type = named_tuple_from_schema(schema_proto)\nextra = set(config.keys()) - set(row_type._fields)\nif extra:\n    raise ValueError('unknown fields: %r' % extra)","typeGuard":"def matches_schema(config, schema_proto) -> bool:\n    fields = set(named_tuple_from_schema(schema_proto)._fields)\n    return set(config.keys()) <= fields","tryCatchPattern":"try:\n    row = builder.build(**config)\nexcept ValueError as e:\n    if 'unknown fields' in str(e):\n        config = {k: v for k, v in config.items() if k in known_fields}\n        row = builder.build(**config)\n    else:\n        raise","preventionTips":["Keep config keys in sync with the transform schema version you deploy.","Validate nested config dicts against the schema before build().","After Beam/expansion-service upgrades, re-check field names in the error message's 'Valid fields' list."],"tags":["python","apache-beam","cross-language","schema"],"backgroundTag":"schema-validation-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}