{"record":{"id":"99b0b7506cd62696","repo":"apache/beam","slug":"f-mapping-destinations-missing-for-type-are-not-in-the","errorCode":null,"errorMessage":"f\"Mapping destinations {missing} for {type} are not in the underlying config schema {list(underlying_schema_fields.keys())}\"","messagePattern":"f\"Mapping destinations (.+?) for (.+?) are not in the underlying config schema (.+?)\"","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/yaml/yaml_provider.py","lineNumber":1542,"sourceCode":"    return self._transforms.keys()\n\n  def config_schema(self, type):\n    underlying_schema = self._underlying_provider.config_schema(\n        self._transforms[type])\n    if underlying_schema is None:\n      return None\n    defaults = self._defaults.get(type, {})\n    underlying_schema_fields = {f.name: f for f in underlying_schema.fields}\n    missing = set(self._mappings[type].values()) - set(\n        underlying_schema_fields.keys())\n    if missing:\n      if 'kwargs' in underlying_schema_fields.keys():\n        # These are likely passed by keyword argument dict rather than missing.\n        for field_name in missing:\n          underlying_schema_fields[field_name] = schema_pb2.Field(\n              name=field_name, type=typing_to_runner_api(Any))\n      else:\n        raise ValueError(\n            f\"Mapping destinations {missing} for {type} are not in the \"\n            f\"underlying config schema {list(underlying_schema_fields.keys())}\")\n\n    def with_name(\n        original: schema_pb2.Field, new_name: str) -> schema_pb2.Field:\n      result = schema_pb2.Field()\n      result.CopyFrom(original)\n      result.name = new_name\n      return result\n\n    return schema_pb2.Schema(\n        fields=[\n            with_name(underlying_schema_fields[dest], src)\n            for (src, dest) in self._mappings[type].items()\n            if dest not in defaults\n        ])\n\n  def description(self, typ):","sourceCodeStart":1524,"sourceCodeEnd":1560,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/yaml/yaml_provider.py#L1524-L1560","documentation":"Raised by RenamingProvider while validating mapped config fields against the underlying transform's schema: after adding any kwargs-style pass-through, destinations that don't exist in the underlying schema are rejected. Mapped config names must correspond to fields the underlying transform actually accepts.","triggerScenarios":"RenamingProvider mappings rename a config field to a name the underlying transform's schema doesn't define (e.g. renaming 'url' to 'endpoint' for a transform expecting 'url'), and the underlying schema has no 'kwargs' field to absorb extras.","commonSituations":"Renaming config keys for readability while targeting a transform whose config schema was updated/renamed in a new Beam version, or simple typos in mapping destinations.","solutions":["Make the mapping destination match a real field of the underlying transform's config schema.","Update the mapping names after a Beam upgrade where the underlying transform's schema changed.","Check the error's listed schema fields and use one of them as the destination; or use an underlying transform that accepts a kwargs dict."],"exampleFix":"# before\nmappings:\n  MyWrite:\n    destination: WriteToText\n    config_mapping:\n      path: 'file_path'\n# after (if underlying schema uses 'file_path')\nmappings:\n  MyWrite:\n    destination: WriteToText\n    config_mapping:\n      file_path: 'file_path'","handlingStrategy":"validation","validationCode":"def validate_mapping_destinations(mappings, underlying_schema_fields):\n    missing = [d for d in mappings.get('config_mapping', {}) if d not in underlying_schema_fields and 'kwargs' not in underlying_schema_fields]\n    if missing:\n        raise ValueError(f'Destinations not in underlying schema: {missing}; valid: {list(underlying_schema_fields)}')","typeGuard":null,"tryCatchPattern":"try:\n    provider = RenamingProvider(base, transforms, mappings)\nexcept ValueError as e:\n    if 'not in the underlying config schema' in str(e):\n        logging.error('Rename destinations must exist in the underlying schema: %s', e)\n        raise\n    raise","preventionTips":["Inspect the underlying transform's config schema before renaming fields.","After Beam upgrades, re-check that mapped destination names still exist.","Keep destination names identical to source names when only renaming transform names, not config fields."],"tags":["python","apache-beam","yaml","schema"],"backgroundTag":"schema-validation-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}