{"record":{"id":"e0042889ddfd7caa","repo":"apache/beam","slug":"incompatible-schema-for-name","errorCode":null,"errorMessage":"Incompatible schema for '{name}'","messagePattern":"Incompatible schema for '(.+?)'","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/yaml/json_utils.py","lineNumber":338,"sourceCode":"  elif weak_schema['type'] == 'object':\n    # If the weak schema allows for arbitrary keys (is a map),\n    # the strong schema must also allow for arbitrary keys.\n    if weak_schema.get('additionalProperties'):\n      if not strong_schema.get('additionalProperties', True):\n        raise ValueError('Incompatible types: map vs object')\n      _validate_compatible(\n          weak_schema['additionalProperties'],\n          strong_schema['additionalProperties'])\n    for required in strong_schema.get('required', []):\n      if required not in weak_schema['properties']:\n        raise ValueError(f\"Missing or unknown property '{required}'\")\n    for name, spec in weak_schema.get('properties', {}).items():\n\n      if name in strong_schema['properties']:\n        try:\n          _validate_compatible(spec, strong_schema['properties'][name])\n        except Exception as exn:\n          raise ValueError(f\"Incompatible schema for '{name}'\") from exn\n      elif not strong_schema.get('additionalProperties', True):\n        # The property is not explicitly in the strong schema, and the strong\n        # schema does not allow for extra properties.\n        raise ValueError(\n            f\"Prohibited property: '{name}'; \"\n            \"perhaps additionalProperties: False is missing?\")\n\n\ndef row_validator(beam_schema: schema_pb2.Schema,\n                  json_schema: dict[str, Any]) -> Callable[[Any], Any]:\n  \"\"\"Returns a callable that will fail on elements not respecting json_schema.\n  \"\"\"\n  if not json_schema:\n    return lambda x: None\n\n  # Validate that this compiles, but avoid pickling the validator itself.\n  _ = jsonschema.validators.validator_for(json_schema)(json_schema)\n  _validate_compatible(beam_schema_to_json_schema(beam_schema), json_schema)","sourceCodeStart":320,"sourceCodeEnd":356,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/yaml/json_utils.py#L320-L356","documentation":"When comparing a property present in both weak and strong schemas, any incompatibility raised by the recursive _validate_compatible call is re-raised as \"Incompatible schema for '<name>'\" with the original error chained as __cause__, so the developer knows which named field failed and why.","triggerScenarios":"row_validator where for some property 'name' present in both schemas, _validate_compatible(spec_weak, spec_strong) raises (type mismatch, nested required missing, prohibited property, etc.) — the message wraps it.","commonSituations":"A nested field changed type (e.g. string to int) in one schema but not the other; nested required/properties inconsistencies; deeply nested schema drift in Beam YAML pipelines.","solutions":["Read the chained __cause__ (printed traceback shows 'The above exception was the direct cause...') to find the underlying nested mismatch.","Fix the named property's schema in either the weak or strong schema so types/requireds align.","Keep weak and strong schema definitions in sync, ideally deriving one from the other instead of duplicating them.","Validate schemas in a unit test (test_validate_compatible-style) before deploying the pipeline."],"exampleFix":"# before\nweak = {'type': 'object', 'properties': {'ts': {'type': 'string'}}}\nstrong = {'type': 'object', 'properties': {'ts': {'type': 'integer'}}}\n# after\nweak = {'type': 'object', 'properties': {'ts': {'type': 'integer'}}}","handlingStrategy":"try-catch","validationCode":"for name, spec in weak_schema.get('properties', {}).items():\n    if name in strong_schema['properties']:\n        try:\n            _validate_compatible(spec, strong_schema['properties'][name])\n        except Exception as exn:\n            print(f'property {name}: {exn!r}')","typeGuard":"def nested_schemas_share_shape(weak, strong, name):\n    return name in weak.get('properties', {}) and name in strong.get('properties', {})","tryCatchPattern":"try:\n    row_validator(beam_schema, json_schema)\nexcept ValueError as e:\n    if str(e).startswith('Incompatible schema for'):\n        log.error('%s | cause: %r', e, e.__cause__)\n    raise","preventionTips":["Always inspect e.__cause__ — this error wraps the real nested mismatch.","Validate nested schemas individually during development to localize failures.","Diff weak vs strong schema JSON in code review whenever either changes."],"tags":["python","apache-beam","yaml","json-schema","nested-schema"],"backgroundTag":"schema-validation-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}