{"record":{"id":"ea642ce87216479f","repo":"apache/beam","slug":"incompatible-types-weak-schema-type-vs-strong-schema-type","errorCode":null,"errorMessage":"Incompatible types: {weak_schema['type']} vs {strong_schema['type']}","messagePattern":"Incompatible types: (.+?) vs (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/yaml/json_utils.py","lineNumber":316,"sourceCode":"  elif type_info == \"logical_type\":\n    return lambda value: value\n  else:\n    raise ValueError(f\"Unrecognized type_info: {type_info!r}\")\n\n\ndef json_formater(\n    beam_schema: schema_pb2.Schema) -> Callable[[beam.Row], bytes]:\n  \"\"\"Returns a callable converting rows of the given schema to Json strings.\"\"\"\n  convert = row_to_json(\n      schema_pb2.FieldType(row_type=schema_pb2.RowType(schema=beam_schema)))\n  return lambda row: json.dumps(convert(row), sort_keys=True).encode('utf-8')\n\n\ndef _validate_compatible(weak_schema, strong_schema):\n  if not weak_schema:\n    return\n  if weak_schema['type'] != strong_schema['type']:\n    raise ValueError(\n        f\"Incompatible types: {weak_schema['type']} vs {strong_schema['type']}\")\n  if weak_schema['type'] == 'array':\n    _validate_compatible(weak_schema['items'], strong_schema['items'])\n  elif weak_schema['type'] == 'object':\n    # If the weak schema allows for arbitrary keys (is a map),\n    # the strong schema must also allow for arbitrary keys.\n    if weak_schema.get('additionalProperties'):\n      if not strong_schema.get('additionalProperties', True):\n        raise ValueError('Incompatible types: map vs object')\n      _validate_compatible(\n          weak_schema['additionalProperties'],\n          strong_schema['additionalProperties'])\n    for required in strong_schema.get('required', []):\n      if required not in weak_schema['properties']:\n        raise ValueError(f\"Missing or unknown property '{required}'\")\n    for name, spec in weak_schema.get('properties', {}).items():\n\n      if name in strong_schema['properties']:","sourceCodeStart":298,"sourceCodeEnd":334,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/yaml/json_utils.py#L298-L334","documentation":"_validate_compatible checks that a weak (loose) JSON schema is compatible with a strong schema. If the top-level 'type' values differ (e.g. 'string' vs 'object'), the rows cannot be validated against the schema and this ValueError is thrown.","triggerScenarios":"Calling row_validator / _validate_compatible with two schemas where weak_schema['type'] != strong_schema['type'], e.g. declaring a YAML provider input as type: string while the Beam schema is an object.","commonSituations":"Mismatched yamlProvider output schema vs declared JSON schema in a Beam YAML pipeline; editing a pipeline so a field changed type without updating the schema; map vs object confusion at top level.","solutions":["Align the 'type' in the JSON schema with the actual Beam schema type for the element.","Update the pipeline declaration (or transform output) so both sides describe the same shape.","If a field intentionally changed type, update all downstream schema declarations and tests together.","Wrap row_validator usage in try/except ValueError to surface which schema pair mismatches before running the pipeline."],"exampleFix":"# before\nweak = {'type': 'string'}\nstrong = {'type': 'object', 'properties': {...}}\n# after\nweak = {'type': 'object', 'properties': {...}}","handlingStrategy":"validation","validationCode":"def types_compatible(weak, strong):\n    return (not weak) or weak.get('type') == strong.get('type')\n# call before row_validator:\nassert types_compatible(weak_schema, strong_schema), weak_schema.get('type')","typeGuard":"def is_object_schema(s):\n    return isinstance(s, dict) and s.get('type') == 'object'","tryCatchPattern":"try:\n    validator = row_validator(beam_schema, json_schema)\nexcept ValueError as e:\n    if 'Incompatible types' in str(e):\n        log.error('Weak/strong schema type mismatch: %s', e)\n    raise","preventionTips":["Derive the JSON schema from the Beam schema instead of maintaining two hand-written copies.","Keep schema definitions co-located with the transform that produces the data.","Test schema compatibility in CI before deploying YAML pipelines."],"tags":["python","apache-beam","yaml","json-schema","validation"],"backgroundTag":"schema-validation-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}