apache/beam · error · ValueError

Incompatible types: {weak_schema['type']} vs {strong_schema[

Error message

Incompatible types: {weak_schema['type']} vs {strong_schema['type']}

What it means

_validate_compatible checks that a weak (loose) JSON schema is compatible with a strong schema. If the top-level 'type' values differ (e.g. 'string' vs 'object'), the rows cannot be validated against the schema and this ValueError is thrown.

Source

Thrown at sdks/python/apache_beam/yaml/json_utils.py:316

  elif type_info == "logical_type":
    return lambda value: value
  else:
    raise ValueError(f"Unrecognized type_info: {type_info!r}")


def json_formater(
    beam_schema: schema_pb2.Schema) -> Callable[[beam.Row], bytes]:
  """Returns a callable converting rows of the given schema to Json strings."""
  convert = row_to_json(
      schema_pb2.FieldType(row_type=schema_pb2.RowType(schema=beam_schema)))
  return lambda row: json.dumps(convert(row), sort_keys=True).encode('utf-8')


def _validate_compatible(weak_schema, strong_schema):
  if not weak_schema:
    return
  if weak_schema['type'] != strong_schema['type']:
    raise ValueError(
        f"Incompatible types: {weak_schema['type']} vs {strong_schema['type']}")
  if weak_schema['type'] == 'array':
    _validate_compatible(weak_schema['items'], strong_schema['items'])
  elif weak_schema['type'] == 'object':
    # If the weak schema allows for arbitrary keys (is a map),
    # the strong schema must also allow for arbitrary keys.
    if weak_schema.get('additionalProperties'):
      if not strong_schema.get('additionalProperties', True):
        raise ValueError('Incompatible types: map vs object')
      _validate_compatible(
          weak_schema['additionalProperties'],
          strong_schema['additionalProperties'])
    for required in strong_schema.get('required', []):
      if required not in weak_schema['properties']:
        raise ValueError(f"Missing or unknown property '{required}'")
    for name, spec in weak_schema.get('properties', {}).items():

      if name in strong_schema['properties']:

View on GitHub (pinned to 12126d8942)

Solutions

1. Align the 'type' in the JSON schema with the actual Beam schema type for the element.
2. Update the pipeline declaration (or transform output) so both sides describe the same shape.
3. If a field intentionally changed type, update all downstream schema declarations and tests together.
4. Wrap row_validator usage in try/except ValueError to surface which schema pair mismatches before running the pipeline.

Example fix

# before
weak = {'type': 'string'}
strong = {'type': 'object', 'properties': {...}}
# after
weak = {'type': 'object', 'properties': {...}}

def types_compatible(weak, strong):
    return (not weak) or weak.get('type') == strong.get('type')
# call before row_validator:
assert types_compatible(weak_schema, strong_schema), weak_schema.get('type')

def is_object_schema(s):
    return isinstance(s, dict) and s.get('type') == 'object'

try:
    validator = row_validator(beam_schema, json_schema)
except ValueError as e:
    if 'Incompatible types' in str(e):
        log.error('Weak/strong schema type mismatch: %s', e)
    raise

When it happens

Trigger: Calling row_validator / _validate_compatible with two schemas where weak_schema['type'] != strong_schema['type'], e.g. declaring a YAML provider input as type: string while the Beam schema is an object.

Common situations: Mismatched yamlProvider output schema vs declared JSON schema in a Beam YAML pipeline; editing a pipeline so a field changed type without updating the schema; map vs object confusion at top level.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• Node ID cannot be empty
• Edge source and target cannot be empty
• Incompatible types: map vs object
• Missing or unknown property '{required}'
• Incompatible schema for '{name}'

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/ea642ce87216479f. Report an issue: GitHub.