apache/beam · error · ValueError

Incompatible types: map vs object

Error message

Incompatible types: map vs object

What it means

When the weak schema is of type 'object' but allows arbitrary keys (additionalProperties set, i.e. a map), the strong schema must also allow arbitrary keys. If strong_schema has additionalProperties falsy (e.g. False or an empty spec), the validator raises 'Incompatible types: map vs object'.

Source

Thrown at sdks/python/apache_beam/yaml/json_utils.py:325

  convert = row_to_json(
      schema_pb2.FieldType(row_type=schema_pb2.RowType(schema=beam_schema)))
  return lambda row: json.dumps(convert(row), sort_keys=True).encode('utf-8')


def _validate_compatible(weak_schema, strong_schema):
  if not weak_schema:
    return
  if weak_schema['type'] != strong_schema['type']:
    raise ValueError(
        f"Incompatible types: {weak_schema['type']} vs {strong_schema['type']}")
  if weak_schema['type'] == 'array':
    _validate_compatible(weak_schema['items'], strong_schema['items'])
  elif weak_schema['type'] == 'object':
    # If the weak schema allows for arbitrary keys (is a map),
    # the strong schema must also allow for arbitrary keys.
    if weak_schema.get('additionalProperties'):
      if not strong_schema.get('additionalProperties', True):
        raise ValueError('Incompatible types: map vs object')
      _validate_compatible(
          weak_schema['additionalProperties'],
          strong_schema['additionalProperties'])
    for required in strong_schema.get('required', []):
      if required not in weak_schema['properties']:
        raise ValueError(f"Missing or unknown property '{required}'")
    for name, spec in weak_schema.get('properties', {}).items():

      if name in strong_schema['properties']:
        try:
          _validate_compatible(spec, strong_schema['properties'][name])
        except Exception as exn:
          raise ValueError(f"Incompatible schema for '{name}'") from exn
      elif not strong_schema.get('additionalProperties', True):
        # The property is not explicitly in the strong schema, and the strong
        # schema does not allow for extra properties.
        raise ValueError(
            f"Prohibited property: '{name}'; "

View on GitHub (pinned to 12126d8942)

Solutions

1. Set additionalProperties in the strong schema (to the value schema or true) so a map is accepted.
2. Alternatively change the source so it produces a fixed-shape object matching the strong schema's properties instead of an arbitrary map.
3. If strictness is required, convert the Beam side from a map type to a Row/struct with explicit fields.
4. Split validation: validate map values against a separate schema rather than through this object-vs-object path.

Example fix

# before
strong = {'type': 'object', 'properties': {'a': {'type': 'string'}}, 'additionalProperties': False}
# after
strong = {'type': 'object', 'additionalProperties': {'type': 'string'}}

def map_compatible(weak, strong):
    if weak.get('type') != 'object':
        return True
    if weak.get('additionalProperties'):
        return bool(strong.get('additionalProperties', True))
    return True
assert map_compatible(weak_schema, strong_schema)

def is_map_schema(s):
    return isinstance(s, dict) and s.get('type') == 'object' and bool(s.get('additionalProperties'))

try:
    row_validator(beam_schema, json_schema)
except ValueError as e:
    if str(e) == 'Incompatible types: map vs object':
        raise ValueError('Source emits a map; consumer schema must allow additionalProperties') from e
    raise

When it happens

Trigger: row_validator with weak_schema = {'type': 'object', 'additionalProperties': <schema>} and strong_schema = {'type': 'object', 'additionalProperties': False} (or missing property-restricted schema) for the same position.

Common situations: Beam map<str, X> fields being validated against a strict JSON object schema with enumerated properties; tightening a previously open schema; YAML map fields declared with fixed property lists.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• Incompatible types: {weak_schema['type']} vs {strong_schema[
• Missing or unknown property '{required}'
• Incompatible schema for '{name}'
• Prohibited property: '{name}'; perhaps additionalProperties:
• Node ID cannot be empty

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/ae2c5c71ae3bec59. Report an issue: GitHub.