apache/beam · error · ValueError

Missing properties for

Error message

Missing properties for {json_schema}.

What it means

json_schema_to_beam_schema rejects object JSON schemas that lack a 'properties' key. Although such a schema is technically valid JSON Schema (vacuously), it cannot be turned into a meaningful Beam row schema, so an informative ValueError is raised instead of producing an empty schema.

Solutions

1. Add a 'properties' mapping listing the row's fields and their types
2. If the data is a free-form map, model it with "additionalProperties" (via json_type_to_beam_type) instead of an empty object
3. Validate the schema JSON before feeding it to the pipeline

Example fix

// before
{"type": "object"}
// after
{"type": "object", "properties": {"id": {"type": "integer"}, "name": {"type": "string"}}}

def assert_properties_present(json_schema):
    if json_schema.get('type') == 'object' and 'properties' not in json_schema:
        raise ValueError('Object schema must define properties (or use additionalProperties at nested level)')

def has_properties(s) -> bool:
    return isinstance(s, dict) and 'properties' in s

try:
    beam_schema = json_schema_to_beam_schema(json_schema)
except ValueError as e:
    if 'Missing properties' in str(e):
        raise ValueError(f'Schema {json_schema!r} needs a properties mapping') from e
    raise

When it happens

Trigger: Passing {"type":"object"} with no 'properties' (and no intent to use additionalProperties) to json_schema_to_beam_schema, e.g. an empty placeholder schema in a yaml pipeline config.

Common situations: Hand-written pipeline YAML where the json_schema option has type object but fields were never filled in; schemas generated from empty records; trimming fields to test and removing properties entirely.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

• Expected object type, got
• Object type must have either properties or…
• A schema is required to write non-schema'd data.
• All dicts in batch must have the same keys. extra keys
• An explicit schema is required to write non-schema'd…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/861a5422a3d00360. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/yaml/json_utils.py:71

}


def json_schema_to_beam_schema(
    json_schema: dict[str, Any]) -> schema_pb2.Schema:
  """Returns a Beam schema equivalent for the given Json schema."""
  def maybe_nullable(beam_type, nullable):
    if nullable:
      beam_type.nullable = True
    return beam_type

  json_type = json_schema.get('type', None)
  if json_type != 'object':
    raise ValueError(f'Expected object type, got {json_type}.')
  if 'properties' not in json_schema:
    # Technically this is a valid (vacuous) schema, but as it's not generally
    # meaningful, throw an informative error instead.
    # (We could add a flag to allow this degenerate case.)
    raise ValueError('Missing properties for {json_schema}.')
  required = set(json_schema.get('required', []))
  return schema_pb2.Schema(
      fields=[
          schemas.schema_field(
              name,
              maybe_nullable(json_type_to_beam_type(t), name not in required),
              description=t.get('description') if isinstance(t, dict) else None)
          for (name, t) in json_schema['properties'].items()
      ])


def json_type_to_beam_type(json_type: dict[str, Any]) -> schema_pb2.FieldType:
  """Returns a Beam schema type for the given Json (schema) type."""
  if not isinstance(json_type, dict) or 'type' not in json_type:
    raise ValueError(f'Malformed type {json_type}.')
  type_name = json_type['type']
  if type_name in JSON_ATOMIC_TYPES_TO_BEAM:
    return schema_pb2.FieldType(

View on GitHub (pinned to 12126d8942)