apache/beam · error · IllegalArgumentException

Unable to parse schema

Error message

Unable to parse schema {}

What it means

JsonUtils.beamSchemaFromJsonSchema builds a Beam Schema from a JSON Schema. Beam requires a deterministic field ordering, so required properties are processed first from jsonSchema.getRequiredProperties(); if a name listed as required has no corresponding entry in the properties map, this IllegalArgumentException is thrown. The JSON Schema is inconsistent (required references an undefined property).

Solutions

1. Remove the undefined name from the JSON Schema's 'required' array, or add a matching 'properties' entry
2. Validate the JSON Schema (required ⊆ properties) before passing it to Beam
3. Regenerate the schema from a single source so required and properties stay in sync

Example fix

// before
{"type":"object","required":["a","b"],"properties":{"a":{"type":"string"}}}
// after
{"type":"object","required":["a"],"properties":{"a":{"type":"string"}}}

static void validateRequiredSubset(JSONObject jsonSchema) {
  JSONObject props = jsonSchema.getJSONObject("properties");
  for (Object req : jsonSchema.optJSONArray("required")) {
    if (!props.has((String) req))
      throw new IllegalArgumentException("required property '" + req + "' missing from properties");
  }
}

static boolean requiredSubsetOfProperties(JSONObject schema) {
  JSONObject props = schema.optJSONObject("properties");
  if (props == null) return schema.optJSONArray("required") == null;
  java.util.stream.StreamSupport.stream(
      java.util.Spliterators.spliteratorUnknownSize(schema.optJSONArray("required")==null?java.util.Collections.emptyListIterator():schema.optJSONArray("required").iterator(),0),false)
      .allMatch(r -> props.has((String) r));
  return true;
}

try {
  Schema s = JsonUtils.beamSchemaFromJsonSchema(jsonSchema);
} catch (IllegalArgumentException e) {
  throw new SchemaParseException("Invalid JSON Schema (required vs properties mismatch): " + e.getMessage(), e);
}

When it happens

Trigger: Parsing a JSON Schema whose 'required' array contains a property name absent from 'properties' — via JsonUtils.beamSchemaFromJsonSchema / fromJsonSchema APIs.

Common situations: Hand-written or tool-generated JSON Schemas with a required field listed but never defined; schema evolution where a property was removed from 'properties' but not from 'required'.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• Array schema is not properly formatted or unsupported
• Tuple-like arrays are unsupported. Expected a single item…
• Unsupported Beam to JSON type
• Unsupported field type
• " " argument must be specified.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/1b470122e13d8787. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/utils/JsonUtils.java:218

  public static Schema beamSchemaFromJsonSchema(String jsonSchemaStr) {
    org.everit.json.schema.ObjectSchema jsonSchema = jsonSchemaFromString(jsonSchemaStr);
    return beamSchemaFromJsonSchema(jsonSchema);
  }

  private static Schema beamSchemaFromJsonSchema(org.everit.json.schema.ObjectSchema jsonSchema) {
    Schema.Builder beamSchemaBuilder = Schema.builder();
    Map<String, org.everit.json.schema.Schema> properties =
        new HashMap<>(jsonSchema.getPropertySchemas());
    // Properties in a JSON Schema are stored in a Map object and unfortunately don't maintain
    // order. However, the schema's required properties is a list of property names that is
    // consistent and is in the same order as when the schema was first created. To create a
    // consistent Beam Schema from the same JSON schema, we add Schema Fields following this order.
    // We can guarantee a consistent Beam schema when all JSON properties are required.
    for (String propertyName : jsonSchema.getRequiredProperties()) {
      org.everit.json.schema.Schema propertySchema = properties.get(propertyName);
      if (propertySchema == null) {
        throw new IllegalArgumentException("Unable to parse schema " + jsonSchema);
      }

      Boolean isNullable =
          Boolean.TRUE.equals(propertySchema.getUnprocessedProperties().get("nullable"));
      beamSchemaBuilder =
          addPropertySchemaToBeamSchema(
              propertyName, propertySchema, beamSchemaBuilder, isNullable);
      // Remove properties we already added.
      properties.remove(propertyName, propertySchema);
    }

    // Now we are potentially left with properties that are not required. Add them too.
    // Note: having more than one non-required properties may result in  inconsistent
    // Beam schema field orderings.
    for (Map.Entry<String, org.everit.json.schema.Schema> entry : properties.entrySet()) {
      String propertyName = entry.getKey();
      org.everit.json.schema.Schema propertySchema = entry.getValue();

View on GitHub (pinned to 12126d8942)