pentaho/pentaho-kettle · error · KettleException

AvroInput.Error.UnableToFindSchemaForUnionMap

AvroInput.Error.UnableToFindSchemaForUnionMap

Error message

AvroInput.Error.UnableToFindSchemaForUnionMap

What it means

While resolving a MAP value whose schema is a UNION, convertToKettleValue scans the union's branches for a MAP-typed schema. If no branch is a MAP (and the remaining structure does not reduce to the two-element [type-marker, value] union case), the reader has no map schema to use and throws this KettleException.

Solutions

  1. Update the field path/Pentaho output type so the union is handled as its actual member type (e.g. string/long) instead of via map conversion.
  2. Change the Avro schema so the union includes the expected MAP branch (e.g. ["null", {"type":"map","values":"string"}]).
  3. Unwrap the union upstream (normalize data before the Avro Input step) so the field arriving is a plain map.

Example fix

// before
{"type": ["null", "string", "long"]}
// after (if map semantics intended)
{"type": ["null", {"type": "map", "values": ["string", "long"]}]}
Defensive patterns

Strategy: validation

Validate before calling

// Check that a union routed to map conversion contains a MAP branch
if (unionSchema.getType() == Schema.Type.UNION
    && unionSchema.getTypes().stream().noneMatch(s -> s.getType() == Schema.Type.MAP)) {
  throw new IllegalArgumentException("Union has no map branch: " + unionSchema);
}

Type guard

boolean unionHasMapBranch(Schema s) {
  return s.getType() == Schema.Type.UNION
      && s.getTypes().stream().anyMatch(t -> t.getType() == Schema.Type.MAP);
}

Try / catch

try {
  Object v = reader.convertToKettleValue(...);
} catch (KettleException e) {
  if (e.getMessage().contains("UnableToFindSchemaForUnionMap")) {
    // read the field as its concrete union member type instead
  } else throw e;
}

Prevention

When it happens

Trigger: An Avro field is a union such as ["null","string","long"] but the field path/step config routes it into the map-conversion branch, so the code expects a map member inside the union and finds none.

Common situations: Generic Avro unions (record|map combos replaced upstream by non-map types); schema evolution changed a map into a union of scalars while paths still assume map semantics.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of pentaho/pentaho-kettle@f3058517a1 (2026-09-13). Data as JSON: /api/errors/ac05e8f94d45a72d. Report an issue: GitHub.

Appendix: source

Thrown at plugins/avro-format/core/src/main/java/org/pentaho/di/trans/steps/avro/input/AvroNestedReader.java:483

    if ( valueType.getType() == Schema.Type.UNION ) {
      if ( value instanceof GenericContainer ) {
        // we can ask these things for their schema (covers
        // records, arrays, enums and fixed)
        valueType = ( (GenericContainer) value ).getSchema();
      } else {
        // either have a map or primitive here
        if ( value instanceof Map ) {
          // now have to look for the schema of the map
          Schema mapSchema = null;
          for ( Schema ts : valueType.getTypes() ) {
            if ( ts.getType() == Schema.Type.MAP ) {
              mapSchema = ts;
              break;
            }
          }
          if ( mapSchema == null ) {
            throw new KettleException( BaseMessages.getString( PKG,
              "AvroInput.Error.UnableToFindSchemaForUnionMap" ) );
          }
          valueType = mapSchema;
        } else {
          if ( avroInputField.getTempValueMeta().getType() != ValueMetaInterface.TYPE_STRING ) {
            // we have a two element union, where one element is the type
            // "null". So in this case we actually have just one type and can
            // output specific values of it (instead of using String as a
            // catch all for varying primitive types in the union)
            valueType = checkUnion( valueType );
          } else {
            // use the string representation of the value
            valueType = Schema.create( Schema.Type.STRING );
          }
        }
      }
    }

View on GitHub (pinned to f3058517a1)