pentaho/pentaho-kettle · error · KettleException

AvroInput.Error.UnexpectedArrayElementTypeAtNonExpansionPoin…

Error message

AvroInput.Error.UnexpectedArrayElementTypeAtNonExpansionPoint

What it means

Mirror of the map error for arrays: while resolving an array path at a non-expansion point, the element's schema type is an array where a non-primitive sub-structure (record) was expected to continue traversal.

Solutions

  1. Add an index for every array level: 'matrix[0][1]'
  2. Flatten nested arrays in the schema or pre-process the data
  3. Enable 'Ignore missing paths' for heterogeneous schema versions
  4. Define paths per schema version when nesting differs

Example fix

// before
path: matrix[0].value   // matrix is array<array<int>>
// after
path: matrix[0][1]
Defensive patterns

Strategy: validation

Validate before calling

// Count array nesting depth in schema and require an index per level
int depth = 0;
Schema s = schema.getField("matrix").schema();
while (s.getType() == Schema.Type.ARRAY) { depth++; s = s.getElementType(); }
long indexesInPath = path.chars().filter(c -> c == '[').count();
if (indexesInPath < depth) {
  throw new IllegalArgumentException("Need " + depth + " indexes, path has " + indexesInPath);
}

Try / catch

try {
  return reader.convertArray(...);
} catch (KettleException e) {
  if (e.getMessage().contains("UnexpectedArrayElementTypeAtNonExpansionPoint")) {
    return emptyRow();
  }
  throw e;
}

Prevention

When it happens

Trigger: Nested arrays (array of arrays) with a path that does not supply an index at the inner level, e.g. 'matrix[0]' where matrix is array<array<int>> and the path expects 'matrix[0][1]'; top-level union or per-row schema switching producing nested arrays.

Common situations: Nested-array schemas where the user forgot the second index; schema evolution wrapped an array inside another array; union branches with different array nesting depths.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of pentaho/pentaho-kettle@f3058517a1 (2026-09-13). Data as JSON: /api/errors/ccbf65455afe1459. Report an issue: GitHub.

Appendix: source

Thrown at plugins/avro-format/core/src/main/java/org/pentaho/di/trans/steps/avro/input/AvroNestedReader.java:1291

          }
        }

        // what have we got?
        if ( elementType.getType() == Schema.Type.RECORD ) {
          return convertToKettleValues( (GenericData.Record) value, elementType, defaultSchema, space, ignoreMissing );
        } else if ( elementType.getType() == Schema.Type.ARRAY ) {
          return convertToKettleValues( (GenericData.Array) value, elementType, defaultSchema, space, ignoreMissing );
        } else if ( elementType.getType() == Schema.Type.MAP ) {
          return convertToKettleValues( (Map<Utf8, Object>) value, elementType, defaultSchema, space, ignoreMissing );
        } else {
          // we shouldn't have a primitive at this point. If we are
          // extracting a particular index from the array then we're not to the
          // expansion phase,
          // so normally there must be a non-primitive sub-structure. Only if
          // the user is switching schema versions on a per-row basis or the
          // schema is a union at the top level could we end up here
          if ( !ignoreMissing ) {
            throw new KettleException( BaseMessages.getString( PKG,
              "AvroInput.Error.UnexpectedArrayElementTypeAtNonExpansionPoint" ) );
          } else {
            Object[][] result = new Object[ 1 ][ m_outputRowMeta.size() + RowDataUtil.OVER_ALLOCATE_SIZE ];
            return result;
          }
        }
      }
    }

    /**
     * Processes a record at this point in the path.
     *
     * @param record        the record to process
     * @param s             the current schema at this point in the path
     * @param space         environment variables
     * @param ignoreMissing true if null is to be returned for user fields that don't appear in the schema
     * @return an array of Kettle rows corresponding to the expanded map/array and containing all leaf values as defined
     * in the paths

View on GitHub (pinned to f3058517a1)