pentaho/pentaho-kettle · error · KettleException

AvroInput.Error.NonExistentField

Error message

AvroInput.Error.NonExistentField

What it means

Kettle exception thrown by convertToKettleValue when the current path part names a field that does not exist in the Avro record's schema (Schema.getField(part) returns null) and ignoreMissing is false. The configured path references a field absent from the data's schema.

Solutions

  1. Correct the field name in the step's path so it matches the Avro schema exactly (names are case-sensitive).
  2. Enable 'Ignore missing fields' in the Avro Input step if nulls are acceptable for absent fields.
  3. Refresh the configured fields from the current file's schema; the schema may have changed (schema evolution).
  4. If schemas vary by file, align the reader/writer schemas or use the default schema mechanism to supply fallback values.

Example fix

// before
Field path: order.CreateDate   // field renamed
// after
Field path: order.created_date
Defensive patterns

Strategy: validation

Validate before calling

// Check the path's leaf field exists in the file's schema before processing
Schema.Field f = recordSchema.getField(lastPathPart);
if (f == null && !ignoreMissing) {
  throw new IllegalArgumentException("Field '" + lastPathPart + "' not in schema; refresh field definitions");
}

Try / catch

try {
  value = reader.convertToKettleValue(record, schema, field);
} catch (KettleException e) {
  if (e.getMessage().contains("NonExistentField")) {
    logBasic("Field missing in current file schema; emitting null (or enable Ignore Missing)");
    value = null;
  }
}

Prevention

When it happens

Trigger: convertToKettleValue reaches a record and pops path part 'part'; record schema has no field with that name and the step is not configured to ignore missing fields.

Common situations: Schema evolution: the writer schema dropped or renamed a field after the transformation was configured; typos in the path (case-sensitive names); reading Avro files from a different producer than expected; field name differences between Avro write and read schemas.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

AI-assisted analysis of pentaho/pentaho-kettle@f3058517a1 (2026-09-13). Data as JSON: /api/errors/464b8b6a9a1d05d4. Report an issue: GitHub.

Appendix: source

Thrown at plugins/avro-format/core/src/main/java/org/pentaho/di/trans/steps/avro/input/AvroNestedReader.java:657

    String part = avroInputField.getTempParts().remove( 0 );
    if ( part.charAt( 0 ) == '[' ) {
      throw new KettleException(
        BaseMessages.getString( PKG, "AvroInput.Error.InvalidPath" ) + avroInputField.getTempParts() );
    }

    if ( part.indexOf( '[' ) > 0 ) {
      String arrayPart = part.substring( part.indexOf( '[' ) );
      part = part.substring( 0, part.indexOf( '[' ) );

      // put the array section back into location zero
      avroInputField.getTempParts().add( 0, arrayPart );
    }

    // part is a named field of the record
    Schema.Field fieldS = s.getField( part );
    if ( fieldS == null && !ignoreMissing ) {
      throw new KettleException( BaseMessages.getString( PKG, "AvroInput.Error.NonExistentField", part ) );
    }
    Object field = record.get( part );

    if ( field == null ) {
      if ( defaultSchema != null ) {
        fieldS = defaultSchema.getField( part );
      }

      if ( fieldS == null || fieldS.defaultVal() == null ) {
        return null;
      }
      field = fieldS.defaultVal();
    }

    Schema.Type fieldT = fieldS.schema().getType();
    Schema fieldSchema = fieldS.schema();

    if ( fieldT == Schema.Type.UNION ) {

View on GitHub (pinned to f3058517a1)