pentaho/pentaho-kettle · error · KettleException

Field name [ ] couldn't be found in the input stream!

Error message

Field name [<name>] couldn't be found in the input stream!

What it means

AvroOutput.processRow throws this KettleException when one of the step's configured output fields cannot be located in the incoming row stream. It looks up each configured output field by PentahoFieldName in getInputRowMeta(); indexOfValue() returning -1 means the field is absent, and the step aborts with 'Field name [<name>] couldn't be found in the input stream!'. This is a configuration/stream mismatch, not an Avro library failure.

Solutions

  1. Open the Avro Output step's Fields tab and correct the field name to exactly match the incoming stream (use 'Get Fields').
  2. Check upstream steps (Select values, Filter, Calculator) for renames/removals and restore or rename the field accordingly.
  3. Run with preview on the incoming hop to see the actual row metadata and compare against the configured output fields.
  4. If the field may be absent, add a step upstream that guarantees the field (e.g. a default value step).

Example fix

// before (configured)
outputFields.add("cust_id"); // upstream renamed it
// after
outputFields.add("customer_id"); // matches upstream stream field
Defensive patterns

Strategy: validation

Validate before calling

// Before running, verify each configured output field exists in the incoming row meta
RowMetaInterface in = getInputRowMeta(); // from a preview/debug hook or upstream step
for (AvroOutputField f : meta.getOutputFields()) {
  if (in.indexOfValue(f.getPentahoFieldName()) < 0) {
    throw new IllegalStateException("Missing input field: " + f.getPentahoFieldName());
  }
}

Try / catch

try {
  processRow();
} catch (KettleException e) {
  if (e.getMessage() != null && e.getMessage().contains("couldn't be found in the input stream")) {
    logError("Fix the Avro Output field mapping; stream fields: " + getInputRowMeta().getFieldNames());
  }
  setErrors(1); stopAll();
}

Prevention

When it happens

Trigger: The Avro Output step's 'fields' table (meta.getOutputFields()) lists a Pentaho field name that the previous step did not emit, or emitted under a different name/case; the hop feeding the step changed upstream (renamed, removed, or conditional field) after the step was configured.

Common situations: Renaming or deleting a field in an upstream step (Select values, Calculator) without updating the Avro Output field mapping; a stream that produces the field only on some paths; copy/pasted transformation where field names differ between environments; case-sensitivity mismatches.

Related errors

AI-assisted analysis of pentaho/pentaho-kettle@f3058517a1 (2026-09-13). Data as JSON: /api/errors/7387659be988f969. Report an issue: GitHub.

Appendix: source

Thrown at plugins/avro-format/core/src/main/java/org/pentaho/di/trans/steps/avro/output/AvroOutput.java:78

          String error = e.getMessage().replaceAll( "TRANS_NAME", getTrans().getName() );
          error = error.replaceAll( "STEP_NAME", getStepname() );
          getLogChannel().logError( error );
          setErrors( 1 );
          setOutputDone();
          return false;
        }
      }

      Object[] currentRow = getRow();
      if ( currentRow != null ) {
        //create new outputMeta
        RowMetaInterface outputRMI = new RowMeta();
        //create data equals with output fileds
        Object[] outputData = new Object[ meta.getOutputFields().size() ];
        for ( int i = 0; i < meta.getOutputFields().size(); i++ ) {
          int inputRowIndex = getInputRowMeta().indexOfValue( meta.getOutputFields().get( i ).getPentahoFieldName() );
          if ( inputRowIndex == -1 ) {
            throw new KettleException( "Field name [" + meta.getOutputFields().get( i ).getPentahoFieldName()
              + " ] couldn't be found in the input stream!" );
          } else {
            ValueMetaInterface vmi = ValueMetaFactory.cloneValueMeta( getInputRowMeta().getValueMeta( inputRowIndex ) );
            //add output value meta according output fields
            outputRMI.addValueMeta( i, vmi );
            //add output data according output fields
            outputData[ i ] = currentRow[ inputRowIndex ];
          }
        }
        RowMetaAndData row = new RowMetaAndData( outputRMI, outputData );
        data.writer.write( row );
        putRow( row.getRowMeta(), row.getData() );
        return true;
      } else {
        // no more input to be expected...
        closeWriter();
        setOutputDone();
        return false;

View on GitHub (pinned to f3058517a1)