pentaho/pentaho-kettle · error · KettleStepException

ConcatFields.Error.RemainingFieldNotFoundInputStream

ConcatFields.Error.RemainingFieldNotFoundInputStream

Error message

ConcatFields.Error.RemainingFieldNotFoundInputStream

What it means

When ConcatFields maps remaining (non-concatenated) fields from input to output, prepareForReMap looks up each output field name in the modified input row metadata. A -1 result means one of the remaining fields is missing from the input stream, so it throws this exception naming the field.

Solutions

  1. Ensure the input stream supplies every non-target field expected in the output row.
  2. Remove or rename stale fields in the transformation design.
  3. Re-verify hop field layouts after upstream changes and re-save the transformation.
Defensive patterns

Strategy: validation

Validate before calling

String[] names = outputRowMeta.getFieldNames();
for ( int i = 0; i < names.length - 1; i++ ) {
  if ( inputRowMetaModified.indexOfValue( names[i] ) < 0 ) throw new IllegalArgumentException( "Missing field: " + names[i] );
}

Try / catch

try {
  step.processRow();
} catch ( KettleStepException e ) {
  if ( e.getMessage().contains( "RemainingFieldNotFoundInputStream" ) ) {
    logError( "Input stream lacks a designed output field: " + e.getMessage() );
  }
}

Prevention

When it happens

Trigger: Called from processRow when output row metadata contains fields (other than the new target field) that cannot be found in data.inputRowMetaModified.

Common situations: A field present in the output layout is not produced by the input stream — e.g., transformation edited so an upstream step no longer emits the field; metadata drift between designed and actual row layouts.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

AI-assisted analysis of pentaho/pentaho-kettle@f3058517a1 (2026-09-13). Data as JSON: /api/errors/cd4f3858ae76eb59. Report an issue: GitHub.

Appendix: source

Thrown at plugins/core/impl/src/main/java/org/pentaho/di/trans/steps/concatfields/ConcatFields.java:189

      if ( !meta.isFileNameInField() && meta.isHeaderEnabled() && data.outputRowMeta != null ) {
        writeHeader();
        // add an empty line for the header
        Object[] row = new Object[ data.outputRowMeta.size() ];
        putRowFromStream( row );
      }
    }
  }

  void prepareForReMap() throws KettleStepException {
    // prepare for re-map when removeSelectedFields
    if ( meta.isRemoveSelectedFields() ) {
      data.remainingFieldsInputOutputMapping = new int[ data.outputRowMeta.size() - 1 ]; // -1: don't need the new
      // target field
      String[] fieldNames = data.outputRowMeta.getFieldNames();
      for ( int i = 0; i < fieldNames.length - 1; i++ ) { // -1: don't search the new target field
        data.remainingFieldsInputOutputMapping[ i ] = data.inputRowMetaModified.indexOfValue( fieldNames[ i ] );
        if ( data.remainingFieldsInputOutputMapping[ i ] < 0 ) {
          throw new KettleStepException( BaseMessages.getString( PKG,
            "ConcatFields.Error.RemainingFieldNotFoundInputStream", "" + fieldNames[ i ] ) );
        }
      }
    }
  }

  // reads the row from the stream, flushs, add target field and call putRow()
  Object[] putRowFromStream( Object[] r ) throws KettleStepException {

    byte[] targetBinary = ( (ConcatFieldsOutputStream) data.writer ).read();
    if ( r == null && targetBinary == null ) {
      return null; // special condition of header/footer/split
    }

    Object[] outputRowData = prepareOutputRow( r );

    // add target field
    if ( outputRowData == null ) { // special condition of header/footer/split

View on GitHub (pinned to f3058517a1)