pentaho/pentaho-kettle · error · KettleStepException

The lookup column '

Error message

The lookup column '

What it means

In lookupValues, the main-stream lookup key columns are resolved by name against the incoming row; if a lookup key field is missing the step throws 'The lookup column X could not be found'. The code comment notes 'we should not get here' — normally earlier validation catches it, so this is a defensive check for runtime metadata drift.

Solutions

1. Ensure the main-stream input actually contains every configured lookup key field at runtime
2. Re-run transformation metadata validation / re-open the step dialog to refresh field names
3. If calling the step programmatically, build the input RowMetaInterface with all key field names present
4. Trace upstream steps (Calculator, Filter, Select Values) that may conditionally drop the field

Example fix

// before (programmatic use)
RowMeta rowMeta = new RowMeta(); // missing 'id'
// after
rowMeta.addValueMeta(new ValueMetaInteger("id")); // include all key fields

// Pre-check main stream row metadata before invoking lookup
int idx = mainRowMeta.indexOfValue(keyFieldName);
if (idx < 0) throw new IllegalArgumentException("Main stream lacks lookup key: " + keyFieldName);

boolean mainStreamHasKey(RowMetaInterface rm, String key) { return rm.indexOfValue(key) >= 0; }

try { outputRow(inputRow); } catch (KettleStepException e) {
  if (e.getMessage().startsWith("The lookup column")) { /* fix main stream layout */ }
  throw e;
}

When it happens

Trigger: lookupValues (called from outputRow) builds data.lookupColumnIndex via rowMeta.indexOfValue(names[i]) and gets -1 for a main-stream key field.

Common situations: Main input stream shape changed at runtime (e.g. fields removed conditionally upstream); step used programmatically with hand-built RowMeta missing key fields; stale transformation after upstream redesign.

Understand the failure class

Background: Record Not Found Errors: "not found", RecordNotFound, and "was not found" — what they mean and how to fix them — this error's family across 28 libraries.

Related errors

• Error converting data while looking up value
• StreamLookup.Exception.CanNotUseIntegerPairAlgorithm
• StreamLookup.Exception.UnableToFindField
• StreamLookup.Log.FieldNotFound
• StreamLookup.Log.GotRowWithoutKeys

AI-assisted analysis of pentaho/pentaho-kettle@f3058517a1 (2026-09-13). Data as JSON: /api/errors/105cc2070728442d. Report an issue: GitHub.

Appendix: source

Thrown at engine/src/main/java/org/pentaho/di/trans/steps/streamlookup/StreamLookup.java:230

    return true;
  }

  private Object[] lookupValues( RowMetaInterface rowMeta, Object[] row ) throws KettleException {
    // See if we need to stop.
    if ( isStopped() ) {
      return null;
    }

    if ( data.lookupColumnIndex == null ) {
      String[] names = data.lookupMeta.getFieldNames();
      data.lookupColumnIndex = new int[names.length];

      for ( int i = 0; i < names.length; i++ ) {
        data.lookupColumnIndex[i] = rowMeta.indexOfValue( names[i] );
        if ( data.lookupColumnIndex[i] < 0 ) {
          // we should not get here
          throw new KettleStepException( "The lookup column '" + names[i] + "' could not be found" );
        }
      }
    }

    // Copy value references to lookup table.
    //
    Object[] lu = new Object[data.keynrs.length];
    for ( int i = 0; i < data.keynrs.length; i++ ) {
      // If the input is binary storage data, we convert it to normal storage.
      //
      if ( data.convertKeysToNative[i] ) {
        lu[i] = data.lookupMeta.getValueMeta( i ).convertBinaryStringToNativeType( (byte[]) row[data.keynrs[i]] );
      } else {
        lu[i] = row[data.keynrs[i]];
      }
    }

    // Handle conflicting types (Number-Integer-String conversion to lookup type in hashtable)

View on GitHub (pinned to f3058517a1)