{"record":{"id":"105cc2070728442d","repo":"pentaho/pentaho-kettle","slug":"the-lookup-column","errorCode":null,"errorMessage":"The lookup column '","messagePattern":"The lookup column '","errorType":"exception","errorClass":"KettleStepException","httpStatus":null,"severity":"error","filePath":"engine/src/main/java/org/pentaho/di/trans/steps/streamlookup/StreamLookup.java","lineNumber":230,"sourceCode":"\n    return true;\n  }\n\n  private Object[] lookupValues( RowMetaInterface rowMeta, Object[] row ) throws KettleException {\n    // See if we need to stop.\n    if ( isStopped() ) {\n      return null;\n    }\n\n    if ( data.lookupColumnIndex == null ) {\n      String[] names = data.lookupMeta.getFieldNames();\n      data.lookupColumnIndex = new int[names.length];\n\n      for ( int i = 0; i < names.length; i++ ) {\n        data.lookupColumnIndex[i] = rowMeta.indexOfValue( names[i] );\n        if ( data.lookupColumnIndex[i] < 0 ) {\n          // we should not get here\n          throw new KettleStepException( \"The lookup column '\" + names[i] + \"' could not be found\" );\n        }\n      }\n    }\n\n    // Copy value references to lookup table.\n    //\n    Object[] lu = new Object[data.keynrs.length];\n    for ( int i = 0; i < data.keynrs.length; i++ ) {\n      // If the input is binary storage data, we convert it to normal storage.\n      //\n      if ( data.convertKeysToNative[i] ) {\n        lu[i] = data.lookupMeta.getValueMeta( i ).convertBinaryStringToNativeType( (byte[]) row[data.keynrs[i]] );\n      } else {\n        lu[i] = row[data.keynrs[i]];\n      }\n    }\n\n    // Handle conflicting types (Number-Integer-String conversion to lookup type in hashtable)","sourceCodeStart":212,"sourceCodeEnd":248,"githubUrl":"https://github.com/pentaho/pentaho-kettle/blob/f3058517a153da500bf4551f46d79b91bf8ec552/engine/src/main/java/org/pentaho/di/trans/steps/streamlookup/StreamLookup.java#L212-L248","documentation":"In lookupValues, the main-stream lookup key columns are resolved by name against the incoming row; if a lookup key field is missing the step throws 'The lookup column X could not be found'. The code comment notes 'we should not get here' — normally earlier validation catches it, so this is a defensive check for runtime metadata drift.","triggerScenarios":"lookupValues (called from outputRow) builds data.lookupColumnIndex via rowMeta.indexOfValue(names[i]) and gets -1 for a main-stream key field.","commonSituations":"Main input stream shape changed at runtime (e.g. fields removed conditionally upstream); step used programmatically with hand-built RowMeta missing key fields; stale transformation after upstream redesign.","solutions":["Ensure the main-stream input actually contains every configured lookup key field at runtime","Re-run transformation metadata validation / re-open the step dialog to refresh field names","If calling the step programmatically, build the input RowMetaInterface with all key field names present","Trace upstream steps (Calculator, Filter, Select Values) that may conditionally drop the field"],"exampleFix":"// before (programmatic use)\nRowMeta rowMeta = new RowMeta(); // missing 'id'\n// after\nrowMeta.addValueMeta(new ValueMetaInteger(\"id\")); // include all key fields","handlingStrategy":"validation","validationCode":"// Pre-check main stream row metadata before invoking lookup\nint idx = mainRowMeta.indexOfValue(keyFieldName);\nif (idx < 0) throw new IllegalArgumentException(\"Main stream lacks lookup key: \" + keyFieldName);","typeGuard":"boolean mainStreamHasKey(RowMetaInterface rm, String key) { return rm.indexOfValue(key) >= 0; }","tryCatchPattern":"try { outputRow(inputRow); } catch (KettleStepException e) {\n  if (e.getMessage().startsWith(\"The lookup column\")) { /* fix main stream layout */ }\n  throw e;\n}","preventionTips":["Avoid steps that conditionally alter row layout upstream of StreamLookup","Pin required fields with a Select Values step (removing others) to stabilize layout","Test transformations with representative data before scheduling"],"tags":["kettle","pdi","stream-lookup","runtime-metadata"],"backgroundTag":"record-not-found","analyzedSha":"f3058517a153da500bf4551f46d79b91bf8ec552","analyzedAt":"2026-09-13T14:04:16.340Z","contentChangedAt":"2026-09-13T14:04:16.340Z","schemaVersion":2},"datasetVersion":"2026-09-20T23:17:15.980Z"}