{"record":{"id":"55cccbf77070e5cc","repo":"pentaho/pentaho-kettle","slug":"error-encountered-during-cache-pre-load","errorCode":null,"errorMessage":"Error encountered during cache pre-load","messagePattern":"Error encountered during cache pre-load","errorType":"exception","errorClass":"KettleException","httpStatus":null,"severity":"error","filePath":"engine/src/main/java/org/pentaho/di/trans/steps/dimensionlookup/DimensionLookup.java","lineNumber":321,"sourceCode":"\n      // Also see what indexes to take to populate the lookup row...\n      // We only ever compare indexes and the lookup date in the cache, the rest is not needed...\n      //\n      data.preloadIndexes = new ArrayList<Integer>();\n      for ( int i = 0; i < meta.getKeyStream().length; i++ ) {\n        int index = data.inputRowMeta.indexOfValue( meta.getKeyStream()[ i ] );\n        if ( index < 0 ) {\n          // Just to be safe...\n          //\n          throw new KettleStepException( BaseMessages.getString(\n            PKG, \"DimensionLookup.Exception.KeyFieldNotFound\", meta.getFieldStream()[ i ] ) );\n        }\n        data.preloadIndexes.add( index );\n      }\n\n      // This is all for now...\n    } catch ( Exception e ) {\n      throw new KettleException( \"Error encountered during cache pre-load\", e );\n    }\n  }\n\n  private synchronized Object[] lookupValues( RowMetaInterface rowMeta, Object[] row ) throws KettleException {\n    Object[] outputRow = new Object[ data.outputRowMeta.size() ];\n\n    RowMetaInterface lookupRowMeta;\n    Object[] lookupRow;\n\n    Object[] returnRow = null;\n\n    Long technicalKey;\n    Long valueVersion;\n    Date valueDate = null;\n    Date valueDateFrom = null;\n    Date valueDateTo = null;\n\n    // Determine the lookup date (\"now\") if we have a field that carries said","sourceCodeStart":303,"sourceCodeEnd":339,"githubUrl":"https://github.com/pentaho/pentaho-kettle/blob/f3058517a153da500bf4551f46d79b91bf8ec552/engine/src/main/java/org/pentaho/di/trans/steps/dimensionlookup/DimensionLookup.java#L303-L339","documentation":"This is the catch-all wrapper KettleException around the whole cache pre-load operation. Whatever happens inside preloadCache (SQL failure, metadata problems, the KeyFieldNotFound error above) is re-thrown with this message and the original exception as the cause, so the real reason is in the chained exception/stack trace.","triggerScenarios":"Any Exception thrown while pre-loading the dimension cache in preloadCache — e.g. the SELECT of all dimension rows fails, or a key field index lookup fails — is wrapped by this KettleDatabaseException/KettleException.","commonSituations":"Dimension table unreadable (permissions, wrong connection), database down, key field not found in stream, OOM when the whole dimension table is loaded into cache on very large dimensions.","solutions":["Read the 'Caused by' exception in the log for the real failure (SQL error vs missing field)","Verify the database connection and that the dimension table exists and is readable by the connection user","Check the log for DimensionLookup.Exception.KeyFieldNotFound and fix the key field mapping","For very large dimension tables, reduce preload scope or disable pre-load caching to avoid memory pressure"],"exampleFix":"// before (log)\n// Error encountered during cache pre-load ... Caused by: Table 'dw.dim_customer' doesn't exist\n// after\n// create/point to the correct table: fix the schema/table name in the Dimension Lookup dialog\n// or fix the DB connection so it targets the right database","handlingStrategy":"try-catch","validationCode":"// before enabling pre-load, verify the dimension table is readable\nDatabase db = new Database(transMeta, databaseMeta);\ndb.connect();\nlong cnt = db.countRows(dimTableMeta); // throws early if table unreadable","typeGuard":null,"tryCatchPattern":"try {\n  processRow(row);\n} catch (KettleException e) {\n  Throwable cause = e;\n  while (cause.getCause() != null) cause = cause.getCause();\n  logError(\"Cache pre-load failed, root cause: \" + cause.getMessage(), e);\n}","preventionTips":["Always inspect the chained 'Caused by' — this message is only a wrapper","Test DB connectivity and table privileges before enabling cache pre-load","Avoid pre-loading very large dimensions; monitor memory when using cache"],"tags":["database","cache","etl","wrapped-exception"],"backgroundTag":"database-query-failed","analyzedSha":"f3058517a153da500bf4551f46d79b91bf8ec552","analyzedAt":"2026-09-13T14:04:16.340Z","contentChangedAt":"2026-09-13T14:04:16.340Z","schemaVersion":2},"datasetVersion":"2026-09-20T23:17:15.980Z"}