pentaho/pentaho-kettle · error · KettleException

Error encountered during cache pre-load

Error message

Error encountered during cache pre-load

What it means

This is the catch-all wrapper KettleException around the whole cache pre-load operation. Whatever happens inside preloadCache (SQL failure, metadata problems, the KeyFieldNotFound error above) is re-thrown with this message and the original exception as the cause, so the real reason is in the chained exception/stack trace.

Solutions

  1. Read the 'Caused by' exception in the log for the real failure (SQL error vs missing field)
  2. Verify the database connection and that the dimension table exists and is readable by the connection user
  3. Check the log for DimensionLookup.Exception.KeyFieldNotFound and fix the key field mapping
  4. For very large dimension tables, reduce preload scope or disable pre-load caching to avoid memory pressure

Example fix

// before (log)
// Error encountered during cache pre-load ... Caused by: Table 'dw.dim_customer' doesn't exist
// after
// create/point to the correct table: fix the schema/table name in the Dimension Lookup dialog
// or fix the DB connection so it targets the right database
Defensive patterns

Strategy: try-catch

Validate before calling

// before enabling pre-load, verify the dimension table is readable
Database db = new Database(transMeta, databaseMeta);
db.connect();
long cnt = db.countRows(dimTableMeta); // throws early if table unreadable

Try / catch

try {
  processRow(row);
} catch (KettleException e) {
  Throwable cause = e;
  while (cause.getCause() != null) cause = cause.getCause();
  logError("Cache pre-load failed, root cause: " + cause.getMessage(), e);
}

Prevention

When it happens

Trigger: Any Exception thrown while pre-loading the dimension cache in preloadCache — e.g. the SELECT of all dimension rows fails, or a key field index lookup fails — is wrapped by this KettleDatabaseException/KettleException.

Common situations: Dimension table unreadable (permissions, wrong connection), database down, key field not found in stream, OOM when the whole dimension table is loaded into cache on very large dimensions.

Understand the failure class

Background: Database query failed: Internal Server Error 500s wrapping SQL, Prisma, and connection failures — what to check first — this error's family across 16 libraries.

Related errors

AI-assisted analysis of pentaho/pentaho-kettle@f3058517a1 (2026-09-13). Data as JSON: /api/errors/55cccbf77070e5cc. Report an issue: GitHub.

Appendix: source

Thrown at engine/src/main/java/org/pentaho/di/trans/steps/dimensionlookup/DimensionLookup.java:321

      // Also see what indexes to take to populate the lookup row...
      // We only ever compare indexes and the lookup date in the cache, the rest is not needed...
      //
      data.preloadIndexes = new ArrayList<Integer>();
      for ( int i = 0; i < meta.getKeyStream().length; i++ ) {
        int index = data.inputRowMeta.indexOfValue( meta.getKeyStream()[ i ] );
        if ( index < 0 ) {
          // Just to be safe...
          //
          throw new KettleStepException( BaseMessages.getString(
            PKG, "DimensionLookup.Exception.KeyFieldNotFound", meta.getFieldStream()[ i ] ) );
        }
        data.preloadIndexes.add( index );
      }

      // This is all for now...
    } catch ( Exception e ) {
      throw new KettleException( "Error encountered during cache pre-load", e );
    }
  }

  private synchronized Object[] lookupValues( RowMetaInterface rowMeta, Object[] row ) throws KettleException {
    Object[] outputRow = new Object[ data.outputRowMeta.size() ];

    RowMetaInterface lookupRowMeta;
    Object[] lookupRow;

    Object[] returnRow = null;

    Long technicalKey;
    Long valueVersion;
    Date valueDate = null;
    Date valueDateFrom = null;
    Date valueDateTo = null;

    // Determine the lookup date ("now") if we have a field that carries said

View on GitHub (pinned to f3058517a1)