pentaho/pentaho-kettle · error · KettleException

Unexpected problem while generating master transformation

Error message

Unexpected problem while generating master transformation

What it means

splitOriginalTransformation wraps its whole master-transformation generation in a catch-all: any Exception raised while building the master TransMeta is rethrown as 'Unexpected problem while generating master transformation' with the original cause attached. It is a generic wrapper, so the root cause is in getCause().

Solutions

1. Inspect the exception's cause (e.getCause()) — the wrapper hides the real failure.
2. Fix the underlying step/metadata issue identified by the cause (often a null cluster schema or invalid remote input/output step).
3. Test the same transformation unclustered locally to isolate which step's metadata breaks clustering.
4. Update Kettle/plugins to a version where the causing bug is fixed.

Example fix

// before
transSplitter.executeClustered(...); // only prints wrapper message
// after
try { transSplitter.executeClustered(...); }
catch (KettleException e) { log.error("root cause", e.getCause()); }

try { transSplitter.executeClustered(); }
catch (KettleException e) {
  Throwable root = e; while (root.getCause() != null) root = root.getCause();
  log.error("Master generation failed: " + root.getMessage(), root);
}

When it happens

Trigger: Any exception during splitOriginalTransformation()'s master-generation phase — e.g. invalid step metadata, NPE from a missing cluster schema, or serialization problems in createMasterTransformation — inside the try block ending at TransSplitter.java:1398.

Common situations: Corrupted or partially configured step metadata in the transformation; plugin steps whose meta classes fail during clustering setup; programming errors in custom step plugins exposed only when clustering.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• At this time we don't support the use of multiple cluster…
• No cluster schemas are being used. As such it is not…
• No clustering is used in this transformation.
• A deadlock was detected between steps
• A server socket allocation always has to accompanied by a…

AI-assisted analysis of pentaho/pentaho-kettle@f3058517a1 (2026-09-13). Data as JSON: /api/errors/dc166a348a0b905e. Report an issue: GitHub.

Appendix: source

Thrown at engine/src/main/java/org/pentaho/di/trans/cluster/TransSplitter.java:1398

        transMeta.clearChanged();
      }
      // do not erase partitioning schema for master transformation
      // if some of steps is expected to run on master partitioned, that is the case
      // when partition schema should exists as 'local' partition schema instead of slave's remote one
      // see PDI-12766
      //NOTE: PDI-18333 keep newly created partitionSchemas and add back in original per PDI-12766
      masterTransMeta.addOrReplacePartitionSchema( originalTransformation.getPartitionSchemas() );

      masterTransMeta.setSlaveStepCopyPartitionDistribution( slaveStepCopyPartitionDistribution );
      if ( encrypt ) {
        masterTransMeta.setKey( pubK.getEncoded() );
        masterTransMeta.setPrivateKey( !false );
      }
      masterTransMeta.clearChanged();

      // We're absolutely done here...
    } catch ( Exception e ) {
      throw new KettleException( "Unexpected problem while generating master transformation", e );
    }
  }

  /**
   * Calculate the number of step copies in a step.<br>
   * If a step is not running clustered, it's simply returning getCopies().<br>
   * If a step is clustered and not doing any partitioning, it's simply returning getCopies().<br>
   * If a step is clustered and partitioned, we need to look in the partitioning map for the specified slave server.<br>
   * That is because the number of copies can vary over the slaves. (5 partitions over 3 slaves for example)
   *
   * @param slaveServer
   *          the slave server
   * @param step
   *          the reference step
   * @return the number of step copies that we run.
   */
  private int determineNrOfStepCopies( SlaveServer slaveServer, StepMeta step ) {
    if ( !step.isClustered() ) {

View on GitHub (pinned to f3058517a1)