{"record":{"id":"1c0999491852acea","repo":"pentaho/pentaho-kettle","slug":"it-doesn-t-make-sense-to-have-a-partitioned-clustered-step","errorCode":null,"errorMessage":"It doesn't make sense to have a partitioned, clustered step with less partitions (\" + nrPartitions + \") than that there are slave servers (\" + nrSlaves + \")","messagePattern":"It doesn't make sense to have a partitioned, clustered step with less partitions \\(\" \\+ nrPartitions \\+ \"\\) than that there are slave servers \\(\" \\+ nrSlaves \\+ \"\\)","errorType":"exception","errorClass":"KettleException","httpStatus":null,"severity":"error","filePath":"engine/src/main/java/org/pentaho/di/trans/cluster/TransSplitter.java","lineNumber":1529,"sourceCode":"      // Also, all cloned steps re-match with the cloned schema name afterwards...\n      //\n      PartitionSchema partitionSchema = (PartitionSchema) stepPartitioningMeta.getPartitionSchema().clone();\n\n      int nrSlaves = clusterSchema.findNrSlaves();\n      if ( nrSlaves == 0 ) {\n        continue; // no slaves: ignore this situation too\n      }\n\n      // Change the partitioning layout dynamically if the user requested this...\n      //\n      if ( partitionSchema.isDynamicallyDefined() ) {\n        partitionSchema.expandPartitionsDynamically( nrSlaves, originalTransformation );\n      }\n\n      int nrPartitions = partitionSchema.getPartitionIDs().size();\n\n      if ( nrPartitions < nrSlaves ) {\n        throw new KettleException(\n          \"It doesn't make sense to have a partitioned, clustered step with less partitions (\"\n            + nrPartitions + \") than that there are slave servers (\" + nrSlaves + \")\" );\n      }\n\n      int slaveServerNr = 0;\n      List<SlaveServer> slaveServers = clusterSchema.getSlaveServers();\n\n      for ( int p = 0; p < nrPartitions; p++ ) {\n        String partitionId = partitionSchema.getPartitionIDs().get( p );\n\n        SlaveServer slaveServer = slaveServers.get( slaveServerNr );\n\n        // Skip the master...\n        //\n        if ( slaveServer.isMaster() ) {\n          slaveServerNr++;\n          if ( slaveServerNr >= slaveServers.size() ) {\n            slaveServerNr = 0; // re-start","sourceCodeStart":1511,"sourceCodeEnd":1547,"githubUrl":"https://github.com/pentaho/pentaho-kettle/blob/f3058517a153da500bf4551f46d79b91bf8ec552/engine/src/main/java/org/pentaho/di/trans/cluster/TransSplitter.java#L1511-L1547","documentation":"When a step is both partitioned and clustered, the TransSplitter constructor dynamically expands the partition schema across slave servers and requires at least as many partitions as slave servers (nrPartitions < nrSlaves throws). Fewer partitions than slaves would leave some slaves idle, so Kettle refuses the configuration.","triggerScenarios":"Constructing a TransSplitter (via execute/executeClustered) where a clustered step uses a partition schema with fewer partition IDs than the number of slave servers in the cluster schema — e.g. 2 partitions on a 3-slave cluster.","commonSituations":"Reusing a partition schema designed for a smaller cluster when adding slave servers; defining partitions manually instead of letting expandPartitionsDynamically size them to the cluster.","solutions":["Add partition IDs so there are at least as many partitions as slave servers.","Keep 'dynamic' partitioning enabled so the schema expands to the current slave count.","Reduce the number of slave servers in the cluster schema to match (or be below) the partition count."],"exampleFix":"// before\nschema.addPartitionID(\"P1\"); schema.addPartitionID(\"P2\"); // 3 slaves\n// after\nschema.expandPartitionsDynamically(nrSlaves, transMeta);","handlingStrategy":"validation","validationCode":"int nrSlaves = clusterSchema.getSlaveServers().size();\nint nrPartitions = partitionSchema.getPartitionIDs().size();\nif (nrPartitions < nrSlaves) partitionSchema.expandPartitionsDynamically(nrSlaves, transMeta);","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Use dynamic partitioning so partitions scale with the cluster","Re-validate partition counts after adding slave servers","Keep partition count >= slave count in schema design"],"tags":["kettle","partitioning","clustering","configuration"],"backgroundTag":"invalid-config-value","analyzedSha":"f3058517a153da500bf4551f46d79b91bf8ec552","analyzedAt":"2026-09-13T14:04:16.340Z","contentChangedAt":"2026-09-13T14:04:16.340Z","schemaVersion":2},"datasetVersion":"2026-09-20T23:17:15.980Z"}