pentaho/pentaho-kettle · error · KettleStepException

SynchronizeAfterMerge.Error.SameColumnInsertedTwice

Error message

SynchronizeAfterMerge.Error.SameColumnInsertedTwice

What it means

During processRow() the step builds the INSERT row metadata by mapping each Update stream field to an Update lookup (table column). If the target table column (updateLookup[i]) was already added — i.e., the same column appears twice in the Update lookup table — the step throws KettleStepException 'SameColumnInsertedTwice' because a row cannot contain two values for one column.

Solutions

Open the Synchronize After Merge dialog and remove or correct duplicate entries in the 'Update lookup' column so each table column appears once
If two stream fields must both be persisted, change one row's lookup column to a distinct table column
Use a stream-to-stream merge (e.g., Stream lookup / Formula) upstream so only one field maps to each column

Example fix

// before (grid)
stream: amount    -> lookup: amount
stream: old_amount-> lookup: amount   // duplicate
// after
stream: amount    -> lookup: amount
stream: old_amount-> lookup: previous_amount  // distinct column

Defensive patterns

Strategy: validation

Validate before calling

var m = stepMeta.getStepMetaInterface();
var seen = {};
for (var i = 0; i < m.getUpdateLookup().length; i++) {
  if (seen[m.getUpdateLookup()[i]]) throw new Error("Duplicate lookup column: " + m.getUpdateLookup()[i]);
  seen[m.getUpdateLookup()[i]] = true;
}

Try / catch

try { trans.execute(...); } catch (KettleException e) { if (e.getMessage().contains("SameColumnInsertedTwice")) { log.error("Remove duplicate update-lookup mapping: " + e.getMessage()); } else throw e; }

Prevention

Never copy-paste rows in the mapping grid without editing the lookup column
Enforce one-to-one mapping between stream fields and table columns
Review the mapping grid after merging transformations
Use distinct target columns for distinct stream values

When it happens

Trigger: The 'Update/Insert lookup' grid lists the same table column name in two rows (two stream fields mapped to the same target column), e.g. mapping both amount and old_amount to column amount.

Common situations: Copy-pasting rows in the step grid and forgetting to change the lookup column; bulk-loading a mapping where multiple stream fields target one column; merge of step settings from two transformations.

Understand the failure class

Background: Conflicting config options: "cannot be used together" — configuration validation errors across open-source libraries — this error's family across 162 libraries.

Related errors

AI-assisted analysis of pentaho/pentaho-kettle@f3058517a1 (2026-09-13). Data as JSON: /api/errors/c0cba4fcf66e5cf9. Report an issue: GitHub.

Appendix: source

Thrown at engine/src/main/java/org/pentaho/di/trans/steps/synchronizeaftermerge/SynchronizeAfterMerge.java:739

            meta.getKeyStream2()[i] ) );
        }

        if ( log.isDebug() ) {
          logDebug( BaseMessages.getString( PKG, "SynchronizeAfterMerge.Log.FieldHasDataNumbers", meta
            .getKeyStream()[i] ) + data.keynrs[i] );
        }
      }

      // Insert the update fields: just names. Type doesn't matter!
      for (int i = 0; i < meta.getUpdateLookup().length; i++) {
        ValueMetaInterface insValue = data.insertRowMeta.searchValueMeta( meta.getUpdateLookup()[i] );
        if ( insValue == null ) { // Don't add twice!
          // we already checked that this value exists so it's probably safe to ignore lookup failure...
          ValueMetaInterface insertValue = data.inputRowMeta.searchValueMeta( meta.getUpdateStream()[i] ).clone();
          insertValue.setName( meta.getUpdateLookup()[i] );
          data.insertRowMeta.addValueMeta( insertValue );
        } else {
          throw new KettleStepException( BaseMessages.getString( PKG,
            "SynchronizeAfterMerge.Error.SameColumnInsertedTwice", insValue.getName() ) );
        }
      }

      // Cache the position of the compare fields in Row row
      //
      data.valuenrs = new int[meta.getUpdateLookup().length];
      for (int i = 0; i < meta.getUpdateLookup().length; i++) {
        data.valuenrs[i] = data.inputRowMeta.indexOfValue( meta.getUpdateStream()[i] );
        if ( data.valuenrs[i] < 0 ) { // couldn't find field!
          throw new KettleStepException( BaseMessages.getString( PKG, "SynchronizeAfterMerge.Exception.FieldRequired",
            meta.getUpdateStream()[i] ) );
        }
        if ( log.isDebug() ) {
          logDebug( BaseMessages.getString( PKG, "SynchronizeAfterMerge.Log.FieldHasDataNumbers", meta
            .getUpdateStream()[i] ) + data.valuenrs[i] );
        }
      }

View on GitHub (pinned to f3058517a1)