What causes "BaseStep.SafeMode.Exception.VaryingSize" in pentaho/pentaho-kettle?

Fix the producing step so every row has the same number of fields (always add all fields, using null for missing values).

pentaho/pentaho-kettle · error · KettleRowException

BaseStep.SafeMode.Exception.VaryingSize

BaseStep.SafeMode.Exception.VaryingSize

Error message

BaseStep.SafeMode.Exception.VaryingSize

What it means

safeModeChecking compares each incoming row's RowMetaInterface against the reference row (typically the first row seen). If the field count differs, KettleRowException is thrown. This catches steps that emit rows with inconsistent arity, which would otherwise corrupt row sets.

Solutions

Fix the producing step so every row has the same number of fields (always add all fields, using null for missing values).
Log the row layout at the failing step boundary (e.g. with a 'Stream Lookup' or copy of getFields) to find which path emits the wrong arity.
For parsing steps, configure explicit column definitions so output arity is fixed regardless of input content.
If a legitimately variable schema is required, disable safe mode and handle rows defensively downstream.

Example fix

// before
if (value != null) { outputRow = RowDataUtil.addRowData(inputRow, 1, new Object[]{value}); }
else { outputRow = inputRow; } // wrong arity
// after
outputRow = RowDataUtil.addRowData(inputRow, 1, new Object[]{value}); // null kept, arity constant

Defensive patterns

Strategy: validation

Validate before calling

if (rowMeta.size() != expectedSize) throw new IllegalStateException("Row has " + rowMeta.size() + " fields, expected " + expectedSize);

Try / catch

try { putRow(outputRowMeta, row); } catch (KettleRowException e) { if (e.getMessage().contains("VaryingSize")) { logError("Inconsistent row arity; fix producing step"); } throw e; }

Prevention

Always allocate all output fields per row, using null placeholders
Use RowDataUtil.addRowData with a fixed-size reservation
Never conditionally skip field creation in processRow
Test custom steps with both empty and populated branches

When it happens

Trigger: Safe mode enabled; a step emits a first row with N fields and a later row with a different number of fields (e.g. conditional putRow calls, variable-length JSON/CSV parsing).

Common situations: A JavaScript/UDJE step that adds fields only under some conditions; a streaming source whose schema changed mid-run; a splitter step that sometimes emits fewer columns for malformed input lines.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of pentaho/pentaho-kettle@f3058517a1 (2026-09-13). Data as JSON: /api/errors/661208ef329812b5. Report an issue: GitHub.

Appendix: source

Thrown at engine/src/main/java/org/pentaho/di/trans/step/BaseStep.java:2171

        outputRowSetsLock.writeLock().unlock();
      }
    }
  }

  /**
   * Safe mode checking.
   *
   * @param referenceRowMeta the reference row meta
   * @param rowMeta          the row meta
   * @throws KettleRowException the kettle row exception
   */
  public static void safeModeChecking( RowMetaInterface referenceRowMeta, RowMetaInterface rowMeta )
    throws KettleRowException {
    // See if the row we got has the same layout as the reference row.
    // First check the number of fields
    //
    if ( referenceRowMeta.size() != rowMeta.size() ) {
      throw new KettleRowException( BaseMessages.getString( PKG, "BaseStep.SafeMode.Exception.VaryingSize", ""
        + referenceRowMeta.size(), "" + rowMeta.size(), rowMeta.toString() ) );
    } else {
      // Check field by field for the position of the names...
      for ( int i = 0; i < referenceRowMeta.size(); i++ ) {
        ValueMetaInterface referenceValue = referenceRowMeta.getValueMeta( i );
        ValueMetaInterface compareValue = rowMeta.getValueMeta( i );

        if ( !referenceValue.getName().equalsIgnoreCase( compareValue.getName() ) ) {
          throw new KettleRowException( BaseMessages.getString(
            PKG, "BaseStep.SafeMode.Exception.MixingLayout", "" + ( i + 1 ), referenceValue.getName()
              + " " + referenceValue.toStringMeta(), compareValue.getName()
              + " " + compareValue.toStringMeta() ) );
        }

        if ( referenceValue.getType() != compareValue.getType() ) {
          throw new KettleRowException( BaseMessages.getString( PKG, "BaseStep.SafeMode.Exception.MixingTypes", ""
            + ( i + 1 ), referenceValue.getName() + " " + referenceValue.toStringMeta(), compareValue.getName()
            + " " + compareValue.toStringMeta() ) );

View on GitHub (pinned to f3058517a1)