pentaho/pentaho-kettle · error · KettleException

All input files need to have the same number of fields. File

Error message

All input files need to have the same number of fields. File '{filename}' has {fieldCount} fields while the first file only had {firstFileFieldCount}

What it means

KettleException thrown by the SAS Input step's processRow when a second (or later) SAS file has a different number of fields than the layout captured from the first file. The step requires all input files to share the same layout and also compares individual ValueMeta per field; it throws this message naming the offending file, its field count, and the first file's count.

Solutions

1. Inspect the named file and compare its columns to the first file; add/remove columns so schemas match.
2. Split processing into multiple SAS Input steps (or transformations) grouped by schema, then merge downstream.
3. Use a wildcard/directory filter to exclude files with differing layouts.
4. Regenerate the outlier file from its SAS source with the same export definition as the others.

Example fix

// before: wildcard mixes schemas
sfr.setFileMask("*.sas7bdat"); // File B has 12 fields vs File A's 10
// after: separate by schema
sfr.setFileMask("sales_*.sas7bdat"); // second step handles returns_*.sas7bdat

// Pre-validate SAS file layouts before the transformation runs
SasInputHelper helper = new SasInputHelper(new File("file.sas7bdat"));
int fields = helper.getRowMeta().size();
if (fields != expectedFieldCount) {
  throw new IllegalStateException("SAS file layout mismatch: " + fields + " vs " + expectedFieldCount);
}

RowMetaInterface layout = data.helper.getRowMeta();
if (layout == null || layout.size() == 0) { throw new IllegalStateException("SAS file produced an empty row layout"); }

try {
  step.run();
} catch (KettleException e) {
  if (e.getMessage().startsWith("All input files need to have the same number of fields")) {
    log.error("Schema mismatch across SAS files: {}", e.getMessage());
    // route offending file to a quarantine folder and re-run
  }
}

When it happens

Trigger: processRow processes multiple accepted SAS files: data.fileLayout is already set and data.helper.getRowMeta().size() differs from data.fileLayout.size(), so the format-uniformity guard throws before per-field comparison.

Common situations: Pointing the step at a directory/wildcard where some SAS files were exported with different schemas (extra or dropped columns); files from different SAS versions or tables mixed in one input; downstream consumers changed one file's export definition.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• SASInput.Log.Error.UnableToFindFilenameField
• Append.Exception.InvalidLayoutDetected
• AutoDoc.Exception.FilenameFieldNotFound
• AutoDoc.Exception.FileTypeFieldNotFound
• AutoDoc.Exception.UnableToDetermineLocation

AI-assisted analysis of pentaho/pentaho-kettle@f3058517a1 (2026-09-13). Data as JSON: /api/errors/e1f2cfa6697f8618. Report an issue: GitHub.

Appendix: source

Thrown at engine/src/main/java/org/pentaho/di/trans/steps/sasinput/SasInput.java:110

        metaStore );
    }

    String rawFilename = getInputRowMeta().getString( fileRowData, meta.getAcceptingField(), null );
    final String filename =
      KettleVFS.getFilename( KettleVFS.getInstance( getTransMeta().getBowl() ).getFileObject( rawFilename ) );

    data.helper = new SasInputHelper( filename );
    logBasic( BaseMessages.getString( PKG, "SASInput.Log.OpenedSASFile" ) + " : [" + data.helper + "]" );

    // verify the row layout...
    //
    if ( data.fileLayout == null ) {
      data.fileLayout = data.helper.getRowMeta();
    } else {
      // Verify that all files are of the same file format, this is a requirement...
      //
      if ( data.fileLayout.size() != data.helper.getRowMeta().size() ) {
        throw new KettleException( "All input files need to have the same number of fields. File '"
          + filename + "' has " + data.helper.getRowMeta().size() + " fields while the first file only had "
          + data.fileLayout.size() );
      }
      for ( int i = 0; i < data.fileLayout.size(); i++ ) {
        ValueMetaInterface first = data.fileLayout.getValueMeta( i );
        ValueMetaInterface second = data.helper.getRowMeta().getValueMeta( i );
        if ( !first.getName().equalsIgnoreCase( second.getName() ) ) {
          throw new KettleException( "Field nr "
            + i + " in file '" + filename + "' is called '" + second.getName() + "' while it was called '"
            + first.getName() + "' in the first file" );
        }
        if ( first.getType() != second.getType() ) {
          throw new KettleException( "Field nr "
            + i + " in file '" + filename + "' is of data type '" + second.getTypeDesc() + "' while it was '"
            + first.getTypeDesc() + "' in the first file" );
        }
      }
    }

View on GitHub (pinned to f3058517a1)