pentaho/pentaho-kettle · error · KettleException

Failed to add dimension rows

Error message

Failed to add dimension rows

What it means

When the input stream ends, the step flushes its buffers: it adds buffered dimension rows and consolidations to the Palo dimension via data.helper.addDimensionRows/addDimensionConsolidations. Any exception in that final flush is wrapped as 'Failed to add dimension rows', meaning the writes to the Palo server failed at commit time.

Solutions

1. Check the wrapped cause for the specific Palo server error
2. Verify network/session stability and re-run; consider smaller batch commits (the 100-row commit size)
3. Ensure the dimension on the server was not restructured while the transformation ran
4. Validate all buffered element values (types, uniqueness) before the run

Example fix

// before
commitBatch(); // every 100 rows, one giant final flush may fail
// after
if (data.elementNamesBatch.size() % 50 == 0) commitBatch(); // smaller batches

// validate buffered elements before final flush
if (data.elementNamesBatch.isEmpty()) { logBasic("Nothing to flush"); return; }

try { flushDimensionRows(); } catch (Exception e) {
  logError("Final dimension flush failed: " + e.getCause(), e); setErrors(1);
  // optionally retry once
}

When it happens

Trigger: Final flush to Palo fails: connection lost during the run, element values invalid for the dimension, dimension structure changed concurrently, or server-side quota/error during bulk add.

Common situations: Long-running transformation whose Palo session expired; bulk insert too large for the server; duplicate/conflicting element definitions submitted at once.

Related errors

• A connection of type PALO is expected
• A connection of type PALO is expected
• A connection of type PALO is expected
• A connection of type PALO is expected
• All fields must have an output type to do the preview

AI-assisted analysis of pentaho/pentaho-kettle@f3058517a1 (2026-09-13). Data as JSON: /api/errors/aba64b16e1b2610e. Report an issue: GitHub.

Appendix: source

Thrown at plugins/palo/core/src/main/java/org/pentaho/di/trans/steps/palo/dimoutput/PaloDimOutput.java:175

        }

        // Because we add rows in bulk, the dimension cache isn't complete. It only keeps a list
        // of added items for validation. If the cache must be pre-loaded, pre-load it now.
        data.helper.clearDimensionCache();
        data.helper.loadDimensionCache( meta.getDimension(), meta.getEnableElementCache(), meta
          .getPreloadElementCache() );

        // if it's the last row create the dimension
        this.logBasic( "All rows have been added. Looking for consolidations" );

        this.logBasic( "Updating consolidations for Dimension" + meta.getDimension() );
        data.helper.addDimensionConsolidations( meta.getDimension(), this.consolidations );
        this.logBasic( "Consolidations updated." );

        setOutputDone();
        return false;
      } catch ( Exception e ) {
        throw new KettleException( "Failed to add dimension rows", e );
      }
    }

    return true;
  }

  private void commitBatch() throws Exception {
    data.helper.addDimensionElements( data.elementNamesBatch, meta.getElementType() );
    while ( rowCount > 0 ) {
      incrementLinesOutput();
      rowCount--;
    }
    data.elementNamesBatch.clear();
  }

  public final boolean init( StepMetaInterface smi, StepDataInterface sdi ) {
    meta = (PaloDimOutputMeta) smi;
    data = (PaloDimOutputData) sdi;

View on GitHub (pinned to f3058517a1)