pentaho/pentaho-kettle · error · KettleValueException

StreamLookup.Exception.CanNotUseIntegerPairAlgorithm

Error message

StreamLookup.Exception.CanNotUseIntegerPairAlgorithm

What it means

The Integer-pair optimization caches lookup rows as a Map<Long,Long> and is only valid when both the key and the value are exactly one Integer field each. addToCache throws KettleValueException when the first lookup row reveals the metadata does not match that constraint (e.g. two keys, or a String value).

Solutions

  1. Uncheck/disable the integer-pair optimization in the step dialog ('Use integer pair...' option) so the general hashtable algorithm is used
  2. Or convert the key and single value fields to Integer upstream so the constraint holds
  3. Or redesign to a single surrogate Integer key if composite keys are needed
  4. Verify field types with a 'Select Values' metadata step immediately before the lookup

Example fix

// before
<use_int_pair>true</use_int_pair> <!-- but value is String -->
// after
<use_int_pair>false</use_int_pair>
Defensive patterns

Strategy: validation

Validate before calling

// Verify integer-pair preconditions before enabling the optimization
boolean intPairOk = keyMeta.size() == 1 && valueMeta.size() == 1
  && keyMeta.getValueMeta(0).isInteger() && valueMeta.getValueMeta(0).isInteger();

Type guard

boolean canUseIntegerPair(RowMetaInterface keys, RowMetaInterface values) {
  return keys.size() == 1 && values.size() == 1
    && keys.getValueMeta(0).isInteger() && values.getValueMeta(0).isInteger();
}

Try / catch

try { addToCache(...); } catch (KettleValueException e) {
  if (e.getMessage().contains("IntegerPair")) { /* disable int-pair option and rerun */ }
  throw e;
}

Prevention

When it happens

Trigger: addToCache (from readLookupValues) checks metadataVerifiedIntegerPair on the first row: keyMeta.size()!=1, valueMeta.size()!=1, or either value meta is not Integer.

Common situations: Step configured with 'key and value are integers' (integer pair algorithm) but the streams actually carry composite keys or non-Integer types; upstream type conversions changed key/value types after the option was set.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

AI-assisted analysis of pentaho/pentaho-kettle@f3058517a1 (2026-09-13). Data as JSON: /api/errors/17ef4ea915eec0ce. Report an issue: GitHub.

Appendix: source

Thrown at engine/src/main/java/org/pentaho/di/trans/steps/streamlookup/StreamLookup.java:307

    if ( meta.isMemoryPreservationActive() ) {
      if ( meta.isUsingSortedList() ) {
        KeyValue keyValue = new KeyValue( keyData, valueData );
        int idx = Collections.binarySearch( data.list, keyValue, data.comparator );
        if ( idx < 0 ) {
          int index = -idx - 1; // this is the insertion point
          data.list.add( index, keyValue ); // insert to keep sorted.
        } else {
          data.list.set( idx, keyValue ); // Overwrite to simulate Hashtable behaviour
        }
      } else {
        if ( meta.isUsingIntegerPair() ) {
          if ( !data.metadataVerifiedIntegerPair ) {
            data.metadataVerifiedIntegerPair = true;
            if ( keyMeta.size() != 1
              || valueMeta.size() != 1 || !keyMeta.getValueMeta( 0 ).isInteger()
              || !valueMeta.getValueMeta( 0 ).isInteger() ) {

              throw new KettleValueException( BaseMessages.getString(
                PKG, "StreamLookup.Exception.CanNotUseIntegerPairAlgorithm" ) );
            }
          }

          Long key = keyMeta.getInteger( keyData, 0 );
          Long value = valueMeta.getInteger( valueData, 0 );
          data.longIndex.put( key, value );
        } else {
          if ( data.hashIndex == null ) {
            data.hashIndex = new ByteArrayHashIndex( keyMeta );
          }
          data.hashIndex
            .put( RowMeta.extractData( keyMeta, keyData ), RowMeta.extractData( valueMeta, valueData ) );
        }
      }
    } else {
      // We can't just put Object[] in the map The compare function is not in it.
      // We need to wrap in and use that. Let's use RowMetaAndData for this one.

View on GitHub (pinned to f3058517a1)