pentaho/pentaho-kettle · error · IllegalArgumentException

some exception while writing avro

Error message

some exception while writing avro

What it means

createAvroRecord catches KettleValueException raised while converting a field value for the Avro record and rewraps it as IllegalArgumentException('some exception while writing avro'). It is a generic wrapper for any field value conversion failure (type conversion, formatting, scaling) during Avro write.

Solutions

  1. Read the wrapped KettleValueException cause to identify the exact field and conversion that failed
  2. Align the Avro output field types with the actual row value types in the step mapping
  3. Pre-convert problem columns (e.g. via Select values / Calculator steps) before Avro output
  4. Catch the exception in write() and route the row to an error stream

Example fix

// before
// field 'amount' declared int in Avro schema, row value is String '12.50'
// after
// convert first in the transformation or set the field type to double/string in the Avro output mapping
field.setAvroType( AvroType.DOUBLE ); // matching the actual data
Defensive patterns

Strategy: try-catch

Validate before calling

// verify each mapped field's Kettle type matches the declared Avro type
for ( IAvroOutputField f : fields ) {
  int valueType = rowMeta.indexOfValue( f.getName() ) >= 0
    ? rowMeta.getValueMeta( rowMeta.indexOfValue( f.getName() ) ).getType() : -1;
  if ( !typeCompatible( valueType, f.getAvroType() ) ) {
    throw new ValidationException( "Field " + f.getName() + " type mismatch" );
  }
}

Try / catch

try {
  writer.write( row );
} catch ( IllegalArgumentException e ) {
  Throwable cause = e.getCause(); // KettleValueException names the failing field/conversion
  logError( "Avro field conversion failed: " + cause.getMessage(), cause );
}

Prevention

When it happens

Trigger: A KettleValueException thrown by value conversion code inside the field loop of createAvroRecord — e.g. converting a non-numeric string to a number, incompatible field type vs declared Avro schema type, or applyScale failures.

Common situations: Avro output field declared as integer/double while the row holds a string; null handling differences; decimal fields whose scale/precision can't be applied; schema changed but data did not.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

AI-assisted analysis of pentaho/pentaho-kettle@f3058517a1 (2026-09-13). Data as JSON: /api/errors/4aeff951db679b67. Report an issue: GitHub.

Appendix: source

Thrown at plugins/avro-format/core/src/main/java/org/pentaho/di/trans/steps/avro/output/PentahoAvroRecordWriter.java:227

              if ( defaultValue != null && !defaultValue.isEmpty() ) {
                String conversionMask =
                  ( vmi.getConversionMask() == null ) ? ValueMetaBase.DEFAULT_TIMESTAMP_PARSE_MASK
                    : vmi.getConversionMask();
                DateTimeFormatter fmt = DateTimeFormatter.ofPattern( conversionMask );
                LocalDateTime ldt = LocalDateTime.parse( defaultValue, fmt );
                defaultTimestamp = Timestamp.valueOf( ldt );
              }
              Timestamp timeStamp = (Timestamp) row.getDate( fieldMetaIndex, defaultTimestamp );
              outputRecord.put( avroFieldName, AvroTimestampHandler.fromTimestamp( timeStamp, avroType ) );
              break;
          }

        }
      }
    } catch ( ArithmeticException e ) {
      throw new IllegalArgumentException( "The date has too much day from epoch day!", e );
    } catch ( KettleValueException e ) {
      throw new IllegalArgumentException( "some exception while writing avro", e );
    }
    return outputRecord;
  }

  private double applyScale( double number, IAvroOutputField outputField ) {
    if ( outputField.getScale() > 0 ) {
      BigDecimal bd = new BigDecimal( number );
      bd = bd.setScale( outputField.getScale(), BigDecimal.ROUND_HALF_UP );
      number = bd.doubleValue();
    }
    return number;
  }

  private float applyScale( float number, IAvroOutputField outputField ) {
    if ( outputField.getScale() > 0 ) {
      BigDecimal bd = new BigDecimal( number );
      bd = bd.setScale( outputField.getScale(), BigDecimal.ROUND_HALF_UP );
      number = bd.floatValue();

View on GitHub (pinned to f3058517a1)