pentaho/pentaho-kettle · error · KettleException

AvroInput.Error.IncommingSchemaIsMissingAndNoDefault

Error message

AvroInput.Error.IncommingSchemaIsMissingAndNoDefault

What it means

In setSchemaToUse, if no incoming schema key is present on the row (empty schemaKey) the step falls back to its default datum reader; when that default was never initialized there is no schema to decode with, so the step throws this KettleException — the row cannot be processed at all.

Solutions

  1. Configure a default Avro schema in the step so rows without a schema key can still be decoded.
  2. Fix the upstream step/field so the schema-key field is populated for every row.
  3. Verify the correct incoming field is selected as the schema source in the step settings.
  4. Add a filter step before Avro Input to route rows with missing schema keys to an error/ignore stream.

Example fix

// before: no default schema in step meta
m_defaultDatumReader = null;
// after: initialize default schema at step init
m_defaultDatumReader = new GenericDatumReader<Object>(defaultSchema);
Defensive patterns

Strategy: validation

Validate before calling

if (Const.isEmpty(row[schemaFieldIndex])) {
  throw new IllegalArgumentException("schema key field is empty on incoming row");
}

Try / catch

try {
  rows = reader.avroObjectToKettle(row, space);
} catch (KettleException e) {
  if (e.getMessage().contains("IncommingSchemaIsMissingAndNoDefault")) { routeToErrorStream(row, e); } else throw e;
}

Prevention

When it happens

Trigger: Step configured to take the schema from an incoming field ('schema from field'/cache mode), but the row's schema-key field is empty/null and m_defaultDatumReader is null (no default schema configured in the step).

Common situations: Upstream step failed to populate the schema field; wrong incoming field selected as the schema source; user assumed a default schema was set but left the default-schema option empty; first row arrives before any cached schema is stored.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

AI-assisted analysis of pentaho/pentaho-kettle@f3058517a1 (2026-09-13). Data as JSON: /api/errors/e7485f8ccd038aed. Report an issue: GitHub.

Appendix: source

Thrown at plugins/avro-format/core/src/main/java/org/pentaho/di/trans/steps/avro/input/AvroNestedReader.java:1622

  }

  public void close() throws IOException {
    if ( m_containerReader != null ) {
      m_containerReader.close();
    }
    if ( m_inStream != null ) {
      m_inStream.close();
    }

  }

  protected void setSchemaToUse( String schemaKey, boolean useCache, VariableSpace space ) throws KettleException {

    if ( Const.isEmpty( schemaKey ) ) {
      // switch to default
      if ( m_defaultDatumReader == null ) {
        // no key, no default schema - can't continue with this row
        throw new KettleException( BaseMessages.getString( PKG,
          "AvroInput.Error.IncommingSchemaIsMissingAndNoDefault" ) );
      }
      if ( m_log.isDetailed() ) {
        m_log.logDetailed( BaseMessages.getString( PKG, "AvroInput.Message.IncommingSchemaIsMissing" ) );
      }
      m_datumReader = m_defaultDatumReader;
      m_schemaToUse = m_datumReader.getSchema();
      setTopLevelStructure( m_defaultTopLevelObject );
      return;
    } else {
      schemaKey = schemaKey.trim();
      schemaKey = space.environmentSubstitute( schemaKey );
    }

    Object[] cached = null;
    if ( useCache ) {
      cached = m_schemaCache.get( schemaKey );
      if ( m_log.isDetailed() && cached != null ) {

View on GitHub (pinned to f3058517a1)