apache/beam · error · IllegalArgumentException

Field not nullable

Error message

Field %s not nullable

What it means

autoCastField enforces the row schema's nullability contract: when a field value is null but the declared Schema.Field.Type is not nullable, it throws IllegalArgumentException. It is a schema-validation guard fired from csvLines2BeamRows when incoming CSV data contains an empty/missing value for a column the Beam schema marks as non-nullable.

Solutions

1. Make the field nullable in the schema (FieldType nullable true) if nulls are acceptable
2. Clean the input data to fill or remove null values before parsing
3. Filter rows containing nulls for required fields before csvLines2BeamRows

Example fix

// before
Schema.Field.of("age", Schema.FieldType.INT32) // non-nullable by default
// after
Schema.Field.of("age", Schema.FieldType.INT32.withNullable(true))

if (rawValue == null && !field.getType().getNullable()) { throw new IllegalArgumentException("missing required field " + field.getName()); }

boolean acceptsNull(Schema.Field f) { return f.getType().getNullable(); }

try { row = BeamTableUtils.csvLines2BeamRows(line, schema, format); } catch (IllegalArgumentException e) { /* route row to dead-letter */ }

When it happens

Trigger: A CSV record contains an empty/missing value for a field whose Schema.Field.Type has nullable=false — e.g. an empty cell for a required INT column.

Common situations: Missing values in real-world CSV data; schema built with non-nullable fields by default while data has gaps.

Related errors

• Column type is not supported yet!
• Expect fields, but actually
• FieldTypeValue has no value but the field cannot be null.
• Map value type cannot be null for type: " + fieldType
• Most ( ) and least ( ) significant bits value shouldn't be…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/cdf4c6650d44d3bd. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/schema/BeamTableUtils.java:114

      }
      printer.println();
    } catch (IOException e) {
      throw new IllegalArgumentException("encodeRecord failed!", e);
    }
    return writer.toString();
  }

  /**
   * Attempt to cast an object to a specified Schema.Field.Type.
   *
   * @throws IllegalArgumentException if the value cannot be cast to that type.
   * @return The casted object in Schema.Field.Type.
   */
  public static Object autoCastField(Schema.Field field, @Nullable Object rawObj) {
    // handle null
    if (rawObj == null) {
      if (!field.getType().getNullable()) {
        throw new IllegalArgumentException(String.format("Field %s not nullable", field.getName()));
      }
      return null;
    }

    FieldType type = field.getType();
    // handle NlsString
    if (CalciteUtils.isStringType(type)) {
      if (rawObj instanceof NlsString) {
        return ((NlsString) rawObj).getValue();
      } else {
        return rawObj;
      }
      // handle date/time
    } else if (CalciteUtils.DATE.typesEqual(type) || CalciteUtils.NULLABLE_DATE.typesEqual(type)) {
      if (rawObj instanceof GregorianCalendar) { // used by the SQL CLI
        GregorianCalendar calendar = (GregorianCalendar) rawObj;
        return Instant.ofEpochMilli(calendar.getTimeInMillis())
            .atZone(calendar.getTimeZone().toZoneId())

View on GitHub (pinned to 12126d8942)