apache/beam · error · java.lang.IllegalArgumentException

Received an empty value for non-nullable Snowflake field

Error message

Received an empty value for non-nullable Snowflake field '%s'.

What it means

toBeamValue rejects empty strings for Snowflake fields whose Beam schema type is not nullable (except STRING, where an empty string is preserved). A non-nullable field receiving an empty value from Snowflake would produce an invalid Beam Row, so it throws an IllegalArgumentException identifying the field.

Solutions

1. Mark the field nullable in the Beam schema: Field.of(name, FieldType.X).withNullable(true).
2. Filter out or transform NULL/empty values in the query (COALESCE(col, default)).
3. Use a STRING field type if empty strings should be preserved as-is.

Example fix

// before
Field.of("age", FieldType.INT64) // non-nullable, column can be NULL
// after
Field.of("age", FieldType.INT64).withNullable(true)

for (int i = 0; i < schema.getFieldCount(); i++) {
  Field f = schema.getField(i);
  if ((parts[i] == null || parts[i].isEmpty()) && !f.getType().getNullable()
      && f.getType().getTypeName() != Schema.TypeName.STRING) {
    throw new IllegalArgumentException("empty value for non-nullable field " + f.getName());
  }
}

try { Row r = SnowflakeSchemaTransformUtils.toRow(parts, schema); }
catch (IllegalArgumentException e) { log.error(e.getMessage()); routeToDeadLetter(parts); }

When it happens

Trigger: A Snowflake column containing NULL or empty string mapped to a Beam Field declared non-nullable (getFieldNullable() false) of any non-STRING type (INTEGER, BOOLEAN, TIMESTAMP, BYTES, etc.) processed via toRow.

Common situations: Sparse columns with NULLs but schema marked required; COALESCE-less joins producing NULLs; CSV-ish exports where missing values become empty strings.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• Field is not nullable.
• Java Bean ' ' contains a setter for field ' ' that has a…
• Null value set on non-nullable field
• Provided length is bigger than max length
• Provided size is bigger than max size

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/97fa3551bf542a85. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/snowflake/src/main/java/org/apache/beam/sdk/io/snowflake/SnowflakeSchemaTransformUtils.java:207

  }

  public static @Nullable Object toBeamValue(String value, Schema.Field field) {
    if (value == null || value.isEmpty()) {
      if (field.getType().getNullable()) {
        return null;
      }

      /*
       * Snowflake COPY encodes NULL as an empty CSV value. Therefore an empty
       * value cannot be represented for a required non-string type.
       *
       * For STRING, preserve the empty string.
       */
      if (field.getType().getTypeName() == Schema.TypeName.STRING) {
        return "";
      }

      throw new IllegalArgumentException(
          String.format(
              "Received an empty value for non-nullable Snowflake field '%s'.", field.getName()));
    }

    try {
      switch (field.getType().getTypeName()) {
        case BYTE:
          return Byte.valueOf(value);

        case INT16:
          return Short.valueOf(value);

        case INT32:
          return Integer.valueOf(value);

        case INT64:
          return Long.valueOf(value);

View on GitHub (pinned to 12126d8942)