apache/beam · error · IllegalArgumentException

encodeRecord failed!

Error message

encodeRecord failed!

What it means

beamRow2CsvLine failed to serialize a Beam Row into a CSV line: while printing the row's fields with the given CSVFormat, an IOException escaped the CSVPrinter, and it is wrapped in an IllegalArgumentException. Typically caused by a field value incompatible with the CSVFormat (e.g. a delimiter/quoting problem or a null in a non-nullable position), not by the CSV library itself failing.

Solutions

1. Check that all Row field values are non-null and serializable to strings
2. Review the CSVFormat settings for compatibility with field contents
3. Catch the IllegalArgumentException to identify the offending row and log it

Example fix

// before
row.getBaseValue(i, Object.class).toString() // value may be null -> NPE/IO issues
// after
Object v = row.getBaseValue(i, Object.class);
printer.print(v == null ? "" : v.toString());

for (int i = 0; i < row.getFieldCount(); i++) { if (row.getBaseValue(i, Object.class) == null) throw new IllegalStateException("null field at " + i); }

try { csv = BeamTableUtils.beamRow2CsvLine(row, format); } catch (IllegalArgumentException e) { log.error("row serialization failed", e); }

When it happens

Trigger: Row field values whose toString/write path makes the CSVPrinter throw an IOException, or the underlying StringWriter failing while printing fields.

Common situations: Rows containing values incompatible with the configured CSVFormat (e.g. characters needing unavailable escaping setups); rare writer/IO failures during serialization.

Understand the failure class

Background: json.Marshal / "failed to marshal" errors in Go: why "unsupported type" happens and how to fix it — this error's family across 22 libraries.

Related errors

• Attempted to encode null for non-nullable field
• Azure credential provider type
• bad base type
• bad channel direction
• bad combinefn

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/3cc3443acb3a2b69. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/schema/BeamTableUtils.java:99

                .mapToObj(idx -> autoCastField(schema.getField(idx), rawRecord.get(idx)))
                .collect(toRow(schema)));
      }
      return rows;
    } catch (IOException e) {
      throw new IllegalArgumentException(
          String.format("Could not parse CSV records from %s with format %s", line, csvFormat), e);
    }
  }

  public static String beamRow2CsvLine(Row row, CSVFormat csvFormat) {
    StringWriter writer = new StringWriter();
    try (CSVPrinter printer = csvFormat.print(writer)) {
      for (int i = 0; i < row.getFieldCount(); i++) {
        printer.print(row.getBaseValue(i, Object.class).toString());
      }
      printer.println();
    } catch (IOException e) {
      throw new IllegalArgumentException("encodeRecord failed!", e);
    }
    return writer.toString();
  }

  /**
   * Attempt to cast an object to a specified Schema.Field.Type.
   *
   * @throws IllegalArgumentException if the value cannot be cast to that type.
   * @return The casted object in Schema.Field.Type.
   */
  public static Object autoCastField(Schema.Field field, @Nullable Object rawObj) {
    // handle null
    if (rawObj == null) {
      if (!field.getType().getNullable()) {
        throw new IllegalArgumentException(String.format("Field %s not nullable", field.getName()));
      }
      return null;
    }

View on GitHub (pinned to 12126d8942)