apache/beam · error · CoderException

cannot encode a null String

Error message

cannot encode a null String

What it means

StringUtf8Coder.encode has no encoding for null: it throws CoderException immediately because null String elements are not encodable under this coder. Beam coders generally reject null elements; use NullableCoder.of(StringUtf8Coder.of()) if nulls must round-trip.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/coders/StringUtf8Coder.java:74

    if (len < 0) {
      throw new CoderException("Invalid encoded string length: " + len);
    }
    byte[] bytes = new byte[len];
    ByteStreams.readFully(dis, bytes);
    return new String(bytes, StandardCharsets.UTF_8);
  }

  private StringUtf8Coder() {}

  @Override
  public void encode(String value, OutputStream outStream) throws IOException {
    encode(value, outStream, Context.NESTED);
  }

  @Override
  public void encode(String value, OutputStream outStream, Context context) throws IOException {
    if (value == null) {
      throw new CoderException("cannot encode a null String");
    }
    if (context.isWholeStream) {
      byte[] bytes = value.getBytes(StandardCharsets.UTF_8);
      if (outStream instanceof ExposedByteArrayOutputStream) {
        ((ExposedByteArrayOutputStream) outStream).writeAndOwn(bytes);
      } else {
        outStream.write(bytes);
      }
    } else {
      writeString(value, outStream);
    }
  }

  @Override
  public String decode(InputStream inStream) throws IOException {
    return decode(inStream, Context.NESTED);
  }

View on GitHub (pinned to 12126d8942)

Solutions

1. Filter out or replace null strings before encoding (map null to "" or a sentinel).
2. Wrap the coder: NullableCoder.of(StringUtf8Coder.of()) and set it via setCoder().
3. Make the schema field @Nullable only if the pipeline uses a null-capable coder.
4. Fail fast upstream with a clear domain error so nulls are caught early.

Example fix

// before
StringUtf8Coder.of().encode(value, out, Context.OUTER);
// after
if (value == null) { value = ""; } // or use NullableCoder.of(StringUtf8Coder.of())
StringUtf8Coder.of().encode(value, out, Context.OUTER);

if (value == null) {
  throw new IllegalArgumentException("Cannot encode null String with StringUtf8Coder; provide a default or use NullableCoder");
}

static boolean encodable(String s) { return s != null; }

try {
  StringUtf8Coder.of().encode(value, out, ctx);
} catch (CoderException e) {
  throw new SerializationException("Null or unencodable String", e);
}

When it happens

Trigger: Calling StringUtf8Coder.of().encode(null, out, context) directly; a PCollection containing null Strings whose coder is plain StringUtf8Coder; schema'd rows with null string fields encoded by a non-nullable coder.

Common situations: Upstream transforms emitting nulls (empty lookups, missing JSON fields) flowing into a coded sink; code ported from languages where null strings are normal; data source changes making a previously non-null column nullable.

Related errors

• cannot encode a null Integer
• cannot encode a null ValueKind
• Cannot encode null window
• {errorContext}: unable to encode value {value} using {coder}
• Unable to encode constant members of ParamWindowedValueCoder

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/7db950843fae1fcc. Report an issue: GitHub.