apache/beam · error · IllegalArgumentException

{errorContext}: unable to encode value {value} using {coder}

Error message

{errorContext}: unable to encode value {value} using {coder}

What it means

ensureSerializableByCoder round-trips a value through a Coder to verify it is encodable/decodable. A CoderException during encoding is rethrown as IllegalArgumentException '<errorContext>: unable to encode value <value> using <coder>'. It indicates the value does not conform to what the coder expects (e.g. nulls where the coder forbids them, or wrong element type).

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/util/SerializableUtils.java:158

            + "implement serialization correctly.  Before: %s, after: %s",
        coder,
        copy);

    return copy;
  }

  /**
   * Serializes an arbitrary T with the given {@code Coder<T>} and verifies that it can be correctly
   * deserialized.
   */
  public static <T> T ensureSerializableByCoder(Coder<T> coder, T value, String errorContext) {
    byte[] encodedValue;
    try {
      encodedValue = encodeToByteArray(coder, value);
    } catch (CoderException exn) {
      // TODO: Put in better element printing:
      // truncate if too long.
      throw new IllegalArgumentException(
          errorContext + ": unable to encode value " + value + " using " + coder, exn);
    }
    try {
      return decodeFromByteArray(coder, encodedValue);
    } catch (CoderException exn) {
      // TODO: Put in better encoded byte array printing:
      // use printable chars with escapes instead of codes, and
      // truncate if too long.
      throw new IllegalArgumentException(
          errorContext
              + ": unable to decode "
              + Arrays.toString(encodedValue)
              + ", encoding of value "
              + value
              + ", using "
              + coder,
          exn);
    }

View on GitHub (pinned to 12126d8942)

Solutions

1. Fix the value so it matches the coder's contract (remove nulls, correct the element type).
2. Use a coder that supports the value (e.g. NullableCoder to permit nulls).
3. Verify the CoderRegistry/coder inference selected the right coder for the PCollection's element type.
4. If you own a custom coder, fix its encode() to handle the failing value or throw a clearer CoderException.

Example fix

// before
Coder<String> c = StringUtf8Coder.of();
ensureSerializableByCoder("check", maybeNull, c); // throws on null
// after
Coder<String> c = NullableCoder.of(StringUtf8Coder.of());
ensureSerializableByCoder("check", maybeNull, c);

if (value == null && !(coder instanceof NullableCoder)) { throw new IllegalArgumentException("coder does not support nulls"); }

static <T> boolean coderAccepts(Coder<T> c, T v) { return v != null || c instanceof NullableCoder; }

try { ensureSerializableByCoder("context", value, coder); } catch (IllegalArgumentException e) { LOG.error("coder mismatch: {}", e.getMessage()); throw e; }

When it happens

Trigger: Calling ensureSerializableByCoder(context, value, coder) where encodeToByteArray throws CoderException — e.g. encoding null with a coder that rejects nulls, or an element that does not match the coder's declared type.

Common situations: Pipeline construction-time validation of DoFn inputs/outputs; feeding null or mismatched elements into PCollections whose coders were inferred for a different type; custom coders that fail on edge-case values.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

• cannot encode a null String
• cannot encode a null Integer
• cannot encode a null ValueKind
• Unable to encode constant members of ParamWindowedValueCoder
• cannot encode a null Count-min Sketch

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/e98f664dc7f43e05. Report an issue: GitHub.