apache/beam · error · IllegalArgumentException

Unable to encode element '" + value + "' with coder '" + thi

Error message

Unable to encode element '" + value + "' with coder '" + this + "'.

What it means

Coder.structuralValue falls back to encoding the value to bytes when the coder does not override structuralValue; if that encode throws, it rethrows IllegalArgumentException naming the value and coder. The real cause is the nested exception, typically a null or unsupported value for the coder.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/coders/Coder.java:268

   *       {@code equals()} method, even if the input value is {@code null}.
   * </ul>
   *
   * <p>See also {@link #consistentWithEquals()}.
   *
   * <p>By default, if this coder is {@link #consistentWithEquals()}, and the value is not null,
   * returns the provided object. Otherwise, encodes the value into a {@code byte[]}, and returns an
   * object that performs array equality on the encoded bytes.
   */
  public Object structuralValue(T value) {
    if (value != null && consistentWithEquals()) {
      return value;
    } else {
      try {
        ByteArrayOutputStream os = new ByteArrayOutputStream();
        encode(value, os, Context.OUTER);
        return new StructuralByteArray(os.toByteArray());
      } catch (Exception exn) {
        throw new IllegalArgumentException(
            "Unable to encode element '" + value + "' with coder '" + this + "'.", exn);
      }
    }
  }

  /**
   * Returns whether {@link #registerByteSizeObserver} cheap enough to call for every element, that
   * is, if this {@code Coder} can calculate the byte size of the element to be coded in roughly
   * constant time (or lazily).
   *
   * <p>Not intended to be called by user code, but instead by {@link PipelineRunner}
   * implementations.
   *
   * <p>By default, returns false. The default {@link #registerByteSizeObserver} implementation
   * invokes {@link #getEncodedElementByteSize} which requires re-encoding an element unless it is
   * overridden. This is considered expensive.
   */
  public boolean isRegisterByteSizeObserverCheap(T value) {

View on GitHub (pinned to 12126d8942)

Solutions

1. Inspect getCause() to find the real encode failure (often CoderException for null)
2. Filter/normalize null or unsupported elements before they reach keyed or side-input paths
3. Use a coder that supports the values (e.g. NullableCoder) or override structuralValue in a custom coder
4. Fix upstream data so all elements are encodable

Example fix

// before
KV<Byte, String> kv = KV.of(nullableByte, v); // structuralValue throws IAE
// after
KV<Byte, String> kv = KV.of(nullableByte == null ? (byte) 0 : nullableByte, v);

// pre-encode to verify encodability
ByteArrayOutputStream test = new ByteArrayOutputStream();
coder.encode(value, test, Coder.Context.OUTER);

static <T> boolean encodable(Coder<T> c, T v) { try { c.encode(v, new ByteArrayOutputStream(), Coder.Context.OUTER); return true; } catch (Exception e) { return false; } }

try { Object sv = coder.structuralValue(v); } catch (IllegalArgumentException e) { LOG.error("value not encodable: " + e.getCause(), e); }

When it happens

Trigger: Calling structuralValue(value) (directly or via keyedValues / side-input helpers) on a value the coder cannot encode — commonly null elements for non-nullable coders like ByteCoder or ByteArrayCoder.

Common situations: Null values in GBK keys, side inputs containing nulls, coders that throw on encode for special values, equality/hashing machinery encountering bad elements.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• cannot encode a null Byte
• cannot estimate size for unsupported null value
• NonDeterministicException(target, message, e)
• Unable to provide coder for %s, this factory can only provid
• Class %s does not have a @DefaultCoder annotation.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/9911747a73d80e48. Report an issue: GitHub.