apache/beam · error · NotSerializableException

Cannot serialize the CombineFn resulting from CombineFnUtil.

Error message

Cannot serialize the CombineFn resulting from CombineFnUtil.bindContext.

What it means

CombineFnUtil.bindContext wraps a SerializableFunction<Context, CombineFn> in a BoundContext CombineFn whose context binding is inherently non-serializable, so its writeObject deliberately throws NotSerializableException. Instances of this wrapper must be created lazily/deserialized via the surrounding serialized lambda/function mechanism, never Java-serialized directly.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/util/CombineFnUtil.java:174

    public Coder<AccumT> getAccumulatorCoder(CoderRegistry registry, Coder<InputT> inputCoder)
        throws CannotProvideCoderException {
      return combineFn.getAccumulatorCoder(registry, inputCoder);
    }

    @Override
    public Coder<OutputT> getDefaultOutputCoder(CoderRegistry registry, Coder<InputT> inputCoder)
        throws CannotProvideCoderException {
      return combineFn.getDefaultOutputCoder(registry, inputCoder);
    }

    @Override
    public void populateDisplayData(DisplayData.Builder builder) {
      combineFn.populateDisplayData(builder);
    }

    private void writeObject(@SuppressWarnings("unused") ObjectOutputStream out)
        throws IOException {
      throw new NotSerializableException(
          "Cannot serialize the CombineFn resulting from CombineFnUtil.bindContext.");
    }
  }
}

View on GitHub (pinned to 12126d8942)

Solutions

1. Serialize the underlying SerializableFunction (context -> CombineFn) instead of the bound wrapper, and call CombineFnUtil.bindContext after deserialization
2. Store the unbound CombineFn/lambda in serializable fields and bind the context at runtime inside the process method
3. Use the public Contextful API (CombineFnBase.Contextful / Contextful.of) which handles serialization of fn + context requirements
4. If a runner fails, check whether it java-serializes functions; configure it to use the supported Beam function serialization path

Example fix

// before
private final CombineFn<MyIn, MyAcc, MyOut> fn =
    CombineFnUtil.bindContext(myFnSupplier, realContext); // not serializable
// after
private final SerializableFunction<Context, CombineFn<MyIn, MyAcc, MyOut>> supplier =
    ctx -> CombineFnUtil.bindContext(myFnSupplier, ctx); // bind at runtime, supplier is serializable

if (fn instanceof BoundContext) {
  throw new IllegalStateException("Do not serialize the bound CombineFn; serialize the supplier");
}

boolean isSerializableFn(Object o) {
  return o instanceof Serializable && !(o.getClass().getSimpleName().equals("BoundContext"));
}

try (ObjectOutputStream out = new ObjectOutputStream(baos)) {
  out.writeObject(fn);
} catch (NotSerializableException e) {
  // serialize the underlying SerializableFunction and rebind context instead
}

When it happens

Trigger: Java-serializing the CombineFn returned by CombineFnUtil.bindContext — e.g. when a runner or DoFn graph serializes the object directly, or user code puts the bound CombineFn into an ObjectOutputStream, Kryo-with-java-serialization, or serialized closure capture.

Common situations: Serializing pipeline closures that captured the bound CombineFn by value; runners or test harnesses deep-copying PTransforms via Java serialization; caching the bound CombineFn in a serializable field.

Related errors

• Cannot provide SerializableCoder because {} does not impleme
• Java Serialization may be non-deterministic.
• cannot encode a null String
• cannot encode a null Integer
• cannot encode a null ValueKind

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/023945f465ac8534. Report an issue: GitHub.