apache/beam · error · RuntimeException

Unable to get key coder

Error message

Unable to get key coder

What it means

OrderedEventProcessor.expand() asks the ComposedAccumulatingProcessingHandler for a Coder for the key type EventKeyT. If the handler cannot infer it from the input PCollection's KvCoder (CannotProvideCoderException), expansion fails with this RuntimeException. Beam requires concrete coders to serialize data between stages, so expansion aborts early rather than failing at runtime.

Solutions

1. Ensure the input PCollection uses a KvCoder, e.g. apply("setKeyCoder", Keys IGNORED) — actually wrap with PCollection.is/applyCoder: use PCollectionLists or KvCoder.of(keyCoder, eventCoder) via setCoder() on the input before expansion.
2. Register a CoderProvider for your custom key type via CoderRegistry.registerCoderProvider or annotate with @DefaultCoder.
3. Implement getKeyCoder in your handler subclass to supply an explicit coder instead of relying on inference.
4. Check for generics erasure: capture the concrete key TypeDescriptor when building the pipeline.

Example fix

// before
PCollection<KV<MyKey, Event>> input = events;
processor.expand(input); // CannotProvideCoderException for MyKey

// after
PCollection<KV<MyKey, Event>> input = events.setCoder(KvCoder.of(MyKeyCoder.of(), EventCoder.of()));
processor.expand(input);

if (!(input.getCoder() instanceof KvCoder)) {
  throw new IllegalArgumentException("input must be coded with KvCoder<K, V>; set it via input.setCoder(...)");
}
KvCoder<?, ?> kv = (KvCoder<?, ?>) input.getCoder();
if (kv.getKeyCoder() == null) { throw new IllegalArgumentException("missing key coder"); }

static boolean hasKeyCoder(PCollection<?> pc) {
  return pc.getCoder() instanceof KvCoder && ((KvCoder<?, ?>) pc.getCoder()).getKeyCoder() != null;
}

try {
  pipeline.apply(processor);
} catch (RuntimeException e) {
  if (e.getCause() instanceof CannotProvideCoderException) {
    throw new IllegalStateException("Register a CoderProvider or set an explicit KvCoder on the input", e);
  }
  throw e;
}

When it happens

Trigger: Applying OrderedEventProcessor.expand() to a PCollection<KV<EventKeyT, EventT>> whose key coder cannot be inferred — e.g. the input was created without a key coder, the key type's type token is erased, or no registered CoderProvider exists for the key type.

Common situations: Using a custom key class without a CoderProvider registered (CoderRegistry not extended); creating the input with Create.of and generics erasure losing the exact key type; using a lambda/anonymous type whose type descriptor cannot be resolved.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

• Unable to get event coder
• ApproximateUnique.PerKey requires its input to use KvCoder
• cannot encode a null BitSet
• cannot encode a null byte[]
• cannot encode a null Integer

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/668104a80fc61f25. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/extensions/ordered/src/main/java/org/apache/beam/sdk/extensions/ordered/OrderedEventProcessor.java:116

  @Override
  public OrderedEventProcessorResult<EventKeyT, ResultT, EventT> expand(
      PCollection<KV<EventKeyT, KV<Long, EventT>>> input) {
    final TupleTag<KV<EventKeyT, ResultT>> mainOutput =
        new TupleTag<KV<EventKeyT, ResultT>>("mainOutput") {};
    final TupleTag<KV<EventKeyT, OrderedProcessingStatus>> statusOutput =
        new TupleTag<KV<EventKeyT, OrderedProcessingStatus>>("status") {};

    final TupleTag<KV<EventKeyT, KV<Long, UnprocessedEvent<EventT>>>> unprocessedEventOutput =
        new TupleTag<KV<EventKeyT, KV<Long, UnprocessedEvent<EventT>>>>("unprocessed-events") {};

    OrderedProcessingHandler<EventT, EventKeyT, StateT, ResultT> handler = getHandler();
    Pipeline pipeline = input.getPipeline();

    Coder<EventKeyT> keyCoder;
    try {
      keyCoder = handler.getKeyCoder(pipeline, input.getCoder());
    } catch (CannotProvideCoderException e) {
      throw new RuntimeException("Unable to get key coder", e);
    }

    Coder<EventT> eventCoder;
    try {
      eventCoder = handler.getEventCoder(pipeline, input.getCoder());
    } catch (CannotProvideCoderException e) {
      throw new RuntimeException("Unable to get event coder", e);
    }

    Coder<StateT> stateCoder;
    try {
      stateCoder = handler.getStateCoder(pipeline);
    } catch (CannotProvideCoderException e) {
      throw new RuntimeException("Unable to get state coder", e);
    }

    Coder<ResultT> resultCoder;
    try {

View on GitHub (pinned to 12126d8942)