{"record":{"id":"3e7d3b9bff6ba2f5","repo":"apache/beam","slug":"s-requires-a-deterministic-key-coder-in-order-to-use-state","errorCode":null,"errorMessage":"%s requires a deterministic key coder in order to use state and timers, the reason is:%n %s","messagePattern":"(.+?) requires a deterministic key coder in order to use state and timers, the reason is:%n (.+?)","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/ParDo.java","lineNumber":447,"sourceCode":"      } catch (IllegalAccessException e) {\n        throw new RuntimeException(e);\n      }\n    }\n  }\n\n  private static void validateStateApplicableForInput(PCollection<?> input) {\n    Coder<?> inputCoder = input.getCoder();\n    checkArgument(\n        inputCoder instanceof KvCoder,\n        \"%s requires its input to use %s in order to use state and timers.\",\n        ParDo.class.getSimpleName(),\n        KvCoder.class.getSimpleName());\n\n    KvCoder<?, ?> kvCoder = (KvCoder<?, ?>) inputCoder;\n    try {\n      kvCoder.getKeyCoder().verifyDeterministic();\n    } catch (Coder.NonDeterministicException exc) {\n      throw new IllegalArgumentException(\n          String.format(\n              \"%s requires a deterministic key coder in order to use state and timers, the reason is:%n %s\",\n              ParDo.class.getSimpleName(), exc.getMessage()));\n    }\n  }\n\n  private static void validateSideInputTypes(\n      Map<String, PCollectionView<?>> sideInputs, DoFn<?, ?> fn) {\n    DoFnSignature signature = DoFnSignatures.getSignature(fn.getClass());\n    DoFnSignature.ProcessElementMethod processElementMethod = signature.processElement();\n    for (SideInputParameter sideInput : processElementMethod.getSideInputParameters()) {\n      PCollectionView<?> view = sideInputs.get(sideInput.sideInputId());\n      checkArgument(\n          view != null,\n          \"the ProcessElement method expects a side input identified with the tag %s, but no such side input was\"\n              + \" supplied. Use withSideInput(String, PCollectionView) to supply this side input.\",\n          sideInput.sideInputId());\n      TypeDescriptor<?> viewType = view.getViewFn().getTypeDescriptor();","sourceCodeStart":429,"sourceCodeEnd":465,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/ParDo.java#L429-L465","documentation":"ParDo requires that when a DoFn uses state or timers, the input PCollection's key coder must be deterministic, since state is keyed and runners rely on consistent byte encoding of keys. If KvCoder.getKeyCoder().verifyDeterministic() fails, the pipeline cannot be validated and this exception is thrown with the non-determinism reason.","triggerScenarios":"Applying a ParDo whose DoFn declares @StateId/@TimerId (or TimerFamily) over an input PCollection whose key coder is non-deterministic, e.g. the default coder for a custom key class or a coder over double/Map/unordered types.","commonSituations":"Grouping/stateful aggregation keyed by custom POJOs using the default Java serialization coder; keys typed as double or nested maps; Avro/POJO coders that don't guarantee stable field order.","solutions":["Make the key coder deterministic: implement a custom deterministic Coder for the key class and set it via input.setCoder(KvCoder.of(deterministicKeyCoder, valueCoder))","Use a structurally ordered key type (e.g. String, Long) or serialize the key canonically (sorted fields, fixed-width encodings)","Restructure the pipeline to use a deterministic surrogate key"],"exampleFix":"// before\nPCollection<KV<MyKey, V>> kv = ...; // MyKey uses default java serialization coder\n// after\nkv.setCoder(KvCoder.of(new DeterministicMyKeyCoder(), vCoder));","handlingStrategy":"validation","validationCode":"if (input.getCoder() instanceof KvCoder) {\n  try {\n    ((KvCoder<?, ?>) input.getCoder()).getKeyCoder().verifyDeterministic();\n  } catch (Coder.NonDeterministicException e) {\n    throw new IllegalArgumentException(\"Key coder not deterministic: \" + e.getMessage());\n  }\n}","typeGuard":"boolean hasDeterministicKeyCoder(PCollection<?> input) {\n  return input.getCoder() instanceof KvCoder\n      && ((KvCoder<?, ?>) input.getCoder()).getKeyCoder() instanceof DeterministicCoder;\n}","tryCatchPattern":"try {\n  kv.apply(ParDo.of(statefulFn));\n} catch (IllegalArgumentException e) {\n  if (e.getMessage().contains(\"deterministic key coder\")) {\n    kv.setCoder(KvCoder.of(new DeterministicKeyCoder(), valueCoder));\n    kv.apply(ParDo.of(statefulFn));\n  } else { throw e; }\n}","preventionTips":["Run verifyDeterministic() on key coders in unit tests for stateful pipelines","Prefer String/Long/byte[] keys for stateful DoFns","Write explicit deterministic Coders for custom key classes instead of relying on defaults"],"tags":["java","apache-beam","state","determinism"],"backgroundTag":"invalid-argument-value","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}