{"record":{"id":"5279f2f0b7c4ec0d","repo":"apache/beam","slug":"key-coder-s-must-be-deterministic","errorCode":null,"errorMessage":"Key coder %s must be deterministic","messagePattern":"Key coder (.+?) must be deterministic","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Watch.java","lineNumber":819,"sourceCode":"        outputKeyFn = (SerializableFunction) SerializableFunctions.identity();\n      } else {\n        if (outputKeyCoder == null) {\n          // If a coder was not specified explicitly, infer it from the OutputT type parameter\n          // of the output key fn.\n          TypeDescriptor<KeyT> keyT = TypeDescriptors.outputOf(getOutputKeyFn());\n          try {\n            outputKeyCoder = input.getPipeline().getCoderRegistry().getCoder(keyT);\n          } catch (CannotProvideCoderException e) {\n            throw new RuntimeException(\n                \"Unable to infer coder for KeyT (\"\n                    + keyT\n                    + \"). Specify it explicitly using withOutputKeyCoder().\");\n          }\n        }\n        try {\n          outputKeyCoder.verifyDeterministic();\n        } catch (Coder.NonDeterministicException e) {\n          throw new IllegalArgumentException(\n              \"Key coder \" + outputKeyCoder + \" must be deterministic\");\n        }\n      }\n\n      PCollection<KV<InputT, List<TimestampedValue<OutputT>>>> polledPc =\n          input\n              .apply(\n                  ParDo.of(new WatchGrowthFn<>(this, outputCoder, outputKeyFn, outputKeyCoder))\n                      .withSideInputs(getPollFn().getRequirements().getSideInputs()))\n              .setCoder(\n                  KvCoder.of(\n                      input.getCoder(),\n                      ListCoder.of(TimestampedValue.TimestampedValueCoder.of(outputCoder))));\n      return polledPc\n          .apply(ParDo.of(new PollResultSplitFn<>()))\n          .setCoder(KvCoder.of(input.getCoder(), outputCoder));\n    }\n  }","sourceCodeStart":801,"sourceCodeEnd":837,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Watch.java#L801-L837","documentation":"Beam requires key coders to be deterministic because keyed data may be shuffled/grouped and must decode identically byte-for-byte. Watch.Growth verifies the output key coder determinism at expansion; if outputKeyCoder.verifyDeterministic() throws Coder.NonDeterministicException, Beam rethrows it as an IllegalArgumentException with this message.","triggerScenarios":"Using Watch.growthOf with an output key function whose KeyT coder is non-deterministic — e.g. a key type using SerializableCoder over a class without stable serialization (HashMap, POJO with unordered fields), or Avro/JSON coders with map fields.","commonSituations":"Choosing POJO or map-based key types for Watch output keys; switching a key type from String to a struct-like class; Beam upgrades tightening determinism checks.","solutions":["Use a deterministic key type (String, Long, byte[]) for the output key","Provide a deterministic custom coder via withOutputKeyCoder() (e.g. an Avro or protobuf-based coder)","Implement Coder.verifyDeterministic() with no exception for a coder you know is stable","Canonicalize keys (e.g. sorted-field serialization) before emitting them"],"exampleFix":"// before\nWatch.growthOf(pollFn).withOutputKeyCoder(SerializableCoder.of(MyPojoKey.class)); // non-deterministic\n// after\nWatch.growthOf(pollFn).withOutputKeyCoder(StringUtf8Coder.of()); // key = MyPojoKey#canonicalString()","handlingStrategy":"validation","validationCode":"try {\n  coder.verifyDeterministic();\n} catch (Coder.NonDeterministicException e) {\n  throw new IllegalStateException(\"Key coder must be deterministic: \" + coder, e);\n}","typeGuard":null,"tryCatchPattern":"try { pc = input.apply(growth); } catch (IllegalArgumentException e) { if (e.getMessage().contains(\"must be deterministic\")) { /* swap key coder */ } }","preventionTips":["Use String/Long/byte[] keys or protobuf/Avro coders for Watch output keys","Never use SerializableCoder for key types with non-canonical serialization","Verify determinism in unit tests before pipeline submission"],"tags":["beam","coder","determinism","watch"],"backgroundTag":"non-deterministic-coder","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}