{"record":{"id":"c6aaa30884d1388a","repo":"apache/beam","slug":"coder-must-be-deterministic-to-perform-this-sketch-e","errorCode":null,"errorMessage":"Coder must be deterministic to perform this sketch.${e.getMessage()}","messagePattern":"Coder must be deterministic to perform this sketch\\.(.+?)","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"sdks/java/extensions/sketching/src/main/java/org/apache/beam/sdk/extensions/sketching/ApproximateDistinct.java","lineNumber":406,"sourceCode":"\n    private final Coder<InputT> inputCoder;\n\n    private ApproximateDistinctFn(int p, int sp, Coder<InputT> coder) {\n      this.p = p;\n      this.sp = sp;\n      inputCoder = coder;\n    }\n\n    /**\n     * Returns an {@link ApproximateDistinctFn} combiner with the given input coder.\n     *\n     * @param coder the coder that encodes the elements' type\n     */\n    public static <InputT> ApproximateDistinctFn<InputT> create(Coder<InputT> coder) {\n      try {\n        coder.verifyDeterministic();\n      } catch (Coder.NonDeterministicException e) {\n        throw new IllegalArgumentException(\n            \"Coder must be deterministic to perform this sketch.\" + e.getMessage(), e);\n      }\n      return new ApproximateDistinctFn<>(12, 0, coder);\n    }\n\n    /**\n     * Returns an {@link ApproximateDistinctFn} combiner with a new precision {@code p}.\n     *\n     * <p>Keep in mind that {@code p} cannot be lower than 4, because the estimation would be too\n     * inaccurate.\n     *\n     * <p>See {@link ApproximateDistinct#precisionForRelativeError(double)} and {@link\n     * ApproximateDistinct#relativeErrorForPrecision(int)} to have more information about the\n     * relationship between precision and relative error.\n     *\n     * @param p the precision value for the normal representation\n     */\n    public ApproximateDistinctFn<InputT> withPrecision(int p) {","sourceCodeStart":388,"sourceCodeEnd":424,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/extensions/sketching/src/main/java/org/apache/beam/sdk/extensions/sketching/ApproximateDistinct.java#L388-L424","documentation":"ApproximateDistinct.ApproximateDistinctFn.create requires the input Coder to be deterministic because HyperLogLog sketches hash the encoded bytes of elements; non-deterministic encodings would give different hashes for equal elements and corrupt the estimate. create calls coder.verifyDeterministic() and converts NonDeterministicException into this IllegalArgumentException.","triggerScenarios":"Calling ApproximateDistinct.globally().withRepresentativeValueFn(...).create(coder) — i.e. ApproximateDistinctFn.create — with a coder like for a List, Map, or custom type whose verifyDeterministic() throws (e.g. new ListCoder(nonDeterministicCoder)).","commonSituations":"Sketching collections, structs, or POJOs whose default coders (ListCoder, MapCoder, Java Bean coders with nulls) are documented non-deterministic; users forgetting to register a custom deterministic coder.","solutions":["Supply a deterministic coder for the input type (e.g. wrap elements in AvroCoder or implement a custom deterministic Coder).","Pre-map elements to a deterministically encodable type (String, byte[], Long) before applying ApproximateDistinct.","If confident your coder is deterministic despite the default check, override verifyDeterministic() in your custom coder to declare determinism."],"exampleFix":"// before\nPCollection<List<String>> words = ...;\nwords.apply(ApproximateDistinct.<List<String>>globally()\n    .create(ListCoder.of(StringUtf8Coder.of()))); // throws: ListCoder non-deterministic\n// after\nwords.apply(MapElements.via(new SimpleFunction<List<String>, String>() {\n  public String apply(List<String> in) { return String.join(\"\\u0001\", in); }\n}))\n .apply(ApproximateDistinct.<String>globally().create(StringUtf8Coder.of()));","handlingStrategy":"validation","validationCode":"// Before calling create, verify determinism yourself\ncoder.verifyDeterministic(); // throws Coder.NonDeterministicException if not","typeGuard":"static boolean isDeterministic(Coder<?> c) { try { c.verifyDeterministic(); return true; } catch (Coder.NonDeterministicException e) { return false; } }","tryCatchPattern":"try { ApproximateDistinctFn.create(coder); } catch (IllegalArgumentException e) { /* fall back to a deterministic representation */ }","preventionTips":["Prefer AvroCoder/protobuf coders for complex element types used in sketches","Avoid sketching List/Map/Bean-typed PCollections directly","Document deterministic-coder requirements next to sketch usage"],"tags":["java","beam","sketching","hyperloglog","determinism"],"backgroundTag":"invalid-argument-value","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}