{"record":{"id":"e7b9433d4a71d601","repo":"apache/beam","slug":"coder-must-be-deterministic-to-perform-this-sketch-e-e7b943","errorCode":null,"errorMessage":"Coder must be deterministic to perform this sketch.${e.getMessage()}","messagePattern":"Coder must be deterministic to perform this sketch\\.(.+?)","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"sdks/java/extensions/sketching/src/main/java/org/apache/beam/sdk/extensions/sketching/SketchFrequencies.java","lineNumber":365,"sourceCode":"    private CountMinSketchFn(final Coder<InputT> coder, double eps, double confidence) {\n      this.epsilon = eps;\n      this.confidence = confidence;\n      this.width = (int) Math.ceil(2 / eps);\n      this.depth = (int) Math.ceil(-Math.log(1 - confidence) / Math.log(2));\n      this.inputCoder = coder;\n    }\n\n    /**\n     * Returns a {@link CountMinSketchFn} combiner with the given input coder. <br>\n     * <b>Warning :</b> the coder must be deterministic.\n     *\n     * @param coder the coder that encodes the elements' type\n     */\n    public static <InputT> CountMinSketchFn<InputT> create(Coder<InputT> coder) {\n      try {\n        coder.verifyDeterministic();\n      } catch (Coder.NonDeterministicException e) {\n        throw new IllegalArgumentException(\n            \"Coder must be deterministic to perform this sketch.\" + e.getMessage(), e);\n      }\n      return new CountMinSketchFn<>(coder, 0.01, 0.999);\n    }\n\n    /**\n     * Returns a new {@link CountMinSketchFn} combiner with new precision accuracy parameters {@code\n     * epsilon} and {@code confidence}.\n     *\n     * <p>Keep in mind that the lower the {@code epsilon} value, the greater the width, and the\n     * greater the confidence, the greater the depth.\n     *\n     * @param epsilon the error relative to the total number of distinct elements\n     * @param confidence the confidence in the result to not exceed the relative error\n     */\n    public CountMinSketchFn<InputT> withAccuracy(double epsilon, double confidence) {\n      if (epsilon <= 0D) {\n        throw new IllegalArgumentException(\"The relative error must be positive\");","sourceCodeStart":347,"sourceCodeEnd":383,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/extensions/sketching/src/main/java/org/apache/beam/sdk/extensions/sketching/SketchFrequencies.java#L347-L383","documentation":"SketchFrequencies.CountMinSketchFn.create requires a deterministic Coder because Count-Min Sketch hashes the encoded bytes of elements; a non-deterministic encoding would hash equal elements differently and break frequency counting. NonDeterministicException from verifyDeterministic() is converted into this IllegalArgumentException.","triggerScenarios":"Calling SketchFrequencies.perElement().create(coder) with a coder for List/Map/custom POJO types whose verifyDeterministic() throws (e.g. ListCoder, MapCoder, or a default Bean coder).","commonSituations":"Counting frequencies of collections or structs; users reusing default coders without providing a deterministic one; refactor changing element type to a non-deterministic one.","solutions":["Provide a deterministic Coder (AvroCoder, custom Coder with stable field ordering, protobuf coder).","Map elements to a deterministically encodable type (String, Long, byte[]) before the transform.","If your custom coder is genuinely deterministic, override verifyDeterministic() to assert it."],"exampleFix":"// before\n.apply(SketchFrequencies.<Map<String,Integer>>perElement()\n    .create(MapCoder.of(StringUtf8Coder.of(), VarIntCoder.of()))); // throws\n// after\n.apply(MapElements.via(new SimpleFunction<Map<String,Integer>, String>() {\n  public String apply(Map<String,Integer> m) { return m.entrySet().stream()\n      .map(e -> e.getKey()+\":\"+e.getValue()).sorted().collect(Collectors.joining(\",\")); }\n}))\n.apply(SketchFrequencies.<String>perElement().create(StringUtf8Coder.of()));","handlingStrategy":"validation","validationCode":"coder.verifyDeterministic(); // run before create to get a precise failure point","typeGuard":"static boolean isDeterministic(Coder<?> c) { try { c.verifyDeterministic(); return true; } catch (Coder.NonDeterministicException e) { return false; } }","tryCatchPattern":"try { CountMinSketchFn.create(coder); } catch (IllegalArgumentException e) { /* switch to deterministic coder */ }","preventionTips":["Use AvroCoder or protobuf coders for sketch inputs","Avoid ListCoder/MapCoder for elements hashed by byte encoding","Check coder determinism in pipeline unit tests"],"tags":["java","beam","sketching","count-min-sketch","determinism"],"backgroundTag":"invalid-argument-value","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}