{"record":{"id":"b519a70c8bd8c3c2","repo":"apache/beam","slug":"cannot-encode-a-null-count-min-sketch","errorCode":null,"errorMessage":"cannot encode a null Count-min Sketch","messagePattern":"cannot encode a null Count-min Sketch","errorType":"exception","errorClass":"CoderException","httpStatus":null,"severity":"error","filePath":"sdks/java/extensions/sketching/src/main/java/org/apache/beam/sdk/extensions/sketching/SketchFrequencies.java","lineNumber":507,"sourceCode":"    }\n\n    /**\n     * Utility class to retrieve the estimate frequency of an element from a {@link CountMinSketch}.\n     */\n    public long estimateCount(T element, Coder<T> coder) {\n      return sketch().estimateCount(hashElement(element, coder));\n    }\n  }\n\n  /** Coder for {@link CountMinSketch} class. */\n  static class CountMinSketchCoder<T> extends CustomCoder<Sketch<T>> {\n\n    private static final ByteArrayCoder BYTE_ARRAY_CODER = ByteArrayCoder.of();\n\n    @Override\n    public void encode(Sketch<T> value, OutputStream outStream) throws IOException {\n      if (value == null) {\n        throw new CoderException(\"cannot encode a null Count-min Sketch\");\n      }\n      BYTE_ARRAY_CODER.encode(CountMinSketch.serialize(value.sketch()), outStream);\n    }\n\n    @Override\n    public Sketch<T> decode(InputStream inStream) throws IOException {\n      byte[] sketchBytes = BYTE_ARRAY_CODER.decode(inStream);\n      CountMinSketch sketch = CountMinSketch.deserialize(sketchBytes);\n      return Sketch.create(sketch);\n    }\n\n    @Override\n    public boolean isRegisterByteSizeObserverCheap(Sketch<T> value) {\n      return true;\n    }\n\n    @Override\n    protected long getEncodedElementByteSize(Sketch<T> value) throws IOException {","sourceCodeStart":489,"sourceCodeEnd":525,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/extensions/sketching/src/main/java/org/apache/beam/sdk/extensions/sketching/SketchFrequencies.java#L489-L525","documentation":"The CountMinSketchCoder's encode() method throws CoderException when asked to encode a null Sketch value. Beam coders must serialize values for shuffle/stage handoff, and this coder deliberately rejects nulls because a null sketch has no serialized representation. The null should never reach encode() in a well-formed pipeline.","triggerScenarios":"A null Sketch<T> value flows into Beam's coder pipeline (e.g. during shuffle of a PCollection of Sketch<T>, GBK output, or state/storage) so Coder.encode is invoked with null — typically when a DoFn emits null sketches or an accumulator is null.","commonSituations":"A CombineFn accumulator path emitting null; user DoFns outputting null sketches instead of empty ones; deserialized/state-backed values that were never initialized.","solutions":["Ensure the code producing Sketch<T> values never emits null — return a new empty CountMinSketch instead.","In CombineFn combine/extractOutput, guard against null accumulators and create one via createAccumulator().","If nulls are legitimately possible, filter or map them to empty sketches before the coder boundary.","Catch CoderException at the transform boundary only to aid debugging; it signals a bug, not expected input."],"exampleFix":"// before\ncontext.output(value == null ? null : sketch);\n\n// after\ncontext.output(value == null ? CountMinSketch.builder().build(CODER) : sketch);","handlingStrategy":"validation","validationCode":"if (sketch == null) {\n  sketch = CountMinSketch.builder().build(); // or the fn's createAccumulator()\n}","typeGuard":"boolean isUsableSketch(Sketch<?> s) { return s != null; }","tryCatchPattern":"try {\n  coded.apply(GroupByKey.create());\n} catch (CoderException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"cannot encode a null\")) {\n    throw new IllegalStateException(\"Null sketch emitted upstream — fix producing DoFn/CombineFn\", e);\n  }\n  throw e;\n}","preventionTips":["Treat null sketches as bugs: always initialize accumulators.","Add assertions (Preconditions.checkNotNull) where sketches are produced.","Never output Optional.empty()/null from DoFns feeding coded PCollections."],"tags":["java","beam","coder","null","serialization"],"backgroundTag":"null-argument","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}