apache/beam · error · CoderException

cannot encode a null HyperLogLogPlus sketch

Error message

cannot encode a null HyperLogLogPlus sketch

What it means

HyperLogLogPlusCoder.encode refuses to encode a null HyperLogLogPlus value, throwing CoderException. Coders in Beam may legally receive null in some contexts, but this coder chose to treat null as an encoding error rather than writing a null marker.

Source

Thrown at sdks/java/extensions/sketching/src/main/java/org/apache/beam/sdk/extensions/sketching/ApproximateDistinct.java:508

          .add(DisplayData.item("sp", sp).withLabel("sparse representation precision"));
    }
  }

  /** Coder for {@link HyperLogLogPlus} class. */
  public static class HyperLogLogPlusCoder extends CustomCoder<HyperLogLogPlus> {

    private static final HyperLogLogPlusCoder INSTANCE = new HyperLogLogPlusCoder();

    private static final ByteArrayCoder BYTE_ARRAY_CODER = ByteArrayCoder.of();

    public static HyperLogLogPlusCoder of() {
      return INSTANCE;
    }

    @Override
    public void encode(HyperLogLogPlus value, OutputStream outStream) throws IOException {
      if (value == null) {
        throw new CoderException("cannot encode a null HyperLogLogPlus sketch");
      }
      BYTE_ARRAY_CODER.encode(value.getBytes(), outStream);
    }

    @Override
    public HyperLogLogPlus decode(InputStream inStream) throws IOException {
      return HyperLogLogPlus.Builder.build(BYTE_ARRAY_CODER.decode(inStream));
    }

    @Override
    public boolean isRegisterByteSizeObserverCheap(HyperLogLogPlus value) {
      return true;
    }

    @Override
    protected long getEncodedElementByteSize(HyperLogLogPlus value) throws IOException {
      if (value == null) {
        throw new CoderException("cannot encode a null HyperLogLogPlus sketch");

View on GitHub (pinned to 12126d8942)

Solutions

1. Ensure no null sketches are emitted: return an empty/new HyperLogLogPlus instead of null from combining or mapping code.
2. Filter nulls before the coded PCollection is materialized (Filter.by(Objects::nonNull)).
3. Wrap with a nullable-tolerant coder if nulls are semantically valid in your pipeline.

Example fix

// before
return acc == null ? null : acc; // null reaches coder
// after
return acc == null ? new HyperLogLogPlus(12, 0) : acc;

PCollection<HyperLogLogPlus> safe = sketches.apply(Filter.by(java.util.Objects::nonNull));

static boolean isNonNullSketch(HyperLogLogPlus s) { return s != null; }

try { BYTE_ARRAY_CODER.encode(value.getBytes(), out); } catch (CoderException e) { /* null sketch leaked from upstream */ throw e; }

When it happens

Trigger: Encoding a PCollection<HyperLogLogPlus> (or accumulator output of ApproximateDistinct) that contains a null sketch — e.g. a null accumulator produced upstream and then serialized.

Common situations: User code returning null from a DoFn/CombineFn that outputs sketches directly to a PCollection, then materializing/writing that PCollection.

Related errors

• The input value cannot be encoded: ${e.getMessage()}
• The input value cannot be encoded: ${e.getMessage()}
• cannot encode a null Count-min Sketch
• cannot encode a null T-Digest sketch
• cannot encode a null SolrDocument

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/ec97806706dffff9. Report an issue: GitHub.