apache/beam · error · CoderException

cannot encode a null SolrDocument

Error message

cannot encode a null SolrDocument

What it means

JavaBinCodecCoder serializes SolrDocument values using Solr's JavaBinCodec, which cannot represent null documents. The coder explicitly rejects null input with a CoderException during encode to keep the length-prefixed wire format valid.

Source

Thrown at sdks/java/io/solr/src/main/java/org/apache/beam/sdk/io/solr/JavaBinCodecCoder.java:56

import org.apache.solr.common.SolrInputDocument;
import org.apache.solr.common.util.JavaBinCodec;

/** A {@link Coder} that encodes using {@link JavaBinCodec}. */
class JavaBinCodecCoder<T> extends AtomicCoder<T> {
  private final Class<T> clazz;

  private JavaBinCodecCoder(Class<T> clazz) {
    this.clazz = clazz;
  }

  public static <T> JavaBinCodecCoder<T> of(Class<T> clazz) {
    return new JavaBinCodecCoder<>(clazz);
  }

  @Override
  public void encode(T value, OutputStream outStream) throws IOException {
    if (value == null) {
      throw new CoderException("cannot encode a null SolrDocument");
    }

    ByteArrayOutputStream baos = new ByteArrayOutputStream();
    JavaBinCodec codec = new JavaBinCodec();
    codec.marshal(value, baos);

    byte[] bytes = baos.toByteArray();
    VarInt.encode(bytes.length, outStream);
    outStream.write(bytes);
  }

  @Override
  public T decode(InputStream inStream) throws IOException {
    DataInputStream in = new DataInputStream(inStream);

    int len = VarInt.decodeInt(in);
    if (len < 0) {
      throw new CoderException("Invalid encoded SolrDocument length: " + len);

View on GitHub (pinned to 12126d8942)

Solutions

1. Filter null elements before the write: PCollection.filter(doc -> doc != null)
2. Fix the upstream transform that produces null SolrDocuments
3. If nulls are meaningful, map them to an empty SolrDocument or a sentinel before the sink

Example fix

// before
pipeline.apply(SolrIO.write(solrIO)); // collection may contain nulls
// after
input.apply("DropNullDocs", Filter.by(doc -> doc != null))
     .apply(SolrIO.write(solrIO));

if (doc == null) {
  throw new IllegalArgumentException("SolrDocument must not be null before encode");
}

boolean isValidDoc(SolrDocument d) { return d != null; }

try {
  coder.encode(doc, out);
} catch (CoderException e) {
  LOG.error("Null or invalid SolrDocument passed to JavaBinCodecCoder", e);
  throw e;
}

When it happens

Trigger: A PCollection fed to SolrIO.write() contains a null element, and Beam invokes JavaBinCodecCoder.encode(null, out) while materializing or serializing the record between stages.

Common situations: Upstream DoFns or joins emitting null documents instead of dropping them; accidental null entries after parsing/transform steps; deserialization of sparse data producing nulls before the Solr sink.

Related errors

• cannot encode a null HyperLogLogPlus sketch
• cannot encode a null Count-min Sketch
• cannot encode a null T-Digest sketch
• Invalid encoded SolrDocument length: ${len}
• cannot encode a null PredictionResult

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/091b9c2eb46a4ab9. Report an issue: GitHub.