apache/beam · error · CoderException

Invalid encoded SolrDocument length: ${len}

Error message

Invalid encoded SolrDocument length: ${len}

What it means

JavaBinCodecCoder prefixes each encoded SolrDocument with a VarInt length. During decode, a negative length means the encoded bytes are corrupt or were produced by an incompatible coder, so decoding cannot proceed and a CoderException is thrown.

Source

Thrown at sdks/java/io/solr/src/main/java/org/apache/beam/sdk/io/solr/JavaBinCodecCoder.java:74

      throw new CoderException("cannot encode a null SolrDocument");
    }

    ByteArrayOutputStream baos = new ByteArrayOutputStream();
    JavaBinCodec codec = new JavaBinCodec();
    codec.marshal(value, baos);

    byte[] bytes = baos.toByteArray();
    VarInt.encode(bytes.length, outStream);
    outStream.write(bytes);
  }

  @Override
  public T decode(InputStream inStream) throws IOException {
    DataInputStream in = new DataInputStream(inStream);

    int len = VarInt.decodeInt(in);
    if (len < 0) {
      throw new CoderException("Invalid encoded SolrDocument length: " + len);
    }

    JavaBinCodec codec = new JavaBinCodec();
    return (T) codec.unmarshal(new BoundedInputStream(in, len));
  }

  @Override
  public TypeDescriptor<T> getEncodedTypeDescriptor() {
    return TypeDescriptor.of(clazz);
  }

  @AutoService(CoderProviderRegistrar.class)
  public static class Provider implements CoderProviderRegistrar {
    @Override
    public List<CoderProvider> getCoderProviders() {
      return Arrays.asList(
          CoderProviders.forCoder(
              TypeDescriptor.of(SolrDocument.class), JavaBinCodecCoder.of(SolrDocument.class)),

View on GitHub (pinned to 12126d8942)

Solutions

1. Regenerate the corrupted state/checkpoint data or rerun the pipeline from the source
2. Ensure the same Beam SDK and SolrIO version are used to encode and decode (avoid mixing versions in a cluster)
3. Verify the coder registered for the PCollection is actually JavaBinCodecCoder and not wrapped/mismatched

int len = VarInt.decodeInt(in);
if (len < 0) {
  throw new CoderException("Invalid encoded SolrDocument length: " + len);
}

try {
  return coder.decode(in);
} catch (CoderException e) {
  LOG.error("Corrupt or incompatible encoded SolrDocument bytes", e);
  throw e;
}

When it happens

Trigger: Decoding a byte stream whose VarInt header is negative — typically caused by corrupt/missing bytes, a coder version mismatch between writer and reader, or reading a stream offset misaligned with the length-prefixed framing.

Common situations: Stale checkpoints or state files written by a different Beam/SolrIO version; data corruption in transit; mixing JavaBinCodecCoder output with another coder's bytes.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• Invalid encoded string length: {}
• cannot encode a null SolrDocument
• Expected 0 or 1, got %d
• invalid length " + length
• cannot encode a null String

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/8ae8fa66505323cf. Report an issue: GitHub.