apache/beam · error · CoderException

Invalid encoded string length: {}

Error message

Invalid encoded string length: {}

What it means

StringUtf8Coder.decode reads a VarInt length prefix followed by that many UTF-8 bytes. A negative decoded length means the stream is corrupt or misaligned — no valid string has negative length, so the coder refuses and throws CoderException. This guards against decoding garbage or decoding at the wrong stream offset.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/coders/StringUtf8Coder.java:57

  public static StringUtf8Coder of() {
    return INSTANCE;
  }

  /////////////////////////////////////////////////////////////////////////////

  private static final StringUtf8Coder INSTANCE = new StringUtf8Coder();
  private static final TypeDescriptor<String> TYPE_DESCRIPTOR = new TypeDescriptor<String>() {};

  private static void writeString(String value, OutputStream dos) throws IOException {
    byte[] bytes = value.getBytes(StandardCharsets.UTF_8);
    VarInt.encode(bytes.length, dos);
    dos.write(bytes);
  }

  private static String readString(InputStream dis) throws IOException {
    int len = VarInt.decodeInt(dis);
    if (len < 0) {
      throw new CoderException("Invalid encoded string length: " + len);
    }
    byte[] bytes = new byte[len];
    ByteStreams.readFully(dis, bytes);
    return new String(bytes, StandardCharsets.UTF_8);
  }

  private StringUtf8Coder() {}

  @Override
  public void encode(String value, OutputStream outStream) throws IOException {
    encode(value, outStream, Context.NESTED);
  }

  @Override
  public void encode(String value, OutputStream outStream, Context context) throws IOException {
    if (value == null) {
      throw new CoderException("cannot encode a null String");
    }

View on GitHub (pinned to 12126d8942)

Solutions

1. Ensure the stream was written by StringUtf8Coder.encode(), including its VarInt length prefix.
2. Verify the decode offset: start at the exact position where encoding began.
3. Regenerate or re-serialize data with the same Beam/coder version that wrote it.
4. Catch CoderException and log/dump the offending bytes to diagnose corruption.

Example fix

// before
String s = StringUtf8Coder.of().decode(rawUtf8Stream);
// after
ByteArrayOutputStream bos = new ByteArrayOutputStream();
StringUtf8Coder.of().encode(myString, bos);
String s = StringUtf8Coder.of().decode(new ByteArrayInputStream(bos.toByteArray()));

in.mark(1);
int first = in.read();
in.reset();
if (first == -1) throw new IllegalStateException("Empty stream, not a StringUtf8-encoded value");

static boolean looksLikeLengthPrefixed(byte[] data) {
  return data != null && data.length >= 1;
}

try {
  String s = StringUtf8Coder.of().decode(inStream);
} catch (CoderException e) {
  log.error("Corrupt or misaligned string encoding", e);
  throw new DataCorruptionException(e);
}

When it happens

Trigger: Decoding a byte stream not produced by StringUtf8Coder (e.g. raw UTF-8 bytes without the VarInt length prefix); decoding from the wrong offset in a concatenated or multiplexed stream; truncated or bit-rotted data where VarInt bytes decode to a negative value.

Common situations: Mixing coder versions between pipeline stages; replaying old serialized data written in a different format; hand-crafted test byte arrays missing the length prefix; seeking into the middle of an encoded stream.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

• error when decoding a textual integer
• EOF encountered decoding a ValueKind
• Unknown ValueKind number: {}
• Unable to decode constant members from payload for ParamWind
• wrong decoding sequence found.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/2ca7b92150279092. Report an issue: GitHub.