{"record":{"id":"ed74176dfec242e3","repo":"apache/beam","slug":"unexpected-extra-bytes-after-decoding","errorCode":null,"errorMessage":" unexpected extra bytes after decoding ","messagePattern":" unexpected extra bytes after decoding ","errorType":"exception","errorClass":"CoderException","httpStatus":null,"severity":"error","filePath":"sdks/java/core/src/main/java/org/apache/beam/sdk/util/CoderUtils.java","lineNumber":104,"sourceCode":"      coder.encode(value, new UnownedOutputStream(stream), context);\n    } catch (IOException exn) {\n      Throwables.propagateIfPossible(exn, CoderException.class);\n      throw new IllegalArgumentException(\"Forbidden IOException when writing to OutputStream\", exn);\n    }\n  }\n\n  /** Decodes the given bytes using the specified Coder, and returns the resulting decoded value. */\n  public static <T> T decodeFromByteArray(Coder<T> coder, byte[] encodedValue)\n      throws CoderException {\n    return decodeFromByteArray(coder, encodedValue, Coder.Context.OUTER);\n  }\n\n  public static <T> T decodeFromByteArray(\n      Coder<T> coder, byte[] encodedValue, Coder.Context context) throws CoderException {\n    try (ExposedByteArrayInputStream stream = new ExposedByteArrayInputStream(encodedValue)) {\n      T result = decodeFromSafeStream(coder, stream, context);\n      if (stream.available() != 0) {\n        throw new CoderException(\n            stream.available() + \" unexpected extra bytes after decoding \" + result);\n      }\n      return result;\n    }\n  }\n\n  /**\n   * Decodes a value from the given ByteString, validating that no bytes are remaining once decoded.\n   */\n  public static <T> T decodeFromByteString(Coder<T> coder, ByteString encodedValue)\n      throws IOException {\n    return decodeFromByteString(coder, encodedValue, Coder.Context.OUTER);\n  }\n\n  /**\n   * Decodes a value from the given ByteString using a given context, validating that no bytes are\n   * remaining once decoded.\n   */","sourceCodeStart":86,"sourceCodeEnd":122,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/core/src/main/java/org/apache/beam/sdk/util/CoderUtils.java#L86-L122","documentation":"decodeFromByteArray checks that the Coder consumed the entire input; if bytes remain after decoding (stream.available() != 0) it throws CoderException naming the leftover count and the decoded value. It means the encoded byte array does not exactly match what the coder would produce — a framing or coder-mismatch problem.","triggerScenarios":"Calling CoderUtils.decodeFromByteArray(coder, bytes) where bytes were produced by a different coder, with trailing padding/terminators (e.g. extra newline from a text-based format), or from concatenating multiple encoded values into one call.","commonSituations":"Encoding with one coder version and decoding after a coder change; appending separators or length prefixes manually; decoding an element from a batch without slicing its exact byte range.","solutions":["Decode with the exact same Coder instance/registration used to encode","Ensure the byte array contains exactly one encoded element — strip trailing newlines/terminators or slice the exact length","If values were concatenated, decode iteratively with the coder consuming one element at a time instead of one decode of all bytes","Verify coder registries/coder clouds match between writer and reader stages"],"exampleFix":"// before\nbyte[] all = concat(e1, e2);\nMyValue v = CoderUtils.decodeFromByteArray(coder, all); // extra bytes\n// after\nMyValue v1 = CoderUtils.decodeFromByteArray(coder, e1);\nMyValue v2 = CoderUtils.decodeFromByteArray(coder, e2);","handlingStrategy":"validation","validationCode":"// before decoding\nif (bytes.length == 0 || bytes.length < minEncodedSize) {\n  throw new IllegalArgumentException(\"buffer too small/empty for one encoded element\");\n}\n// after decoding in dev tests:\n// assert re-encode(decoded).length == bytes.length","typeGuard":null,"tryCatchPattern":"try {\n  T v = CoderUtils.decodeFromByteArray(coder, bytes);\n} catch (CoderException e) {\n  // coder mismatch or trailing bytes; re-slice or use correct coder\n}","preventionTips":["Always decode with the same coder that encoded the bytes","Never concatenate encoded values into a single decode call","Strip trailing delimiters/newlines before decoding","Keep coder registries in sync across pipeline stages and versions"],"tags":["java","coders","serialization","decoding"],"backgroundTag":"unexpected-extra-bytes-after-decoding","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}