apache/beam · error · CoderException

wrong decoding sequence found.

Error message

wrong decoding sequence found.

What it means

The Histogram coder encodes an accumulator with a one-byte CoderType discriminator (NON_EMPTY_BUCKETS_CODER or ALL_BUCKETS_CODER). During decode, any discriminator value outside the known ordinals means the byte stream does not match this coder's encoding, so CoderException('wrong decoding sequence found.') is thrown. This protects against decoding data written by a different coder or corrupted stream.

Source

Thrown at sdks/java/extensions/combiners/src/main/java/org/apache/beam/sdk/extensions/combiners/Histogram.java:534

      INT_CODER.encode(numBuckets, outStream);

      INT_CODER.encode(numBuckets - numEmptyBucketsAtTheEnd, outStream);
      for (int i = 0; i < numBuckets - numEmptyBucketsAtTheEnd; i++) {
        LONG_CODER.encode(value.counts[i], outStream);
      }
    }

    @Override
    public HistogramAccumulator decode(InputStream inStream) throws IOException {
      int coder = inStream.read();
      int numBuckets = INT_CODER.decode(inStream);

      if (coder == CoderType.NON_EMPTY_BUCKETS_CODER.ordinal()) {
        return decodeNonEmptyBuckets(numBuckets, inStream);
      } else if (coder == CoderType.ALL_BUCKETS_CODER.ordinal()) {
        return decodeAllBuckets(numBuckets, inStream);
      } else {
        throw new CoderException("wrong decoding sequence found.");
      }
    }

    private HistogramAccumulator decodeNonEmptyBuckets(int numBuckets, InputStream inStream)
        throws IOException {
      HistogramAccumulator histogramAccumulator = new HistogramAccumulator(numBuckets);
      int nonEmptyBucketCounts = INT_CODER.decode(inStream);

      for (int i = 0; i < nonEmptyBucketCounts; i++) {
        histogramAccumulator.counts[INT_CODER.decode(inStream)] = LONG_CODER.decode(inStream);
      }

      return histogramAccumulator;
    }

    private HistogramAccumulator decodeAllBuckets(int numBuckets, InputStream inStream)
        throws IOException {
      HistogramAccumulator histogramAccumulator = new HistogramAccumulator(numBuckets);

View on GitHub (pinned to 12126d8942)

Solutions

1. Ensure the same Beam version encodes and decodes the histogram state; re-encode any persisted accumulators after upgrading.
2. Verify the InputStream is positioned at the start of a HistogramCoder-encoded payload (no offset/truncation).
3. Confirm the coder used to write the bytes is HistogramCoder, not another coder for the same type.
4. Wrap decode in try/catch(CoderException) and rebuild the histogram from source data as a fallback.

if (bytes == null || bytes.length == 0 || (bytes[0] != NON_EMPTY_BUCKETS_CODER && bytes[0] != ALL_BUCKETS_CODER)) {
  throw new IOException("payload is not a HistogramCoder-encoded accumulator");
}

try {
  acc = coder.decode(inStream, Coder.Context.OUTER);
} catch (CoderException e) {
  LOG.warn("corrupt/foreign histogram encoding, rebuilding", e);
  acc = new HistogramAccumulator(numBuckets);
}

When it happens

Trigger: Decoding bytes that were not produced by HistogramCoder; deserializing persisted accumulator state written by an older/newer Beam version whose encoding changed; a stream positioned incorrectly (offset/truncation) so the discriminator byte is garbage.

Common situations: Job upgrades: state/checkpoints or test fixtures encoded with a previous HistogramCoder format are decoded after a library update; manually crafted byte arrays in unit tests; mixing accumulators from different histogram definitions.

Related errors

• Invalid encoded string length: {}
• Expected 0 or 1, got %d
• error when decoding a textual integer
• EOF encountered decoding a ValueKind
• Unknown ValueKind number: {}

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/289313d226a61eb6. Report an issue: GitHub.