apache/beam · error

top.accum: element encoder unspecified with non-zero…

Error message

top.accum: element encoder unspecified with non-zero elements: %v data available

What it means

In Beam's top transform, the accum (accumulator for CombineFn) holds decoded elements plus buffered raw data and an optional element coder. This error is raised by the accumulator encoding function when the accumulator has decoded elements in its list but no encoder is set, making it impossible to serialize the accum (e.g. forGBK state or shuffle). It's an internal invariant violation: an enc-less accum should never hold non-empty list elements.

Solutions

1. Ensure the element type has a registered/inferable Beam coder (implement Coder interfaces or register external coder)
2. Check that the PCollection fed to Top has a non-nil element coder
3. Update Beam SDK version; older versions had top coder-inference bugs
4. If reproducible, file/report with the pipeline graph since this is an internal invariant violation

pc := beam.ParDo(s, fn, input)
if pc.Coder() == nil {
	return fmt.Errorf("element coder missing for Top input")
}

When it happens

Trigger: Combining with the top transform where the element coder is unspecified/nil (e.g. coder couldn't be inferred) while the accumulator already carries decoded elements during merge/extract of an accum that has both list and data populated.

Common situations: Pipeline uses transforms/top with a coder Beam could not infer (custom types without registered coders), or accumulation data flows across worker boundaries requiring serialization of an improperly initialized accum.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• ApproximateUnique.PerKey requires its input to use KvCoder
• array len mismatch. decoding
• AvroCoder for GenericRecord requires a schema
• buildDescriptor: couldn't retrieve coder
• cannot encode a null BitSet

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/282b3f7f503542a9. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/transforms/top/top.go:158

}

var (
	accumType = reflect.TypeOf((*accum)(nil)).Elem()
)

func init() {
	beam.RegisterType(accumType)
	beam.RegisterCoder(accumType, accumEnc(), accumDec())
}

func accumEnc() func(accum) ([]byte, error) {
	byteEnc, err := coder.EncoderForSlice(reflect.TypeOf((*[][]byte)(nil)).Elem())
	if err != nil {
		panic(err)
	}
	return func(a accum) ([]byte, error) {
		if len(a.list) > 0 && a.enc == nil {
			return nil, errors.Errorf("top.accum: element encoder unspecified with non-zero elements: %v data available", len(a.data))
		}
		var values [][]byte
		if len(a.list) == 0 && len(a.data) > 0 {
			values = a.data
		}
		for _, value := range a.list {
			var buf bytes.Buffer
			if err := a.enc.Encode(value, &buf); err != nil {
				return nil, errors.WithContextf(err, "top.accum: marshalling %v", value)
			}
			values = append(values, buf.Bytes())
		}

		var buf bytes.Buffer
		if err := coder.WriteSimpleRowHeader(1, &buf); err != nil {
			return nil, err
		}
		if err := byteEnc(values, &buf); err != nil {

View on GitHub (pinned to 12126d8942)