apache/beam · error

unexpected expand coder

Error message

unexpected expand coder: %v

What it means

The URNExpand transform (expanding a CoGBK result back into per-input streams) requires the output PCollection's coder to be CoGBK so per-input decoders can be built from its components. A non-CoGBK coder fails translation with this error.

Solutions

1. Align SDK harness and pipeline builder versions and resubmit
2. Verify the output coder ID in the pipeline proto resolves to a CoGBK coder; correct coder payloads if custom
3. Ensure custom coder implementations preserve the CoGBK component structure (one component per input, index 0 = window/header).

Example fix

null

c, _, err := b.makeCoderForPCollection(pid)
if err != nil {
	return err
}
if !coder.IsCoGBK(c) {
	return fmt.Errorf("expand output must be CoGBK-coded, got %v", c)
}

null

null

When it happens

Trigger: Coder for the Expand transform's output PCollection is not CoGBK — e.g. runner/SDK coder inference disagreement, custom coders collapsing the CoGBK structure, or a hand-built pipeline proto with a mismatched coder ID.

Common situations: CoGBK/Flatten pipelines crossing SDK version upgrades; custom coders registered on runner but altering component counts; foreign runners mislabeling the expand output coder.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

• failed to expand CoGBK transform for edge
• want all values to be iterables
• array len mismatch. decoding
• coder must not be nil
• coder type must be identical to node type

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/588d12a68f7e6b62. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/core/runtime/exec/translate.go:733

			// JIRA BEAM-12438 - an extra LP coder can get added here, but isn't added
			// on decode. Strip them until we get a better fix.
			if valCoder.Kind == coder.LP {
				// strip unexpected length prefix coder.
				valCoder = valCoder.Components[0]
			}
			u = &Inject{UID: b.idgen.New(), N: (int)(tp.GetInject().GetN()), ValueEncoder: MakeElementEncoder(valCoder), Out: out[0]}

		case graphx.URNExpand:
			var pid string
			for _, id := range transform.GetOutputs() {
				pid = id
			}
			c, _, err := b.makeCoderForPCollection(pid)
			if err != nil {
				return nil, err
			}
			if !coder.IsCoGBK(c) {
				return nil, errors.Errorf("unexpected expand coder: %v", c)
			}

			var decoders []ElementDecoder
			for _, dc := range c.Components[1:] {
				decoders = append(decoders, MakeElementDecoder(dc))
			}
			// Strip PCollections from Expand nodes, as CoGBK metrics are handled by
			// the DataSource that preceeds them.
			trueOut := out[0]
			if pcol, ok := trueOut.(*PCollection); ok {
				trueOut = pcol.Out
			}
			b.units = b.units[:len(b.units)-1]
			u = &Expand{UID: b.idgen.New(), ValueDecoders: decoders, Out: trueOut}

		case graphx.URNReshuffleInput:
			_, w, err := b.makeCoderForPCollection(from)
			if err != nil {

View on GitHub (pinned to 12126d8942)