apache/beam · error

failed to make KV Union coder.

Error message

failed to make KV Union coder.

What it means

Produced by MakeKVUnionCoder when its internal makeUnionCoder() call fails while constructing KV<int,[]byte>. The SDK wraps that failure with 'failed to make KV Union coder.' so callers know the CoGBK reshuffle coder could not be assembled.

Solutions

1. Inspect the wrapped cause from makeUnionCoder for the root failure.
2. Retry after upgrading to a stable Beam release, since this is an internal construction path.
3. Verify no local modifications to graphx coder construction (varintz coder) are broken.
4. If reproducible, file an Apache Beam issue with the wrapped error and pipeline shape.
5. As a workaround, avoid forcing reshuffle (or use a Beam version where expandCoGBK union coding is healthy).

Example fix

// before
c, err := graphx.MakeKVUnionCoder(gbkEdge) // err: failed to make KV Union coder.
// after
if err != nil {
    log.Printf("union coder build failed: %v", err) // inspect wrapped cause
    return err
}

if gbk == nil || len(gbk.Input) == 0 || gbk.Input[0].From == nil || gbk.Input[0].From.Coder == nil {
    return errors.New("CoGBK edge lacks input coder needed for union coder")
}

func hasInputCoder(e *graph.MultiEdge) bool {
    return e != nil && len(e.Input) > 0 && e.Input[0].From != nil && e.Input[0].From.Coder != nil
}

kvCoder, err := makeUnionCoder()
if err != nil {
    return nil, fmt.Errorf("KV union coder: %w", err)
}

When it happens

Trigger: Calling MakeKVUnionCoder on a valid CoGBK MultiEdge, but makeUnionCoder() returns an error (e.g. failure constructing the varintz/bytes component coders).

Common situations: Rare internal failure while setting up reshuffle coders for CoGBK; corrupted or edge-case coder registries in custom builds; surfaced mainly in tests like TestMakeKVUnionCoder_bad and pipeline expansion code.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• failed to make GBK Union coder.
• expected CoGBK, got
• failed to expand CoGBK transform for edge
• unexpected expand coder
• values of cannot bind to

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/55ae84a4192e9f75. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/core/runtime/graphx/cogbk.go:82

// expressed as Go user code.

const (
	URNInject = "beam:go:transform:inject:v1"
	URNExpand = "beam:go:transform:expand:v1"
)

// MakeKVUnionCoder returns KV<K,KV<int,[]byte>> for a given CoGBK.
func MakeKVUnionCoder(gbk *graph.MultiEdge) (*coder.Coder, error) {
	if gbk.Op != graph.CoGBK {
		err := errors.Errorf("expected CoGBK, got %v", gbk)
		return nil, errors.WithContext(err, "failed to make KV Union coder")
	}

	from := gbk.Input[0].From
	key := from.Coder.Components[0]
	kvCoder, err := makeUnionCoder()
	if err != nil {
		return nil, errors.Wrapf(err, "failed to make KV Union coder.")
	}
	return coder.NewKV([]*coder.Coder{key, kvCoder}), nil
}

// MakeGBKUnionCoder returns CoGBK<K,KV<int,[]byte>> for a given CoGBK.
func MakeGBKUnionCoder(gbk *graph.MultiEdge) (*coder.Coder, error) {
	if gbk.Op != graph.CoGBK {
		err := errors.Errorf("expected CoGBK, got %v", gbk)
		return nil, errors.WithContext(err, "failed to make GBK Union coder")
	}

	from := gbk.Input[0].From
	key := from.Coder.Components[0]
	kvCoder, err := makeUnionCoder()
	if err != nil {
		return nil, errors.Wrapf(err, "failed to make GBK Union coder.")
	}
	return coder.NewCoGBK([]*coder.Coder{key, kvCoder}), nil

View on GitHub (pinned to 12126d8942)