apache/beam · error

failed to make GBK Union coder.

Error message

failed to make GBK Union coder.

What it means

Produced by MakeGBKUnionCoder when its internal makeUnionCoder() call fails while building the KV<int,[]byte> union value coder. The SDK wraps the cause with 'failed to make GBK Union coder.', indicating the CoGBK output coder for reshuffle could not be constructed.

Solutions

1. Unwrap the error to read the underlying makeUnionCoder cause.
2. Rebuild/upgrade to an unmodified stable Apache Beam release.
3. Check that default component coders (varintz, bytes) are intact and not shadowed by custom code.
4. Reproduce minimally and report to Apache Beam if it persists.
5. Avoid custom reshuffle paths that force GBK union coder construction until fixed.

Example fix

// before
c, err := graphx.MakeGBKUnionCoder(gbkEdge) // err: failed to make GBK Union coder.
// after
if err != nil {
    return fmt.Errorf("gbk union coder: %w", err) // surface wrapped cause
}

if gbk == nil || len(gbk.Input) == 0 || gbk.Input[0].From == nil || gbk.Input[0].From.Coder == nil {
    return errors.New("CoGBK edge lacks input coder needed for GBK union coder")
}

func hasInputCoder(e *graph.MultiEdge) bool {
    return e != nil && len(e.Input) > 0 && e.Input[0].From != nil && e.Input[0].From.Coder != nil
}

kvCoder, err := makeUnionCoder()
if err != nil {
    return nil, fmt.Errorf("GBK union coder: %w", err)
}

When it happens

Trigger: Calling MakeGBKUnionCoder on a valid CoGBK MultiEdge while makeUnionCoder() errors during construction of its component coders (varintz int + raw bytes).

Common situations: Internal construction failure in reshuffle coder assembly; broken local builds/patches of graphx; exercised primarily by tests (TestMakeGBKUnionCoder_bad) and expandCoGBK in production code.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• failed to make KV Union coder.
• expected CoGBK, got
• failed to expand CoGBK transform for edge
• unexpected expand coder
• values of cannot bind to

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/2e8b0f607e144846. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/core/runtime/graphx/cogbk.go:98

	kvCoder, err := makeUnionCoder()
	if err != nil {
		return nil, errors.Wrapf(err, "failed to make KV Union coder.")
	}
	return coder.NewKV([]*coder.Coder{key, kvCoder}), nil
}

// MakeGBKUnionCoder returns CoGBK<K,KV<int,[]byte>> for a given CoGBK.
func MakeGBKUnionCoder(gbk *graph.MultiEdge) (*coder.Coder, error) {
	if gbk.Op != graph.CoGBK {
		err := errors.Errorf("expected CoGBK, got %v", gbk)
		return nil, errors.WithContext(err, "failed to make GBK Union coder")
	}

	from := gbk.Input[0].From
	key := from.Coder.Components[0]
	kvCoder, err := makeUnionCoder()
	if err != nil {
		return nil, errors.Wrapf(err, "failed to make GBK Union coder.")
	}
	return coder.NewCoGBK([]*coder.Coder{key, kvCoder}), nil
}

// makeUnionCoder returns a coder for the raw union value, KV<int,[]byte>. It uses
// varintz instead of the built-in varint to avoid the implicit length-prefixing
// of varint otherwise introduced by Dataflow.
func makeUnionCoder() (*coder.Coder, error) {
	c, err := coderx.NewVarIntZ(reflectx.Int)
	if err != nil {
		return nil, err
	}
	return coder.NewKV([]*coder.Coder{
		{Kind: coder.Custom, T: typex.New(reflectx.Int), Custom: c},
		coder.NewBytes(),
	}), nil
}

View on GitHub (pinned to 12126d8942)