What causes "failed to expand CoGBK transform for edge: %v" in apache/beam?

Inspect the wrapped inner error (usually coder or windowing marshal failure)

failed to expand CoGBK transform for edge

Error message

failed to expand CoGBK transform for edge: %v

What it means

expandCoGBK implements CoGBK as a composite of GBK + Flatten + a KV-union coder (since CoGBK is not yet a primitive). Any failure while making the union coder, the per-input nodes, or the intermediate GBK transforms is wrapped as 'failed to expand CoGBK transform for edge: %v'. handleErr wraps errors from MakeKVUnionCoder and the sub-translations.

Solutions

Inspect the wrapped inner error (usually coder or windowing marshal failure)
Ensure all CoGBK inputs use compatible, serializable coders
Use beam.CoGroupByKey with standard KV types; avoid custom coders on the inputs
Upgrade the Go SDK; CoGBK translation has received fixes since the TODO referenced

Example fix

null

Defensive patterns

Strategy: validation

Validate before calling

// check CoGBK inputs use marshalable coders
for _, col := range cogbkInputs {
    if reflect.TypeOf(beam.EncodedCoder{}) == nil {
        _ = col // ensure standard KV types
    }
    // simplest check: build the union coder beforehand
    if _, err := graphx.MakeKVUnionCoder(nil); err != nil {
        return err
    }
}

Try / catch

err := beam.Run(ctx, pr)
if err != nil && strings.Contains(err.Error(), "failed to expand CoGBK transform") {
    log.Printf("CoGBK expansion failed — check input coders/windowing: %v", err)
    return err
}

Prevention

Use beam.CoGroupByKey on plain KV PCollections with standard coders
Avoid mixing bounded/unbounded inputs with exotic windowing in CoGBK
Keep SDK current (CoGBK expansion had known limitations, issue #18032)

When it happens

Trigger: A CoGBK (beam.CoGroupByKey or multi-input CoGBK) edge where MakeKVUnionCoder fails (e.g. input coders cannot form a union coder) or any intermediate node/GBK expansion fails.

Common situations: CoGroupByKey over PCollections with incompatible or custom coders; unbounded/unsupported windowing on CoGBK inputs; SDK versions where the CoGBK expansion is still limited (see issue #18032).

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/343ec37ceb9fc858. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/core/runtime/graphx/translate.go:807

			outputs[fmt.Sprintf("i%v", i)] = nodeID(out.To)
		}
		transform.Outputs = outputs
		environment, err := ExpandedTransform(edge.External.Expanded)
		if err != nil {
			return "", errors.Wrapf(err, "failed to expand cross language transform for edge: %v", namedEdge)
		}
		transform.EnvironmentId = environment.EnvironmentId
	}

	m.transforms[id] = transform
	return id, nil
}

func (m *marshaller) expandCoGBK(edge NamedEdge) (string, error) {
	// TODO(https://github.com/apache/beam/issues/18032): replace once CoGBK is a primitive. For now, we have to translate
	// CoGBK with multiple PCollections as described in cogbk.go.
	handleErr := func(err error) (string, error) {
		return "", errors.Wrapf(err, "failed to expand CoGBK transform for edge: %v", edge)
	}

	id := edgeID(edge.Edge)
	kvCoder, err := MakeKVUnionCoder(edge.Edge)
	if err != nil {
		return handleErr(err)
	}
	kvCoderID, err := m.coders.Add(kvCoder)
	if err != nil {
		return handleErr(err)
	}
	gbkCoder, err := MakeGBKUnionCoder(edge.Edge)
	if err != nil {
		return handleErr(err)
	}
	gbkCoderID, err := m.coders.Add(gbkCoder)
	if err != nil {
		return handleErr(err)

View on GitHub (pinned to 12126d8942)