apache/beam · error

bad CoGBK: %v

Error message

bad CoGBK: %v

What it means

EncodeCoderRef encodes CoGBK coders as a pair of (tag, stream-of-list) refs, which requires at least two component coders (the key plus at least one value collection). A CoGBK with fewer than two components cannot be represented in the Dataflow form, so encoding fails with the offending coder in the message.

Source

Thrown at sdks/go/pkg/beam/core/runtime/graphx/dataflow.go:145

		innerref, err := EncodeCoderRef(c.Components[0])
		if err != nil {
			return nil, err
		}
		return &CoderRef{Type: nullableType, Components: []*CoderRef{innerref}}, nil

	case coder.Iterable:
		if len(c.Components) != 1 {
			return nil, errors.Errorf("bad I: %v", c)
		}
		innerref, err := EncodeCoderRef(c.Components[0])
		if err != nil {
			return nil, err
		}
		return &CoderRef{Type: streamType, Components: []*CoderRef{innerref}}, nil

	case coder.CoGBK:
		if len(c.Components) < 2 {
			return nil, errors.Errorf("bad CoGBK: %v", c)
		}

		refs, err := EncodeCoderRefs(c.Components)
		if err != nil {
			return nil, err
		}

		value := refs[1]
		if len(c.Components) > 2 {
			// TODO(https://github.com/apache/beam/issues/18032): don't inject union coder for CoGBK.

			union := &CoderRef{Type: cogbklistType, Components: refs[1:]}
			value = &CoderRef{Type: lengthPrefixType, Components: []*CoderRef{union}}
		}

		stream := &CoderRef{Type: streamType, Components: []*CoderRef{value}, IsStreamLike: true}
		return &CoderRef{Type: pairType, Components: []*CoderRef{refs[0], stream}, IsPairLike: true}, nil

View on GitHub (pinned to 12126d8942)

Solutions

1. Ensure the CoGBK coder includes the key component plus one component per input PCollection (>= 2 total).
2. Use beam.CoGBK / the SDK's coder construction for multi-input transforms instead of raw coder.Coder literals.
3. Check any custom coder provider that substitutes coders for CoGBK outputs and restore the full component list.
4. Re-derive the coder from the transform's inputs if it was cached from a stale pipeline version.

Example fix

// before
c := &coder.Coder{Kind: coder.CoGBK, T: t, Components: []*coder.Coder{key}}
ref, err := graphx.EncodeCoderRef(c)

// after
c := &coder.Coder{Kind: coder.CoGBK, T: t, Components: []*coder.Coder{key, value1}}
ref, err := graphx.EncodeCoderRef(c)

func validCoGBK(c *coder.Coder) bool {
	return c != nil && c.Kind == coder.CoGBK && len(c.Components) >= 2
}

func isCoGBKWithInputs(c *coder.Coder, n int) bool { return c != nil && c.Kind == coder.CoGBK && len(c.Components) == n+1 }

ref, err := graphx.EncodeCoderRef(c)
if err != nil {
	if strings.Contains(err.Error(), "bad CoGBK") {
		return fmt.Errorf("CoGBK coder needs key + >=1 input components: %w", err)
	}
	return err
}

When it happens

Trigger: Encoding a coder.Coder of Kind coder.CoGBK with len(c.Components) < 2 — e.g. a CoGBK coder built with only a key component, or an empty component list — during pipeline graph serialization.

Common situations: Custom transforms constructing CoGBK output coders without all input collections; programmatic pipeline builders that misassemble CoGBK coders; refactors that drop components from multi-input coders.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• input type must be KV: %v
• key coder for %v is %v, want %v
• mismatched CoGBK windowing strategies: %v, want %v
• failed to decode union value '%v' for key %v
• failed to marshal CoGBK coder %v

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/aa7151a2683b78c7. Report an issue: GitHub.