apache/beam · error

bad KV

Error message

bad KV: %v

What it means

Produced by EncodeCoderRef in graphx/dataflow.go when converting a coder.KV to its Dataflow-protocol CoderRef: the legacy Dataflow representation requires exactly two components (key and value), so a KV with any other component count is rejected with 'bad KV'.

Solutions

1. Log c.Components length; rebuild the KV with exactly two components via coder.NewKV([]*coder.Coder{key, value}).
2. Trace where the malformed KV was created (custom coder construction or cross-language expansion).
3. Ensure pipeline coders are generated by Beam constructors rather than assembled manually.
4. If the coder comes from another SDK, check version compatibility and the pipeline's serialized graph.
5. For non-pair structured types, use an iterable/custom coder instead of KV.

Example fix

// before
bad := coder.NewKV([]*coder.Coder{keyOnly})
ref, err := graphx.EncodeCoderRef(bad)
// after
good := coder.NewKV([]*coder.Coder{key, value})
ref, err := graphx.EncodeCoderRef(good)

if c.Kind == coder.KV && len(c.Components) != 2 {
    return fmt.Errorf("KV coder must have exactly 2 components, got %d", len(c.Components))
}

func isWellFormedKV(c *coder.Coder) bool {
    return c != nil && c.Kind == coder.KV && len(c.Components) == 2
}

ref, err := graphx.EncodeCoderRef(c)
if err != nil && strings.HasPrefix(err.Error(), "bad KV") {
    return fmt.Errorf("KV arity invalid (%d components): %w", len(c.Components), err)
}

When it happens

Trigger: EncodeCoderRef/EncodeCoder on a *coder.Coder with Kind == coder.KV and len(c.Components) != 2 — e.g. a KV constructed with 0, 1, or 3+ components.

Common situations: Hand-built coder.KV with a wrong component list; coders arriving from other SDKs or legacy job submissions whose KV arity differs; corrupted pipeline graphs deserialized incorrectly; submitting to legacy Dataflow where EncodeCoderRef is used.

Related errors

• AfterProcessingTime trigger set without a delay or…
• array len mismatch. decoding
• At least one subtrigger required for composite triggers.
• attempted to add namespace to missing coder id
• attempted to add namespace to missing windowing strategy id

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/0dd72a8d5687a5bb. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/core/runtime/graphx/dataflow.go:110

func EncodeCoderRef(c *coder.Coder) (*CoderRef, error) {
	switch c.Kind {
	case coder.Custom:
		ref, err := encodeCustomCoder(c.Custom)
		if err != nil {
			return nil, err
		}
		data, err := protox.EncodeBase64(ref)
		if err != nil {
			return nil, err
		}
		return &CoderRef{
			Type:       lengthPrefixType,
			Components: []*CoderRef{{Type: data, PipelineProtoCoderID: c.Custom.ID}},
		}, nil

	case coder.KV:
		if len(c.Components) != 2 {
			return nil, errors.Errorf("bad KV: %v", c)
		}

		key, err := EncodeCoderRef(c.Components[0])
		if err != nil {
			return nil, err
		}
		value, err := EncodeCoderRef(c.Components[1])
		if err != nil {
			return nil, err
		}
		return &CoderRef{Type: pairType, Components: []*CoderRef{key, value}, IsPairLike: true}, nil

	case coder.Nullable:
		if len(c.Components) != 1 {
			return nil, errors.Errorf("bad N: %v", c)
		}
		innerref, err := EncodeCoderRef(c.Components[0])
		if err != nil {

View on GitHub (pinned to 12126d8942)