apache/beam · error

want all values to be iterables

Error message

want all values to be iterables: %v

What it means

convertToCoGBK rewrites a CoGBK-style data source coder into a CoGBK coder, requiring every value component after the key to be an Iterable coder. If any value component's kind is not coder.Iterable, the function panics, since the CoGBK conversion cannot represent non-iterable values.

Solutions

1. Inspect the coder in the panic message (%v) and confirm which component is not Iterable
2. Ensure upstream GBK outputs produce Iterable values (use beam.ParDo + beam.GroupByKey rather than hand-built coders)
3. Check custom translation/coder override code for incorrect NewCoGBK construction
4. Update the Go SDK — older versions had coder inference bugs around CoGBK sources

Example fix

// before
c := coder.NewCoGBK([]*coder.Coder{kCoder, vScalarCoder}) // value not iterable
// after
c := coder.NewW(coder.NewCoGBK([]*coder.Coder{kCoder, coder.NewIter(vCoder)}), window)

for _, comp := range coder.SkipW(oc).Components[1:] {
    if comp.Kind != coder.Iterable {
        return fmt.Errorf("CoGBK value %v is not Iterable", comp)
    }
}

When it happens

Trigger: A CoGBK output coder whose value side is a scalar or list/other composite instead of Iterable; graph translation from a runner port where the coder was inferred incorrectly; mayFixDataSourceCoder handing a coder that was already converted.

Common situations: Cross-language pipelines where the other SDK encodes GBK values differently; custom coder registration that mislabels a kind; version mismatches between runner and SDK coder representations.

Related errors

• failed to expand CoGBK transform for edge
• unexpected expand coder
• array len mismatch. decoding
• coder must not be nil
• coder type must be identical to node type

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/de5a070250fd7e97. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/core/runtime/exec/translate.go:154

		if len(in) < 2 {
			return // Somehow there's only a single value, so we're done. (Defense against generic KVs)
		}
		// It's an iterator, so we can assume it's a GBK, due to previous pre-conditions.
		if sig.Param[in[1]].Kind == funcx.FnIter {
			u.Coder = convertToCoGBK(u.Coder)
			return
		}
	}
}

func convertToCoGBK(oc *coder.Coder) *coder.Coder {
	ocnw := coder.SkipW(oc)
	// Validate that all values from the coder are iterables.
	comps := make([]*coder.Coder, 0, len(ocnw.Components))
	comps = append(comps, ocnw.Components[0])
	for _, c := range ocnw.Components[1:] {
		if c.Kind != coder.Iterable {
			panic(fmt.Sprintf("want all values to be iterables: %v", oc))
		}
		comps = append(comps, c.Components[0])
	}
	return coder.NewW(coder.NewCoGBK(comps), oc.Window)
}

type builder struct {
	desc   *fnpb.ProcessBundleDescriptor
	coders *graphx.CoderUnmarshaller

	prev map[string]int      // PCollectionID -> #incoming
	succ map[string][]linkID // PCollectionID -> []linkID

	windowing map[string]*window.WindowingStrategy
	nodes     map[string]*PCollection // PCollectionID -> Node (cache)
	links     map[linkID]Node         // linkID -> Node (cache)

	units       []Unit // result

View on GitHub (pinned to 12126d8942)