apache/beam · error

invalid pcollection to CoGBK: index

Error message

invalid pcollection to CoGBK: index %v

What it means

TryCoGroupByKey (sdks/go/pkg/beam/gbk.go) validates each input PCollection before constructing the CoGBK node; an invalid input at index i produces this error, wrapped with scope context via addCoGBKCtx. It is raised by TryGroupByKey and CoGroupByKey when any keyed collection passed in is a zero value or not tied to a valid scope.

Solutions

1. Check the collection at the reported index was produced by a valid keyed transform (e.g. beam.ParDo/KV output).
2. Validate with in.IsValid() before invoking CoGroupByKey.
3. Prefer beam.TryCoGroupByKey/TryGroupByKey to receive an error rather than a panic.
4. Review the scope context appended by addCoGBKCtx to locate the failing scope in the pipeline.

Example fix

// before
var left beam.PCollection
beam.CoGroupByKey(s, left, right)
// after
left := beam.ParDo(s, &extractKeyFn{}, raw)
if !left.IsValid() { return errors.New("left join side missing") }
beam.CoGroupByKey(s, left, right)

for i, c := range joinSides {
    if !c.IsValid() {
        return fmt.Errorf("CoGBK side %d invalid", i)
    }
}

res, err := beam.TryCoGroupByKey(s, cols...)
if err != nil {
    return fmt.Errorf("CoGBK failed (check scope ctx in error): %w", err)
}

When it happens

Trigger: Calling beam.CoGroupByKey or beam.TryGroupByKey with a zero-value or uninitialized PCollection among the cols inputs.

Common situations: Assembling multiple PCollections for a join where one side of the join was never produced (skipped transform branch), or a typo assigning to the wrong variable.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• invalid pcollection to external: index
• invalid pcollection to flatten: index
• bad CoGBK
• Compression factor should be greater than 0.
• Could not find a transform with id

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/08943c972ce3104f. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/gbk.go:81

	return Must(TryCoGroupByKey(s, cols...))
}

func addCoGBKCtx(err error, s Scope) error {
	return errors.WithContextf(err, "inserting CoGroupByKey in scope %s", s)
}

// TryCoGroupByKey inserts a CoGBK transform into the pipeline. Returns
// an error on failure.
func TryCoGroupByKey(s Scope, cols ...PCollection) (PCollection, error) {
	if !s.IsValid() {
		return PCollection{}, addCoGBKCtx(errors.New("invalid scope"), s)
	}
	if len(cols) < 1 {
		return PCollection{}, addCoGBKCtx(errors.New("need at least 1 pcollection"), s)
	}
	for i, in := range cols {
		if !in.IsValid() {
			return PCollection{}, addCoGBKCtx(errors.Errorf("invalid pcollection to CoGBK: index %v", i), s)
		}
	}

	var in []*graph.Node
	for _, s := range cols {
		in = append(in, s.n)
	}

	edge, err := graph.NewCoGBK(s.real, s.scope, in)
	if err != nil {
		return PCollection{}, err
	}
	ret := PCollection{edge.Output[0].To}
	ret.SetCoder(NewCoder(ret.Type()))
	return ret, nil
}

// Reshuffle copies a PCollection of the same kind and using the same element

View on GitHub (pinned to 12126d8942)