apache/beam · error

need at least 1 pcollection

Error message

need at least 1 pcollection

What it means

TryCoGroupByKey needs at least one PCollection to co-group; with zero inputs the CoGBK transform has nothing to work on, so the library returns this error with context about the scope. Valid collections are then additionally checked individually.

Source

Thrown at sdks/go/pkg/beam/gbk.go:77

}

// CoGroupByKey inserts a CoGBK transform into the pipeline.
func CoGroupByKey(s Scope, cols ...PCollection) PCollection {
	return Must(TryCoGroupByKey(s, cols...))
}

func addCoGBKCtx(err error, s Scope) error {
	return errors.WithContextf(err, "inserting CoGroupByKey in scope %s", s)
}

// TryCoGroupByKey inserts a CoGBK transform into the pipeline. Returns
// an error on failure.
func TryCoGroupByKey(s Scope, cols ...PCollection) (PCollection, error) {
	if !s.IsValid() {
		return PCollection{}, addCoGBKCtx(errors.New("invalid scope"), s)
	}
	if len(cols) < 1 {
		return PCollection{}, addCoGBKCtx(errors.New("need at least 1 pcollection"), s)
	}
	for i, in := range cols {
		if !in.IsValid() {
			return PCollection{}, addCoGBKCtx(errors.Errorf("invalid pcollection to CoGBK: index %v", i), s)
		}
	}

	var in []*graph.Node
	for _, s := range cols {
		in = append(in, s.n)
	}

	edge, err := graph.NewCoGBK(s.real, s.scope, in)
	if err != nil {
		return PCollection{}, err
	}
	ret := PCollection{edge.Output[0].To}
	ret.SetCoder(NewCoder(ret.Type()))

View on GitHub (pinned to 12126d8942)

Solutions

1. Guard: if len(cols) == 0 skip the CoGroupByKey call or return a descriptive error from your own code.
2. Ensure at least one input PCollection is produced upstream before grouping.
3. Log the number of candidate collections just before the call to locate where they were dropped.

Example fix

// before
grouped := beam.CoGroupByKey(s, cols...) // errors when len(cols)==0

// after
if len(cols) == 0 {
	return fmt.Errorf("no inputs to CoGroupByKey")
}
grouped := beam.CoGroupByKey(s, cols...)

if len(cols) < 1 {
	return fmt.Errorf("CoGroupByKey requires at least one pcollection")
}
out := beam.CoGroupByKey(s, cols...)

When it happens

Trigger: beam.CoGroupByKey(s) or beam.TryCoGroupByKey(s) called with an empty variadic list, typically when collections are accumulated dynamically (map/slice iteration) and the container is empty.

Common situations: Aggregating inputs from a config-driven list of sources that resolved to nothing; conditional pipeline construction where all candidate branches were omitted; passing an empty slice with '...' expansion.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

• no input pcollections
• invalid scope
• must have at least one input to a KeyedPCollections
• Could not resolve empty field path
• quoted identifier cannot be empty

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/248b7e28e580ccc8. Report an issue: GitHub.