apache/beam · error

invalid scope

Error message

invalid scope

What it means

TryFlatten (and its wrapper Flatten) rejects the merge of multiple PCollections when the supplied Scope is invalid (its pipeline/scope is zero-valued or uninitialized). A Scope identifies which pipeline and enclosing transform the flatten edge is inserted into; without a valid one the transform cannot be added to the pipeline graph, so the call fails fast with this error.

Source

Thrown at sdks/go/pkg/beam/flatten.go:39

)

// Flatten is a PTransform that takes either multiple PCollections of type 'A'
// and returns a single PCollection of type 'A' containing all the elements in
// all the input PCollections. The name "Flatten" suggests taking a list of lists
// and flattening them into a single list.
//
// By default, the Coder of the output PCollection is the same as the Coder
// of the first PCollection.
func Flatten(s Scope, cols ...PCollection) PCollection {
	return Must(TryFlatten(s, cols...))
}

// TryFlatten merges incoming PCollections of type 'A' to a single PCollection
// of type 'A'. Returns an error indicating the set of PCollections that could
// not be flattened.
func TryFlatten(s Scope, cols ...PCollection) (PCollection, error) {
	if !s.IsValid() {
		return PCollection{}, errors.New("invalid scope")
	}
	for i, in := range cols {
		if !in.IsValid() {
			return PCollection{}, errors.Errorf("invalid pcollection to flatten: index %v", i)
		}
	}
	if len(cols) == 0 {
		return PCollection{}, errors.New("no input pcollections")
	}
	if len(cols) == 1 {
		return cols[0], nil // no-op
	}

	var in []*graph.Node
	for _, s := range cols {
		in = append(in, s.n)
	}
	edge, err := graph.NewFlatten(s.real, s.scope, in)

View on GitHub (pinned to 12126d8942)

Solutions

1. Create the pipeline with p := beam.NewPipeline() and pass a valid scope such as p or s := beam.Scope("name") derived from it as the first argument to Flatten.
2. Check s.IsValid() (or log s) right before the call to confirm the scope is valid; if it is not, trace where the Scope came from and fix its construction.
3. Ensure any function that receives a Scope propagates error returns so a zero-value Scope is not silently used.

Example fix

// before
var s beam.Scope
out := beam.Flatten(s, a, b) // panic-free but errors: invalid scope

// after
p := beam.NewPipeline()
out := beam.Flatten(p, a, b)

if !s.IsValid() {
	return fmt.Errorf("cannot Flatten: scope is invalid; build it from beam.NewPipeline() or beam.Scope(...)")
}
out := beam.Flatten(s, a, b)

func validScope(s beam.Scope) bool { return s.IsValid() }

When it happens

Trigger: Calling beam.Flatten(s, cols...) or beam.TryFlatten(s, cols...) where s was produced by a zero-value Scope{}, an invalid Scope returned from a failed operation, or a Scope from a pipeline that was never constructed via beam.NewPipeline() / a valid named scope (s.scope).

Common situations: Storing a Scope in a struct that is initialized with Go's zero value; using a Scope variable whose assignment path returned an error earlier and was ignored; calling Flatten outside any transform context after constructing a pipeline but passing an invalid nested scope.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• empty pipeline
• create has no values
• invalid scope
• invalid scope
• invalid main pcollection

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/20186933cd66efd4. Report an issue: GitHub.