apache/beam · error

Invalid PCollection

Error message

Invalid PCollection

What it means

PCollection.Type returns the full element type of the collection. Because PCollection is a lightweight wrapper, calling Type on a zero-value or unset PCollection is invalid, so the library panics with "Invalid PCollection" rather than dereferencing a nil node.

Solutions

1. Check p.IsValid() before calling Type, or ensure the PCollection comes from a Must*-style API (e.g. ParDo, Impulse) that cannot return an invalid value.
2. Handle the error return of TryParDo/TryCoGBK/TryReshuffle etc. before using the returned PCollections.
3. Audit code paths where a PCollection variable is assigned conditionally and may remain the zero value.

Example fix

// before
ret, err := beam.TryParDo(s, &fn{}, col)
_ = err
t := ret[0].Type() // panics: Invalid PCollection

// after
ret, err := beam.TryParDo(s, &fn{}, col)
if err != nil {
    log.Fatal(err)
}
t := ret[0].Type()

if !p.IsValid() {
    return errors.New("PCollection is invalid; a Try* call likely failed")
}
t := p.Type()

defer func() { if r := recover(); r != nil { err = fmt.Errorf("invalid PCollection access: %v", r) } }()

When it happens

Trigger: Calling .Type() on a zero-value PCollection (var p beam.PCollection), on a PCollection from a failed/unchecked TryParDo or TryCoGBK result, or one returned by an API whose error was ignored.

Common situations: Ignoring the error from Try* variants (which return empty PCollections on failure) and immediately calling .Type(); storing PCollections in structs initialized without values; out-of-range indexing into a returned slice producing an invalid wrapper.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• AfterProcessingTime trigger set without a delay or…
• At least one subtrigger required for composite triggers.
• attempted to add namespace to missing coder id
• attempted to add namespace to missing windowing strategy id
• batch: failed to marshal worker UUID

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/71b56c09000174b7. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/pcollection.go:58

type PCollection struct {
	// n is the graph node that PCollection wraps. If there is no node, the
	// PCollection is invalid.
	n *graph.Node
}

// IsValid returns true iff the PCollection is valid and part of a Pipeline.
// Any use of an invalid PCollection will result in a panic.
func (p PCollection) IsValid() bool {
	return p.n != nil
}

// TODO(herohde) 5/30/2017: add name for PCollections? Java supports it.

// Type returns the full type 'A' of the elements. 'A' must be a concrete
// type, such as int or KV<int,string>.
func (p PCollection) Type() FullType {
	if !p.IsValid() {
		panic("Invalid PCollection")
	}
	return p.n.Type()
}

// Coder returns the coder for the collection. The Coder is of type 'A'.
func (p PCollection) Coder() Coder {
	if !p.IsValid() {
		panic("Invalid PCollection")
	}
	return Coder{p.n.Coder}
}

// SetCoder set the coder for the collection. The Coder must be of type 'A'.
func (p PCollection) SetCoder(c Coder) error {
	if !p.IsValid() {
		panic("Invalid PCollection")
	}

View on GitHub (pinned to 12126d8942)