apache/beam · error

tried converting invalid Node

Error message

tried converting invalid Node

What it means

nodeToPCollection wraps a graph.Node as a PCollection during cross-language pipeline translation. The library panics when the node pointer is nil, because a nil Node cannot carry a type or coder — proceeding would produce a corrupt pipeline graph. This is an internal invariant check that fires when pipeline translation produces an unexpected nil node.

Solutions

1. Inspect the pipeline before Run: verify every external transform's expanded outputs are referenced by valid, non-nil PCollections
2. Check Beam Go SDK version compatibility with the expansion service and other language SDKs; upgrade to a matching release
3. Report a bug to the Beam project with the pipeline and expansion service logs if nil nodes arise from standard xlang usage

Example fix

// before: passing a possibly-nil node
pc := beam.PCollection{} // built from mapNodeToPCollection(maybeNilNode)

// after: guard before conversion
if n == nil {
    return fmt.Errorf("expansion returned nil node for external transform output")
}
pc := nodeToPCollection(n)

if n == nil {
    return fmt.Errorf("xlang expansion produced nil node; check external transform outputs")
}
pc := nodeToPCollection(n)

func isValidNode(n *graph.Node) bool { return n != nil }

// Go panics are not recoverable at the panic site; wrap pipeline construction
func buildPC(n *graph.Node) (pc PCollection, err error) {
    defer func() {
        if r := recover(); r != nil {
            err = fmt.Errorf("nodeToPCollection failed: %v", r)
        }
    }()
    return nodeToPCollection(n), nil
}

When it happens

Trigger: Calling nodeToPCollection (via mapNodeToPCollection) with a nil *graph.Node during cross-language expansion/translation, typically when an expansion result or predecessor node lookup returned nil.

Common situations: Cross-language (XLang) transforms referencing a PCollection that was never created or was removed; version mismatch between Beam SDKs where an expansion response lacks expected nodes; malformed pipeline construction with external transforms.

Related errors

• panic(err)
• AfterProcessingTime trigger set without a delay or…
• At least one subtrigger required for composite triggers.
• attempted to add namespace to missing coder id
• attempted to add namespace to missing windowing strategy id

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/b1f5e49d156ed353. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/xlang.go:230

	graphx.VerifyNamedOutputs(&ext)
	// Using the expanded outputs, the graph's counterpart outputs are updated with bounded values
	graphx.ResolveOutputIsBounded(edge, isBoundedUpdater)

	return mapNodeToPCollection(graphx.ExternalOutputs(edge)), nil
}

// Wrapper functions to handle beam <-> graph boundaries

func pCollectionToNode(p PCollection) *graph.Node {
	if !p.IsValid() {
		panic("tried converting invalid PCollection")
	}
	return p.n
}

func nodeToPCollection(n *graph.Node) PCollection {
	if n == nil {
		panic("tried converting invalid Node")
	}
	c := PCollection{n}
	c.SetCoder(NewCoder(c.Type()))
	return c
}

func mapPCollectionToNode(pMap map[string]PCollection) map[string]*graph.Node {
	if pMap == nil {
		return nil
	}

	nMap := make(map[string]*graph.Node)
	for k, p := range pMap {
		nMap[k] = pCollectionToNode(p)
	}
	return nMap
}

View on GitHub (pinned to 12126d8942)