apache/beam · error

invalid DoFn

Error message

invalid DoFn

What it means

NewDoFn wraps any error from NewFn (or AsDoFn) and adds the message "invalid DoFn" plus context naming the DoFn being constructed. It is the top-level error users see when a DoFn value fails reflection or signature validation.

Source

Thrown at sdks/go/pkg/beam/core/graph/fn.go:477

// CoGBKMainInput is an optional config to NewDoFn which specifies the number
// of components of a CoGBK input to the DoFn being created, allowing for more complete
// validation.
//
// Example usage:
//
//	var col beam.PCollection
//	graph.NewDoFn(fn, graph.CoGBKMainInput(len(col.Type().Components())))
func CoGBKMainInput(components int) func(*config) {
	return func(cfg *config) {
		cfg.numMainIn = mainInputs(components)
	}
}

// NewDoFn constructs a DoFn from the given value, if possible.
func NewDoFn(fn any, options ...func(*config)) (*DoFn, error) {
	ret, err := NewFn(fn)
	if err != nil {
		return nil, errors.WithContext(errors.Wrapf(err, "invalid DoFn"), "constructing DoFn")
	}
	cfg := defaultConfig()
	for _, opt := range options {
		opt(cfg)
	}
	return AsDoFn(ret, cfg.numMainIn)
}

// AsDoFn converts a Fn to a DoFn, if possible. numMainIn specifies how many
// main inputs are expected in the DoFn's method signatures. Valid inputs are
// the package constants of type mainInputs. If that number is MainUnknown then
// validation is done by best effort and may miss some edge cases.
func AsDoFn(fn *Fn, numMainIn mainInputs) (*DoFn, error) {
	addContext := func(err error, fn *Fn) error {
		return errors.WithContextf(err, "graph.AsDoFn: for Fn named %v", fn.Name())
	}

	if fn.methods == nil {

View on GitHub (pinned to 12126d8942)

Solutions

1. Read the wrapped cause for the real failure (method invalid, wrong kind, missing ProcessElement, bad params)
2. Fix the DoFn value/signature accordingly
3. Validate the fn in a unit test with graph.NewDoFn before running the pipeline

Example fix

// before
beam.ParDo(s, myFnValue, in) // structural fn passed by value
// after
beam.ParDo(s, &myFnValue, in)

if _, err := graph.NewDoFn(myFn); err != nil { return fmt.Errorf("DoFn %T invalid: %w", myFn, err) }

ret, err := graph.NewDoFn(fn)
if err != nil {
    var ec errctx.Error
    if errors.As(err, &ec) { log.Printf("context: %v", ec.Context) }
    return fmt.Errorf("constructing DoFn: %w", err)
}

When it happens

Trigger: Calling NewDoFn (or indirectly beam.ParDo/beam.TryParDo) with an invalid value or a struct whose method signatures fail validation — the underlying cause is chained.

Common situations: Wrong receiver type, unsupported method signatures, missing ProcessElement on a structural DoFn; typically during pipeline graph construction at submit time.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• OnTimer function is defined for the DoFn but no TimerProvide
• OnTimer and ProcessElement functions for DoFn should have ex
• OnTimer and ProcessElement functions for DoFn should have ex
• error writing state: %v
• value %v must be function or (ptr to) struct

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/233d5f347721cf54. Report an issue: GitHub.