apache/beam · error

failed to encode DoFn

Error message

failed to encode DoFn %v, missing fn

What it means

encodeFn reached its default case: the graph.Fn did not contain a DynFn, UserFn, or a structural fn value it knows how to encode, so there is no encoding path for this DoFn. It indicates an Fn populated in a way the graphx encoder does not support.

Solutions

1. Build transforms with the public beam.ParDo/beam.Impulse APIs rather than hand-constructing graph.Fn.
2. Ensure the DoFn implements a supported form (named struct, func, or DynFn).
3. Check Beam version skew between the pipeline construction code and graphx serialization library.
4. Add debugging to print the graph.Fn before encoding to confirm which field is unset.

Example fix

// before
fn := &graph.Fn{} // nothing set

// after
fn, err := graph.NewFn(&myDoFn{}) // backed by a registered DoFn

if fn == nil || (fn.Dyn == nil && fn.Fn == nil && fn.T == nil) {
    return fmt.Errorf("graph.Fn has no encodable fn payload")
}

if _, err := graph.NewFn(myDoFn{}); err != nil {
    return fmt.Errorf("DoFn not constructible: %w", err)
}

When it happens

Trigger: EncodeMultiEdge on a *graph.Fn whose Fn/T/receiver fields are all unset or hold an unsupported fn variant (default switch branch).

Common situations: Custom transforms constructing graph.Fn directly without setting a fn; internally-constructed Fn nodes from newer/older Beam versions with mismatched shapes.

Understand the failure class

Background: "must not be empty", "cannot be empty" — required-field validation errors across open-source libraries — this error's family across 41 libraries.

Related errors

• bad base type
• bad channel direction
• bad element type
• bad field type
• bad parameter type

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/d46e4eb05a276c7a. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/core/runtime/graphx/serialize.go:265

		if _, ok := runtime.LookupType(k); !ok {
			err := errors.Errorf("receiver type %v must be registered", t)
			return nil, errors.WithContextf(err, "encoding structural DoFn %v", u)
		}
		typ, err := encodeType(t)
		if err != nil {
			wrapped := errors.Wrapf(err, "failed to encode receiver type %T", u.Recv)
			return nil, errors.WithContextf(wrapped, "encoding structural DoFn %v", u)
		}

		data, err := jsonx.Marshal(u.Recv)
		if err != nil {
			wrapped := errors.Wrapf(err, "failed to marshal receiver %v", u.Recv)
			return nil, errors.WithContextf(wrapped, "encoding structural DoFn %v", u)
		}
		return &v1pb.Fn{Type: typ, Opt: string(data)}, nil

	default:
		return nil, errors.Errorf("failed to encode DoFn %v, missing fn", u)
	}
}

func decodeFn(u *v1pb.Fn) (*graph.Fn, error) {
	if u.Dynfn != nil {
		gen, err := runtime.ResolveFunction(u.Dynfn.Gen, genFnType)
		if err != nil {
			wrapped := errors.Wrapf(err, "bad symbol %v", u.Dynfn.Gen)
			return nil, errors.WithContextf(wrapped, "decoding dynamic DoFn %v", u)
		}

		t, err := decodeType(u.Dynfn.Type)
		if err != nil {
			wrapped := errors.Wrap(err, "bad type")
			return nil, errors.WithContextf(wrapped, "failed to decode dynamic DoFn %v", u)
		}
		return graph.NewFn(&graph.DynFn{
			Name: u.Dynfn.Name,

View on GitHub (pinned to 12126d8942)