apache/beam · error

failed to decode userfn

Error message

failed to decode userfn

What it means

decodeFn fails while decoding the Fn's userfn sub-message via decodeUserFn (a protobuf decode of the encoded function descriptor), wrapped as "failed to decode userfn". The serialized function payload in the pipeline proto is corrupt or from an incompatible schema version.

Solutions

1. Regenerate the pipeline with the same Beam Go SDK version used by the worker/decoder.
2. Check the inner cause of the wrap for the specific unresolvable symbol or type and register it.
3. Avoid hand-editing or caching serialized pipeline protos across releases.
4. If the pipeline comes from a file, re-export it from the driver rather than reusing stale artifacts.

Example fix

// before
golang "github.com/apache/beam/sdks/v2@v2.30.0/go/pkg/beam"
// worker compiled against v2.50.0

// after
// pin both driver and worker to the same Beam version, e.g. v2.50.0

fn, err := decodeUserFn(u.Fn)
if err != nil {
    return fmt.Errorf("userfn decode failed (check Beam SDK version parity between submitter and worker): %w", err)
}

When it happens

Trigger: DecodeMultiEdge decoding a Fn whose u.Fn payload fails proto unmarshaling or refers to symbols/types that cannot be resolved inside decodeUserFn.

Common situations: Pipeline proto produced by a different Beam version than the decoder; truncated/corrupted job submission graph; manually edited or cached pipeline files.

Understand the failure class

Background: "cannot parse invalid wire-format data", "cannot unmarshal", "failed unmarshalling": protobuf unmarshal errors explained — this error's family across 10 libraries.

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• bad element
• bad struct encoding
• bad symbol
• bad type
• Could not find enum descriptor

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/876ef086c0566a3a. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/core/runtime/graphx/serialize.go:292

			return nil, errors.WithContextf(wrapped, "decoding dynamic DoFn %v", u)
		}

		t, err := decodeType(u.Dynfn.Type)
		if err != nil {
			wrapped := errors.Wrap(err, "bad type")
			return nil, errors.WithContextf(wrapped, "failed to decode dynamic DoFn %v", u)
		}
		return graph.NewFn(&graph.DynFn{
			Name: u.Dynfn.Name,
			T:    t,
			Data: u.Dynfn.Data,
			Gen:  gen.(func(string, reflect.Type, []byte) reflectx.Func),
		})
	}
	if u.Fn != nil {
		fn, err := decodeUserFn(u.Fn)
		if err != nil {
			wrapped := errors.Wrap(err, "failed to decode userfn")
			return nil, errors.WithContextf(wrapped, "decoding DoFn %v", u)
		}
		fx, err := funcx.New(reflectx.MakeFunc(fn))
		if err != nil {
			wrapped := errors.Wrap(err, "failed to construct userfn")
			return nil, errors.WithContextf(wrapped, "decoding DoFn %v", u)
		}
		return &graph.Fn{Fn: fx}, nil
	}

	t, err := decodeType(u.Type)
	if err != nil {
		wrapped := errors.Wrap(err, "bad type")
		return nil, errors.WithContextf(wrapped, "decoding structural DoFn %v", u)
	}
	elem := reflect.New(t)
	if err := jsonx.UnmarshalFrom(elem.Interface(), strings.NewReader(u.Opt)); err != nil {
		wrapped := errors.Wrap(err, "bad struct encoding")

View on GitHub (pinned to 12126d8942)