apache/beam · error

unexpected payload

Error message

unexpected payload: %v

What it means

Inside the ParDo/Combine payload switch in makeLink, the TransformPayload's URN was not any of the known edge URNs (DoFn, IterableSideInputKey, Inject, Expand, ReshuffleInput/Output). The translator has no handler for that payload and fails with 'unexpected payload'.

Solutions

1. Upgrade the Go Beam SDK harness to match the version that constructed the pipeline
2. Print & inspect tp.GetUrn() to identify the unhandled payload kind
3. Update the SDK so the transform uses a supported representation, or remove the unsupported transform from the pipeline

Example fix

null

var tp v1pb.TransformPayload
if err := protox.DecodeBase64(data, &tp); err != nil {
	return err
}
switch tp.GetUrn() {
case graphx.URNDoFn, graphx.URNInject, graphx.URNExpand,
	graphx.URNReshuffleInput, graphx.URNReshuffleOutput,
	graphx.URNIterableSideInputKey:
	// supported
default:
	return fmt.Errorf("unsupported transform payload URN %q", tp.GetUrn())
}

null

if _, err := graph.MakePipeline(pipelineProto); err != nil {
	var transErr interface{ Error() string }
	if errors.As(err, &transErr) && strings.Contains(err.Error(), "unexpected payload") {
		// version skew: upgrade harness and retry resubmission
	}
}

When it happens

Trigger: A TransformPayload with an unknown/unsupported URN inside a ParDo transform — usually produced by a newer SDK writing a payload kind an older harness can't handle, or by a corrupted/misparsed base64 payload that decoded with an empty URN.

Common situations: Beam version skew (pipeline built with newer SDK, Go harness older); cross-language pipelines emitting payload types the Go exec translator lacks; corrupt job submission.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• expected one output from DataSource, got
• failed to serialize
• invalid transform payload
• pcollection not found
• unexpected sideinput to combine: got

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/9164fbff33a08262. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/core/runtime/exec/translate.go:780

			var pid string
			// There's only one output PCollection, and iterating through the map
			// is the only way to extract it.
			for _, id := range transform.GetOutputs() {
				pid = id
			}
			_, w, err := b.makeCoderForPCollection(pid)
			if err != nil {
				return nil, err
			}
			preservedCoderID := tp.GetReshuffle().GetCoderId()
			pc, err := unmarshalReshuffleCoders(preservedCoderID, tp.GetReshuffle().GetCoderPayloads())
			if err != nil {
				return nil, err
			}
			u = &ReshuffleOutput{UID: b.idgen.New(), Coder: coder.NewW(pc, w), Out: out[0]}

		default:
			return nil, errors.Errorf("unexpected payload: %v", &tp)
		}

	case graphx.URNWindow:
		var wp pipepb.WindowIntoPayload
		if err := proto.Unmarshal(payload, &wp); err != nil {
			return nil, errors.Wrapf(err, "invalid WindowInto payload for %v", transform)
		}
		wfn, err := unmarshalWindowFn(wp.GetWindowFn())
		if err != nil {
			return nil, err
		}
		u = &WindowInto{UID: b.idgen.New(), Fn: wfn, Out: out[0]}

	case graphx.URNMapWindows:
		var fn pipepb.FunctionSpec
		if err := proto.Unmarshal(payload, &fn); err != nil {
			return nil, errors.Wrapf(err, "invalid SideInput payload for %v", transform)
		}

View on GitHub (pinned to 12126d8942)