apache/beam · error

invalid CombinePayload payload for %v

Error message

invalid CombinePayload payload for %v

What it means

makeLink decodes pipepb.CombinePayload for per-key combine transforms (pre/merge/extract/convert URNs). A payload that fails proto.Unmarshal is wrapped as this error with the transform name, stopping plan translation.

Source

Thrown at sdks/go/pkg/beam/core/runtime/exec/translate.go:490

		var userTimers map[string]*pipepb.TimerFamilySpec
		switch urn {
		case graphx.URNParDo,
			urnPairWithRestriction,
			urnSplitAndSizeRestrictions,
			urnProcessSizedElementsAndRestrictions,
			urnTruncateSizedRestrictions:
			var pardo pipepb.ParDoPayload
			if err := proto.Unmarshal(payload, &pardo); err != nil {
				return nil, errors.Wrapf(err, "invalid ParDo payload for %v", transform)
			}
			data = string(pardo.GetDoFn().GetPayload())
			sides = pardo.GetSideInputs()
			userState = pardo.GetStateSpecs()
			userTimers = pardo.GetTimerFamilySpecs()
		case urnPerKeyCombinePre, urnPerKeyCombineMerge, urnPerKeyCombineExtract, urnPerKeyCombineConvert:
			var cmb pipepb.CombinePayload
			if err := proto.Unmarshal(payload, &cmb); err != nil {
				return nil, errors.Wrapf(err, "invalid CombinePayload payload for %v", transform)
			}
			data = string(cmb.GetCombineFn().GetPayload())
		default:
			// TODO(herohde) 12/4/2017: we see DoFns directly with Dataflow. Handle that
			// case here, for now, so that the harness can use this logic.

			data = string(payload)
		}

		// TODO(herohde) 1/28/2018: Once Dataflow's fully off the old way,
		// we can simply switch on the ParDo DoFn URN directly.

		var tp v1pb.TransformPayload
		if err := protox.DecodeBase64(data, &tp); err != nil {
			return nil, errors.Wrapf(err, "invalid transform payload for %v", transform)
		}

		switch tpUrn := tp.GetUrn(); tpUrn {

View on GitHub (pinned to 12126d8942)

Solutions

1. Ensure combine transforms serialize a proper pipepb.CombinePayload including the CombineFn
2. Construct combines via the beam.Combine API so payload encoding is done by the SDK
3. Verify the runner preserves payload bytes unchanged through the protocol

// Validate combine payload ahead of translation
var cmb pipepb.CombinePayload
if err := proto.Unmarshal(payload, &cmb); err != nil {
    return fmt.Errorf("bad CombinePayload: %w", err)
}

if err := exec.UnmarshalPlan(desc); err != nil {
    if strings.Contains(err.Error(), "invalid CombinePayload payload") {
        return fmt.Errorf("combine transform graph malformed: %w", err)
    }
    return err
}

When it happens

Trigger: A combine transform (urnPerKeyCombinePre/Merge/Extract/Convert) whose spec payload isn't a valid serialized CombinePayload, encountered in makeLink.

Common situations: Manually assembled pipeline protos missing CombinePayload fields; cross-SDK/runner encoding mismatches for combine transforms; corrupted payloads.

Understand the failure class

Background: "cannot parse invalid wire-format data", "cannot unmarshal", "failed unmarshalling": protobuf unmarshal errors explained — this error's family across 10 libraries.

Related errors

• combine does not support side inputs
• empty type
• invalid ParDo payload for %v
• invalid transform payload for %v
• invalid transform payload %v for %v

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/419b30bf4da2962a. Report an issue: GitHub.