apache/beam · error

missing side input info for collection

Error message

missing side input info for collection %v

What it means

makeLink wires up the inputs of a transform during pipeline translation. When an input edge is expected to be a side input, the translation looks up its side-input descriptor (window mapping etc.) in the collected side info; if absent for the given input PCollection, translation fails naming the collection. This means the pipeline proto lacks side-input metadata for an input the consumer treats as a side input.

Solutions

1. Rebuild the pipeline with the same Beam SDK version for construction and execution so side-input info is serialized.
2. Inspect the pipeline proto: ensure each side-input input index has a matching entry in the transform's side inputs map.
3. Check custom translators don't reorder/drop inputs so that indexToInputId mapping breaks.
4. If hand-crafting protos (tests), populate the side input spec for every side-input PCollection.

Example fix

// before: side input declared in DoFn but not in payload
sidePB, ok := sides[indexToInputId(i)]
// after (pipeline construction): ensure the input is registered as a side input
beam.ParDo(s, fn, main, beam.SideInput{Input: sidePCol}) // not a plain input arg

if _, ok := sides[indexToInputId(i)]; !ok { return fmt.Errorf("input %d (%v) lacks side input info", i, input[i]) }

if err != nil && strings.Contains(err.Error(), "missing side input info") {
    return nil, fmt.Errorf("pipeline proto corrupt or version-skewed: %w", err)
}

When it happens

Trigger: Translating a pipeline where a ParDo declares an input index as a side input, but the corresponding SideInput proto is missing from the transform's spec (indexToInputId lookup misses).

Common situations: Pipelines produced/modified by other SDKs or tools that dropped side-input metadata; version skew between job-submission and runner; hand-built pipeline protos in tests; custom translators reordering inputs.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

• cannot make a keyed iterable for an unkeyed side input
• error with side input
• main input is global windowed in DoFn
• unsupported window mapping fn URN
• Attempted to get side input window for GlobalWindow from…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/af7d9432a7b091a9. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/core/runtime/exec/translate.go:627

							if err != nil {
								return nil, errors.WithContextf(err, "couldn't retreive coder for timer %v in DoFn %v, ID %v", fam, dofn.Name(), n.PID)
							}
							familyToSpec[fam] = newTimerFamilySpec(domain, timerCoder)
						}
						n.TimerTracker = newUserTimerAdapter(sID, familyToSpec)
					}

					for i := 1; i < len(input); i++ {
						// TODO(https://github.com/apache/beam/issues/18602) Handle ViewFns for side inputs

						ec, wc, err := b.makeCoderForPCollection(input[i])
						if err != nil {
							return nil, err
						}

						sidePB, ok := sides[indexToInputId(i)]
						if !ok {
							return nil, fmt.Errorf("missing side input info for collection %v", input[i])
						}

						mapper, err := unmarshalAndMakeWindowMapping(sidePB.GetWindowMappingFn())
						if err != nil {
							return nil, err
						}

						sid := StreamID{
							Port:         Port{URL: b.desc.GetStateApiServiceDescriptor().GetUrl()},
							PtransformID: id.to,
						}
						sideInputID := fmt.Sprintf("i%v", i) // SideInputID (= local id, "iN")
						side := NewSideInputAdapter(sid, sideInputID, coder.NewW(ec, wc), mapper)
						n.Side = append(n.Side, side)
					}
					u = n
					if urn == urnProcessSizedElementsAndRestrictions {
						outputs := make([]string, len(transform.GetOutputs()))

View on GitHub (pinned to 12126d8942)