apache/beam · error

expected one output from DataSource, got %v

Error message

expected one output from DataSource, got %v

What it means

UnmarshalPlan translates a pipeline description into an executable plan. The DataSource transform must have exactly one output; if the harness receives a DataSource transform with zero or multiple outputs, the plan cannot be linked and this error is returned.

Source

Thrown at sdks/go/pkg/beam/core/runtime/exec/translate.go:64

	urnPerKeyCombineConvert                = "beam:transform:combine_per_key_convert_to_accumulators:v1"
	urnPairWithRestriction                 = "beam:transform:sdf_pair_with_restriction:v1"
	urnSplitAndSizeRestrictions            = "beam:transform:sdf_split_and_size_restrictions:v1"
	urnProcessSizedElementsAndRestrictions = "beam:transform:sdf_process_sized_element_and_restrictions:v1"
	urnTruncateSizedRestrictions           = "beam:transform:sdf_truncate_sized_restrictions:v1"
)

// UnmarshalPlan converts a model bundle descriptor into an execution Plan.
func UnmarshalPlan(desc *fnpb.ProcessBundleDescriptor, dataSampler *DataSampler) (*Plan, error) {
	b, err := newBuilder(desc, dataSampler)
	if err != nil {
		return nil, err
	}
	for id, transform := range desc.GetTransforms() {
		if transform.GetSpec().GetUrn() != urnDataSource {
			continue
		}
		if len(transform.GetOutputs()) != 1 {
			return nil, errors.Errorf("expected one output from DataSource, got %v", transform.GetOutputs())
		}

		port, cid, err := unmarshalPort(transform.GetSpec().GetPayload())
		if err != nil {
			return nil, err
		}

		u := &DataSource{UID: b.idgen.New()}
		u.Coder, err = b.coders.Coder(cid) // Expected to be windowed coder
		if err != nil {
			return nil, err
		}
		if !coder.IsW(u.Coder) {
			return nil, errors.Errorf("unwindowed coder %v on DataSource %v: %v", cid, id, u.Coder)
		}

		// There's only a single pair in this map, but a for loop range statement
		// is the easiest way to extract it, so this loop will iterate only once.

View on GitHub (pinned to 12126d8942)

Solutions

1. Inspect the pipeline description proto and ensure the DataSource transform declares exactly one output
2. Upgrade/align the Beam Go SDK and runner (e.g. Flink/Spark/Dataflow) versions so the graph encoding matches
3. Reproduce with a minimal pipeline and file an issue if the runner emits multi-output DataSource transforms

// Pre-validate the incoming pipeline description
for _, t := range desc.GetTransforms() {
    if t.GetSpec().GetUrn() == "beam:runner:source:v1" && len(t.GetOutputs()) != 1 {
        return fmt.Errorf("bad DataSource outputs: %v", t.GetOutputs())
    }
}

plan, err := exec.UnmarshalPlan(desc)
if err != nil {
    return fmt.Errorf("unmarshal plan failed: %w", err) // surface for runner-side diagnosis
}

When it happens

Trigger: Unmarshalling a runner-provided pipeline description (desc) whose DataSource transform (urnDataSource) has an Outputs map with a length other than 1, via getOrCreatePlan/UnmarshalPlan.

Common situations: Buggy or incompatible runner/harness protocol negotiation; hand-crafted or mutated pipeline protos in tests; SDK/runner version mismatch producing malformed graph payloads.

Related errors

• pcollection %v not found
• windowing strategy %v not found
• invalid transform payload %v for %v
• Unrecognized state type %v
• unexpected sideinput to combine: got %d, want 1

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/8d0026469395e740. Report an issue: GitHub.