apache/beam · error
Expected single input PCollection in reshuffle:
Error message
Expected single input PCollection in reshuffle:
What it means
A Reshuffle transform must consume exactly one input PCollection. When the submitted PTransform's inputs map does not have exactly one entry, the runner panics with the transform's prototext dump, because the elision optimization (rewiring input to downstream consumers) only works 1:1.
Solutions
- Inspect the prototext dump in the panic to see the actual inputs wired into the reshuffle transform.
- Regenerate the pipeline with the standard beam.Reshuffle() API so the SDK produces a single-input reshuffle.
- Fix any custom graph construction/rewriting code to keep reshuffle transforms strictly single-input/single-output.
- Report upstream if a standard pipeline produced a multi-input reshuffle.
Defensive patterns
Strategy: validation
Validate before calling
if t.GetUrnReshuffle() && len(t.GetInputs()) != 1 { /* fix graph before submission */ } Try / catch
defer func(){ if r := recover(); r != nil && strings.Contains(fmt.Sprint(r), "Expected single input PCollection in reshuffle") { /* inspect transform dump */ } }() Prevention
- Build reshuffles only via beam.Reshuffle()
- Keep custom graph rewriters from adding inputs to reshuffle nodes
When it happens
Trigger: A pipeline graph reaches handleReshuffle with a Reshuffle/RedistributeByKey/RedistributeArbitrarily transform that has zero or multiple inputs — typically from a malformed or hand-built graph rather than SDK-generated code.
Common situations: Custom pipeline builders or graph-rewriting tools that wire extra inputs into a reshuffle node; internal prism composites that were flattened incorrectly; corrupted job-submission payloads.
Understand the failure class
Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.
Related errors
- Expected single output PCollection in reshuffle:
- SDK side reshuffle not yet supported
- expected CoGBK, got
- failed to expand Reshuffle transform for edge
- invalid pcollection
AI-assisted analysis of apache/beam@12126d8942 (2026-09-13).
Data as JSON: /api/errors/c58e10aa04ab2975.
Report an issue: GitHub.
Appendix: source
Thrown at sdks/go/pkg/beam/runners/prism/internal/handlerunner.go:185
if h.config.SDKReshuffle {
panic("SDK side reshuffle not yet supported")
}
// A Reshuffle, in principle, is a no-op on the pipeline structure, WRT correctness.
// It could however affect performance, so it exists to tell the runner that this
// point in the pipeline needs a fusion break, to enable the pipeline to change it's
// degree of parallelism.
//
// The change of parallelism goes both ways. It could allow for larger batch sizes
// enable smaller batch sizes downstream if it is infact paralleizable.
//
// But for a single transform node per stage runner, we can elide it entirely,
// since the input collection and output collection types match.
// Get the input and output PCollections, there should only be 1 each.
if len(t.GetInputs()) != 1 {
panic("Expected single input PCollection in reshuffle: " + prototext.Format(t))
}
if len(t.GetOutputs()) != 1 {
panic("Expected single output PCollection in reshuffle: " + prototext.Format(t))
}
inColID := getOnlyValue(t.GetInputs())
outColID := getOnlyValue(t.GetOutputs())
// We need to find all Transforms that consume the output collection and
// replace them so they consume the input PCollection directly.
// We need to remove the consumers of the output PCollection.
toRemove := []string{}
// We need to force the consumers to be stage root,
// because reshuffle should be a fusion break.
forcedRoots := []string{}
for tid, t := range comps.GetTransforms() {View on GitHub (pinned to 12126d8942)