apache/beam · error

panic(formatParDoError(dofn, len(ret), 1))

Error message

panic(formatParDoError(dofn, len(ret), 1))

What it means

ParDo applies a ParDo transform that must produce exactly one output PCollection. If TryParDo returns a number of PCollections other than 1, it panics with a formatted ParDo error naming the dofn, actual count, and expected count of 1.

Source

Thrown at sdks/go/pkg/beam/pardo.go:431

// one between a producer ParDo and a consumer ParDo), the PCollection itself
// is "fused away" and won't ever be written to disk, saving all the I/O and
// space expense of constructing it.
//
// When Beam runners apply fusion optimization, it is essentially "free" to
// write ParDo operations in a very modular, composable style, each ParDo
// operation doing one clear task, and stringing together sequences of ParDo
// operations to get the desired overall effect. Such programs can be easier to
// understand, easier to unit-test, easier to extend and evolve, and easier to
// reuse in new programs. The predefined library of PTransforms that come with
// Beam makes heavy use of this modular, composable style, trusting to the
// runner to "flatten out" all the compositions into highly optimized stages.
//
// See https://beam.apache.org/documentation/programming-guide/#pardo
// for the web documentation for ParDo
func ParDo(s Scope, dofn any, col PCollection, opts ...Option) PCollection {
	ret := MustN(TryParDo(s, dofn, col, opts...))
	if len(ret) != 1 {
		panic(formatParDoError(dofn, len(ret), 1))
	}
	return ret[0]
}

// TODO(herohde) 6/1/2017: add windowing aspects to above documentation.

// ParDo2 inserts a ParDo with 2 outputs into the pipeline.
func ParDo2(s Scope, dofn any, col PCollection, opts ...Option) (PCollection, PCollection) {
	ret := MustN(TryParDo(s, dofn, col, opts...))
	if len(ret) != 2 {
		panic(formatParDoError(dofn, len(ret), 2))
	}
	return ret[0], ret[1]
}

// ParDo3 inserts a ParDo with 3 outputs into the pipeline.
func ParDo3(s Scope, dofn any, col PCollection, opts ...Option) (PCollection, PCollection, PCollection) {
	ret := MustN(TryParDo(s, dofn, col, opts...))

View on GitHub (pinned to 12126d8942)

Solutions

1. Ensure the DoFn's ProcessElement returns exactly one output value (plus optional error per Beam Go conventions)
2. Use ParDo0 for no-output DoFns or ParDo2/ParDo3... for multi-output DoFns
3. Read formatParDoError's message to see the actual vs expected count and adjust accordingly

Example fix

// before
beam.ParDo(s, func(x int, emit func(int), emit2 func(int)) {...}, col) // 2 outputs
// after
o1, o2 := beam.ParDo2(s, func(x int, emit func(int), emit2 func(int)) {...}, col)

// A ParDo DoFn must produce exactly one output per element

func isSingleOutputDoFn(fn any) bool {
    t := reflect.TypeOf(fn)
    for t.Kind() == reflect.Ptr {
        t = t.Elem()
    }
    m, ok := t.MethodByName("ProcessElement")
    if !ok { return false }
    // heuristic: exactly one emit param or one non-error return value
    numEmit := 0
    for i := 0; i < m.Type.NumIn(); i++ {
        if m.Type.In(i).Kind() == reflect.Func { numEmit++ }
    }
    return numEmit == 1 || (m.Type.NumOut() == 1)
}

defer func() {
    if r := recover(); r != nil {
        err = fmt.Errorf("ParDo arity mismatch: %v", r)
    }
}()

When it happens

Trigger: Calling beam.ParDo with a DoFn that returns zero values (side-effect only) or multiple outputs (e.g. (string, int) or ProcessElement with multiple emitters).

Common situations: Using a multi-output DoFn (written for ParDo2/ParDo3) with ParDo, or a sink-style DoFn with no returns passed to ParDo after a refactor.

Related errors

• panic(formatParDoError(dofn, len(ret), 0))
• panic(formatParDoError(dofn, len(ret), 2))
• unable to decode ParDoPayload for %v
• %v cannot bind to %v
• CreateWatermarkEstimator fn %v has unexpected number of para

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/249e579f0b1f5f95. Report an issue: GitHub.