What causes "panic(formatParDoError(dofn, len(ret), 2))" in apache/beam?

Give the DoFn exactly two outputs (two emit funcs or two return values)

panic(formatParDoError(dofn, len(ret), 2))

Error message

panic(formatParDoError(dofn, len(ret), 2))

What it means

ParDo2 applies a ParDo transform that must produce exactly two output PCollections. If TryParDo returns a count other than 2, it panics with a formatted ParDo error stating the dofn, actual count, and expected count of 2.

Source

Thrown at sdks/go/pkg/beam/pardo.go:442

// runner to "flatten out" all the compositions into highly optimized stages.
//
// See https://beam.apache.org/documentation/programming-guide/#pardo
// for the web documentation for ParDo
func ParDo(s Scope, dofn any, col PCollection, opts ...Option) PCollection {
	ret := MustN(TryParDo(s, dofn, col, opts...))
	if len(ret) != 1 {
		panic(formatParDoError(dofn, len(ret), 1))
	}
	return ret[0]
}

// TODO(herohde) 6/1/2017: add windowing aspects to above documentation.

// ParDo2 inserts a ParDo with 2 outputs into the pipeline.
func ParDo2(s Scope, dofn any, col PCollection, opts ...Option) (PCollection, PCollection) {
	ret := MustN(TryParDo(s, dofn, col, opts...))
	if len(ret) != 2 {
		panic(formatParDoError(dofn, len(ret), 2))
	}
	return ret[0], ret[1]
}

// ParDo3 inserts a ParDo with 3 outputs into the pipeline.
func ParDo3(s Scope, dofn any, col PCollection, opts ...Option) (PCollection, PCollection, PCollection) {
	ret := MustN(TryParDo(s, dofn, col, opts...))
	if len(ret) != 3 {
		panic(formatParDoError(dofn, len(ret), 3))
	}
	return ret[0], ret[1], ret[2]
}

// ParDo4 inserts a ParDo with 4 outputs into the pipeline.
func ParDo4(s Scope, dofn any, col PCollection, opts ...Option) (PCollection, PCollection, PCollection, PCollection) {
	ret := MustN(TryParDo(s, dofn, col, opts...))
	if len(ret) != 4 {
		panic(formatParDoError(dofn, len(ret), 4))

View on GitHub (pinned to 12126d8942)

Solutions

Give the DoFn exactly two outputs (two emit funcs or two return values)
Use ParDo for one output or ParDo3 for three outputs to match the DoFn
Check the counts in the panic message and align the helper variant with the DoFn signature

Example fix

// before
beam.ParDo2(s, func(x int, emit func(int)) {...}, col) // 1 output, expects 2
// after
beam.ParDo(s, func(x int, emit func(int)) {...}, col)

Defensive patterns

Strategy: type-guard

Validate before calling

// A ParDo2 DoFn must produce exactly two outputs

Type guard

func isTwoOutputDoFn(fn any) bool {
    t := reflect.TypeOf(fn)
    for t.Kind() == reflect.Ptr {
        t = t.Elem()
    }
    m, ok := t.MethodByName("ProcessElement")
    if !ok { return false }
    numEmit := 0
    for i := 0; i < m.Type.NumIn(); i++ {
        if m.Type.In(i).Kind() == reflect.Func { numEmit++ }
    }
    return numEmit == 2 || m.Type.NumOut() == 2
}

Try / catch

defer func() {
    if r := recover(); r != nil {
        err = fmt.Errorf("ParDo2 arity mismatch: %v", r)
    }
}()

Prevention

Ensure DoFns passed to ParDo2 emit exactly two outputs (two emits or two returns)
Update the helper variant whenever a DoFn's output count changes
Add small pipeline tests per transform to catch arity drift at construction time

When it happens

Trigger: Calling beam.ParDo2 with a DoFn that emits only one output or three+ outputs (wrong number of emit functions or return values).

Common situations: Adding or removing an output emitter from a DoFn without switching from ParDo2 to ParDo/ParDo3, or copy-pasting a DoFn between call sites with different arity helpers.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/e9f5eb537312433e. Report an issue: GitHub.