What causes "runner %v already defined" in apache/beam?

Remove the duplicate RegisterRunner call

runner already defined

Error message

runner %v already defined

What it means

RegisterRunner panics when asked to register a runner name that is already present in the global runner registry. Runners are registered once at package init time, so a duplicate name indicates two registrations of the same runner identifier.

Solutions

Remove the duplicate RegisterRunner call
Use a unique runner name for your custom runner
Guard registration with a check of an exported map or a sync.Once before calling RegisterRunner
If intentionally overriding, fork the registry instead of re-registering the built-in name

Example fix

// before
func init() { beam.RegisterRunner("direct", myFn) } // panics: already defined
// after
func init() { beam.RegisterRunner("mydirect", myFn) }

Defensive patterns

Strategy: validation

Validate before calling

if _, ok := beam.Runners()[name]; ok {
    return // already registered
}
beam.RegisterRunner(name, fn)

Try / catch

// Go panics are not recoverable by type; use defer/recover at init boundary if needed
func safeRegister(name string, fn RunnerFn) {
    defer func() { _ = recover() }()
    beam.RegisterRunner(name, fn)
}

Prevention

Register each runner name exactly once, ideally in a single package
Use unique names like org-runner-name for custom runners
Never re-register built-in runner names
Keep init() registration in one file per runner

When it happens

Trigger: Calling beam.RegisterRunner("x", fn) twice, or two packages' init() functions both registering the same runner name (including re-registering a built-in like "direct" or "dataflow").

Common situations: Importing a custom runner package plus manually registering the same name, or copy-pasted init code in test files that registers the standard runner again.

Understand the failure class

Background: "already exists" / EEXIST / FileAlreadyExistsException: what the 'file already exists' error means and how to fix it — this error's family across 37 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/da83bc69ffae7dd3. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/runner.go:33

package beam

import (
	"context"
	"fmt"

	"github.com/apache/beam/sdks/v2/go/pkg/beam/log"
)

var (
	runners = make(map[string]func(ctx context.Context, p *Pipeline) (PipelineResult, error))
)

// RegisterRunner associates the name with the supplied runner, making it available
// to execute a pipeline via Run.
func RegisterRunner(name string, fn func(ctx context.Context, p *Pipeline) (PipelineResult, error)) {
	if _, ok := runners[name]; ok {
		panic(fmt.Sprintf("runner %v already defined", name))
	}
	runners[name] = fn
}

// Run executes the pipeline using the selected registred runner. It is customary
// to define a "runner" with no default as a flag to let users control runner
// selection.
func Run(ctx context.Context, runner string, p *Pipeline) (PipelineResult, error) {
	fn, ok := runners[runner]
	if !ok {
		log.Exitf(ctx, "Runner %v not registered. Forgot to _ import it?", runner)
	}
	return fn(ctx, p)
}

View on GitHub (pinned to 12126d8942)