What causes "could not detect user main" in apache/beam?

Ensure the pipeline is submitted with a path to the Go file containing func main() (package main)

could not detect user main

Error message

could not detect user main

What it means

When the universal runner stages a job it must compile a worker binary, which requires locating the user's main package file. buildWorkerBinary walks the candidate files and sets `program`; if after the search `program` does not end with '.go', no user main was found and this error is returned. It means Beam could not determine which Go file holds the pipeline entry point to cross-compile.

Source

Thrown at sdks/go/pkg/beam/runners/universal/runnerlib/compile.go:84

//	  /Users/herohde/go/src/github.com/apache/beam/sdks/go/pkg/beam/runners/beamexec/main.go (skip: 2)
//	* /Users/herohde/go/src/github.com/apache/beam/sdks/go/examples/wordcount/wordcount.go (skip: 3)
//	  /usr/local/go/src/runtime/proc.go (skip: 4)      // not always present
//	  /usr/local/go/src/runtime/asm_amd64.s (skip: 4 or 5)
func buildWorkerBinary(ctx context.Context, filename string, opts CompileOpts) error {
	program := ""
	var isTest bool
	for i := 3; ; i++ {
		_, file, _, ok := runtime.Caller(i)
		if !ok || !strings.HasSuffix(file, ".go") || strings.HasSuffix(file, "runtime/proc.go") {
			break
		} else if strings.HasSuffix(file, "testing/testing.go") {
			isTest = true
			break
		}
		program = file
	}
	if !strings.HasSuffix(program, ".go") {
		return errors.New("could not detect user main")
	}
	goos := "linux"
	goarch := "amd64"

	if opts.OS != "" {
		goos = opts.OS
	}
	if opts.Arch != "" {
		goarch = opts.Arch
	}

	cgo := "0"

	log.Infof(ctx, "Cross-compiling %v with GOOS=%s GOARCH=%s CGO_ENABLED=%s as %v", program, goos, goarch, cgo, filename)

	// Cross-compile given go program. Not awesome.
	program = program[:strings.LastIndex(program, "/")+1]
	program = program + "."

View on GitHub (pinned to 12126d8942)

Solutions

Ensure the pipeline is submitted with a path to the Go file containing func main() (package main)
Run `go build` locally first to confirm the main package compiles and is locatable
Pass an explicit pre-built worker binary via --worker_binary to skip source detection
Check opts.WorkingDir/env so the file scan actually reaches your .go files

Example fix

// before
beamx.Run(ctx, p) // in a file without package main
// after
// ensure cmd/pipeline/main.go declares:
package main

func main() {
	beam.Init()
	// ... build and run pipeline
}

Defensive patterns

Strategy: validation

Validate before calling

// before submitting, confirm the main file exists and is package main
fi, err := os.Stat(mainFile)
if err != nil || fi.IsDir() || !strings.HasSuffix(mainFile, ".go") {
	return fmt.Errorf("main file %q is not a .go source file", mainFile)
}

Try / catch

if _, err := runnerlib.BuildTempWorkerBinary(ctx, prog, opts); err != nil {
	if strings.Contains(err.Error(), "could not detect user main") {
		return fmt.Errorf("supply --worker_binary or run from a package/main directory: %w", err)
	}
	return err
}

Prevention

Run jobs from a directory whose main package contains func main()
Verify `go build ./...` succeeds before remote submission
Pass an explicit --worker_binary when source layout is nonstandard

When it happens

Trigger: Calling universal runner's BuildTempWorkerBinary / running on remote runners (Dataflow, Flink, Spark) with a program path that is not a .go file, an empty program, or a directory of files where none was identified as the main.

Common situations: Cross-compiling pipelines where --worker_binary or the main package path is misconfigured; running compiled test binaries; submitting jobs from modules whose main package is not discoverable.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/5478ec1c6fc379c1. Report an issue: GitHub.