apache/beam · error

num_workers ( ) cannot be negative

Error message

num_workers (%d) cannot be negative

What it means

validateWorkerSettings checks Dataflow pipeline options before job translation. If opts.NumWorkers is negative, the Dataflow job request would be invalid, so translation fails early with 'num_workers (%d) cannot be negative'. This is a straightforward config sanity check.

Solutions

1. Set NumWorkers to 0 to let Dataflow auto-select workers, or a positive integer; remove any -1 sentinel usage.
2. Validate user-supplied flags before constructing options: reject or clamp negatives at the CLI/config layer.
3. Fix arithmetic that derives NumWorkers (e.g. clamp with math.Max(0, computed)).
4. If using Go flag parsing, add a custom flag validator or check os.Args values early.

Example fix

// before
opts.NumWorkers = requested - reserved // can go negative

// after
opts.NumWorkers = requested - reserved
if opts.NumWorkers < 0 {
    opts.NumWorkers = 0 // let Dataflow autoscale
}

if opts.NumWorkers < 0 {
    return errors.New("num_workers must be >= 0 (0 lets Dataflow choose)")
}

func numWorkersOk(n int) bool { return n >= 0 }

When it happens

Trigger: Submitting a Dataflow job where PipelineOptions.NumWorkers was computed from user input/flags/env and ended up negative — e.g. a flag parsed without bounds, a subtraction that underflowed, or a default set to -1 to mean 'unset'.

Common situations: CLI flags like --num-workers passed a negative value; templating code that does baseWorkers - reservedWorkers going negative; using -1 as a sentinel for auto-scaling instead of omitting the field.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

• max_num_workers ( ) cannot be negative
• Do not set use_gbek directly, pass in the --gbek pipeline…
• Unknown or inapplicable phase for pre_optimize
• You are submitting a pipeline with Apache Beam Python SDK
• A list of URNs for overriding transforms was provided but…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/281a0a73fb53c14f. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/runners/dataflow/dataflowlib/job.go:439

		return errors.New("experiment worker_region and option workerRegion are mutually exclusive")
	}
	if hasExperimentWorkerRegion && opts.WorkerZone != "" {
		return errors.New("experiment worker_region and option workerZone are mutually exclusive")
	}
	if hasExperimentWorkerRegion && opts.Zone != "" {
		return errors.New("experiment worker_region and option Zone are mutually exclusive")
	}

	if opts.Zone != "" {
		log.Warn(ctx, "Option --zone is deprecated. Please use --workerZone instead.")
		opts.WorkerZone = opts.Zone
		opts.Zone = ""
	}

	numWorkers := opts.NumWorkers
	maxNumWorkers := opts.MaxNumWorkers
	if numWorkers < 0 {
		return fmt.Errorf("num_workers (%d) cannot be negative", numWorkers)
	}
	if maxNumWorkers < 0 {
		return fmt.Errorf("max_num_workers (%d) cannot be negative", maxNumWorkers)
	}
	if numWorkers > 0 && maxNumWorkers > 0 && numWorkers > maxNumWorkers {
		return fmt.Errorf("num_workers (%d) cannot exceed max_num_workers (%d)", numWorkers, maxNumWorkers)
	}
	return nil
}

View on GitHub (pinned to 12126d8942)