apache/beam · error

max_num_workers ( ) cannot be negative

Error message

max_num_workers (%d) cannot be negative

What it means

validateWorkerSettings in dataflowlib also rejects a negative opts.MaxNumWorkers before submitting to Dataflow, since the API requires non-negative worker settings. The job fails at translation time with 'max_num_workers (%d) cannot be negative'.

Solutions

1. Set MaxNumWorkers to 0 (no cap enforced by this check) or a positive integer; never use -1 as a sentinel.
2. Validate/clamp the value at config parse time, e.g. if v < 0 { v = 0 }.
3. Ensure NumWorkers <= MaxNumWorkers when both are positive, or the subsequent check will also fail.
4. Fix quota/derivation logic that can yield a negative max.

Example fix

// before
opts.MaxNumWorkers = -1 // meant "unlimited"
// max_num_workers (-1) cannot be negative

// after
opts.MaxNumWorkers = 0 // unset => Dataflow default cap

if opts.MaxNumWorkers < 0 {
    return errors.New("max_num_workers must be >= 0 (0 = unset)")
}
if opts.NumWorkers > 0 && opts.MaxNumWorkers > 0 && opts.NumWorkers > opts.MaxNumWorkers {
    return errors.New("num_workers exceeds max_num_workers")
}

func maxNumWorkersOk(n int) bool { return n >= 0 }

When it happens

Trigger: Submitting a Dataflow job with PipelineOptions.MaxNumWorkers set below zero — usually from unparsed/negative flag input or arithmetic producing a negative cap, or a -1 sentinel meaning 'no cap'.

Common situations: Config file or CLI passing --max-num-workers=-1 to mean unlimited; computing max from quotas that resolved to negative; copy-pasted defaults containing -1.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

• num_workers ( ) cannot be negative
• Do not set use_gbek directly, pass in the --gbek pipeline…
• Unknown or inapplicable phase for pre_optimize
• You are submitting a pipeline with Apache Beam Python SDK
• A list of URNs for overriding transforms was provided but…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/f2c6200ac1872488. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/runners/dataflow/dataflowlib/job.go:442

		return errors.New("experiment worker_region and option workerZone are mutually exclusive")
	}
	if hasExperimentWorkerRegion && opts.Zone != "" {
		return errors.New("experiment worker_region and option Zone are mutually exclusive")
	}

	if opts.Zone != "" {
		log.Warn(ctx, "Option --zone is deprecated. Please use --workerZone instead.")
		opts.WorkerZone = opts.Zone
		opts.Zone = ""
	}

	numWorkers := opts.NumWorkers
	maxNumWorkers := opts.MaxNumWorkers
	if numWorkers < 0 {
		return fmt.Errorf("num_workers (%d) cannot be negative", numWorkers)
	}
	if maxNumWorkers < 0 {
		return fmt.Errorf("max_num_workers (%d) cannot be negative", maxNumWorkers)
	}
	if numWorkers > 0 && maxNumWorkers > 0 && numWorkers > maxNumWorkers {
		return fmt.Errorf("num_workers (%d) cannot exceed max_num_workers (%d)", numWorkers, maxNumWorkers)
	}
	return nil
}

View on GitHub (pinned to 12126d8942)