apache/beam · error

SourceConfig.NumElements must be >= 1. Got

Error message

SourceConfig.NumElements must be >= 1. Got: %v

What it means

Panic raised by the synthetic source config builder's Build when cfg.NumElements is zero or negative. The synthetic source must know how many elements to generate per split; Build validates all numeric knobs up front and fails fast on invalid values rather than misbehaving at runtime.

Solutions

1. Call builder.NumElements(n) with n >= 1 before Build().
2. Default a zero computed element count to a sane positive value.
3. Validate the element-count config field before constructing the pipeline.

Example fix

// before
num := flags.Int("num", 0, "elements")
cfg := synthetic.NewSourceConfig().NumElements(*num).InitialSplits(2).KeySize(8).ValueSize(8).Build()
// after
if *num <= 0 {
    *num = 1000
}
cfg := synthetic.NewSourceConfig().NumElements(*num).InitialSplits(2).KeySize(8).ValueSize(8).Build()

if numElements < 1 {
    return fmt.Errorf("synthetic NumElements must be >= 1, got %d", numElements)
}

defer func() {
    if r := recover(); r != nil {
        err = fmt.Errorf("invalid synthetic source config: %v", r)
    }
}()

When it happens

Trigger: Building a SourceConfig without NumElements(n) or with NumElements(0) / negative, often when the count comes from an unset flag or computed value.

Common situations: Load-test scripts where element count is parameterized and the parameter defaults to 0, or tests that only set KeySize/ValueSize.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

• SourceConfig.InitialSplits must be >= 1. Got
• SourceConfig.KeySize must be >= 1. Got
• SourceConfig.NumHotKeys must be >= 0. Got
• SourceConfig.ValueSize must be >= 1. Got
• capacity of cache cannot be negative, got

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/db9dc46f8f6547a4. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/io/synthetic/source.go:260

}

// HotKeyFraction determines the value of hot key fraction.
//
// Valid values are floating point numbers from 0 to 1.
func (b *SourceConfigBuilder) HotKeyFraction(val float64) *SourceConfigBuilder {
	b.cfg.HotKeyFraction = val
	return b
}

// Build constructs the SourceConfig initialized by this builder. It also
// performs error checking on the fields, and panics if any have been set to
// invalid values.
func (b *SourceConfigBuilder) Build() SourceConfig {
	if b.cfg.InitialSplits <= 0 {
		panic(fmt.Sprintf("SourceConfig.InitialSplits must be >= 1. Got: %v", b.cfg.InitialSplits))
	}
	if b.cfg.NumElements <= 0 {
		panic(fmt.Sprintf("SourceConfig.NumElements must be >= 1. Got: %v", b.cfg.NumElements))
	}
	if b.cfg.KeySize <= 0 {
		panic(fmt.Sprintf("SourceConfig.KeySize must be >= 1. Got: %v", b.cfg.KeySize))
	}
	if b.cfg.ValueSize <= 0 {
		panic(fmt.Sprintf("SourceConfig.ValueSize must be >= 1. Got: %v", b.cfg.ValueSize))
	}
	if b.cfg.NumHotKeys < 0 {
		panic(fmt.Sprintf("SourceConfig.NumHotKeys must be >= 0. Got: %v", b.cfg.HotKeyFraction))
	}
	if b.cfg.HotKeyFraction < 0 || b.cfg.HotKeyFraction > 1 {
		panic(fmt.Sprintf("SourceConfig.HotKeyFraction must be a floating point number from 0 and 1. Got: %v", b.cfg.NumHotKeys))
	}
	return b.cfg
}

// BuildFromJSON constructs the SourceConfig by populating it with the parsed
// JSON. Panics if there is an error in the syntax of the JSON or if the input

View on GitHub (pinned to 12126d8942)