apache/beam · error

Params.BatchSize must be >= 0; got

Error message

Params.BatchSize must be >= 0; got %d

What it means

GroupIntoBatches validates its Params before building the transform: BatchSize counts elements per batch and must be non-negative. A negative value indicates a configuration bug and is rejected immediately.

Solutions

1. Set BatchSize to a positive value, or 0 if BatchSizeBytes is used instead
2. Clamp/validate user- or config-supplied sizes before constructing Params: if n < 0 { n = 0 }
3. Use an unsigned or validated config source for batch sizing

Example fix

// before
p := batch.Params{BatchSize: cfg.Size} // may be -1
// after
size := cfg.Size
if size < 0 { size = 0 }
p := batch.Params{BatchSize: size}

if p.BatchSize < 0 { return errors.New("BatchSize must be >= 0") } // before calling GroupIntoBatches

if err := p.Validate(); err != nil { // or catch from GroupIntoBatches
    return fmt.Errorf("invalid batch params: %w", err)
}

When it happens

Trigger: Constructing batch.Params{BatchSize: -1} (or any negative) and passing it to GroupIntoBatches or GroupIntoBatchesWithShardedKey, often from a computed or user-supplied value.

Common situations: BatchSize computed from a config variable that is negative or uninitialized via a signed type, off-by-one negation when deriving size from another value.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

• Params: at least one of BatchSize or BatchSizeBytes must be…
• Params.BatchSizeBytes must be >= 0; got
• project cannot be empty, got
• buffer_sec must be >= 0, got
• capacity of cache cannot be negative, got

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/d7d1a0a9240f459b. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/transforms/batch/batch.go:183

	// batch is emitted as soon as it holds BatchSize elements. Zero
	// disables the count-based trigger.
	BatchSize int64

	// BatchSizeBytes is the target maximum cumulative byte size per
	// batch. A batch is emitted as soon as adding another element
	// would exceed BatchSizeBytes. Zero disables the byte-based
	// trigger.
	BatchSizeBytes int64

	// MaxBufferingDuration, when > 0, triggers emission of a partial
	// batch after this much processing time has elapsed since the
	// first element of the current batch was buffered.
	MaxBufferingDuration time.Duration
}

func (p Params) validate() error {
	if p.BatchSize < 0 {
		return fmt.Errorf("Params.BatchSize must be >= 0; got %d", p.BatchSize)
	}
	if p.BatchSizeBytes < 0 {
		return fmt.Errorf("Params.BatchSizeBytes must be >= 0; got %d", p.BatchSizeBytes)
	}
	if p.BatchSize == 0 && p.BatchSizeBytes == 0 {
		return fmt.Errorf("Params: at least one of BatchSize or BatchSizeBytes must be > 0")
	}
	if p.MaxBufferingDuration < 0 {
		return fmt.Errorf("Params.MaxBufferingDuration must be >= 0; got %s", p.MaxBufferingDuration)
	}
	return nil
}

const (
	sizerNone      int32 = 0
	sizerPrimitive int32 = 1
)

View on GitHub (pinned to 12126d8942)