apache/beam · error

Params.BatchSizeBytes must be >= 0; got

Error message

Params.BatchSizeBytes must be >= 0; got %d

What it means

GroupIntoBatches validates Params.BatchSizeBytes, the byte-size threshold per batch, and requires it to be non-negative. Negative values are treated as invalid configuration and rejected before the pipeline runs.

Solutions

1. Set BatchSizeBytes to a non-negative value (0 is allowed if BatchSize > 0)
2. Validate limits at config load time before constructing Params
3. Check the sign of computed byte budgets (e.g. total - overhead can go negative)

Example fix

// before
p := batch.Params{BatchSizeBytes: budget - overhead} // can be negative
// after
bs := budget - overhead
if bs < 0 { bs = 0 }
p := batch.Params{BatchSizeBytes: bs}

if p.BatchSizeBytes < 0 { return errors.New("BatchSizeBytes must be >= 0") } // before calling GroupIntoBatches

if err := p.Validate(); err != nil {
    return fmt.Errorf("invalid batch params: %w", err)
}

When it happens

Trigger: Passing batch.Params with a negative BatchSizeBytes (e.g. -1024) to GroupIntoBatches or GroupIntoBatchesWithShardedKey.

Common situations: Computing byte limits from a signed config that went negative, typos in defaults, or misreading MaxBufferingDuration as a byte field and negating it.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

• Params: at least one of BatchSize or BatchSizeBytes must be…
• Params.BatchSize must be >= 0; got
• project cannot be empty, got
• buffer_sec must be >= 0, got
• capacity of cache cannot be negative, got

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/52700b579b124fbf. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/transforms/batch/batch.go:186

	// BatchSizeBytes is the target maximum cumulative byte size per
	// batch. A batch is emitted as soon as adding another element
	// would exceed BatchSizeBytes. Zero disables the byte-based
	// trigger.
	BatchSizeBytes int64

	// MaxBufferingDuration, when > 0, triggers emission of a partial
	// batch after this much processing time has elapsed since the
	// first element of the current batch was buffered.
	MaxBufferingDuration time.Duration
}

func (p Params) validate() error {
	if p.BatchSize < 0 {
		return fmt.Errorf("Params.BatchSize must be >= 0; got %d", p.BatchSize)
	}
	if p.BatchSizeBytes < 0 {
		return fmt.Errorf("Params.BatchSizeBytes must be >= 0; got %d", p.BatchSizeBytes)
	}
	if p.BatchSize == 0 && p.BatchSizeBytes == 0 {
		return fmt.Errorf("Params: at least one of BatchSize or BatchSizeBytes must be > 0")
	}
	if p.MaxBufferingDuration < 0 {
		return fmt.Errorf("Params.MaxBufferingDuration must be >= 0; got %s", p.MaxBufferingDuration)
	}
	return nil
}

const (
	sizerNone      int32 = 0
	sizerPrimitive int32 = 1
)

// codecCache keeps a per-value-type ElementEncoder/Decoder pair.
type codecCache struct {
	once sync.Once

View on GitHub (pinned to 12126d8942)