JuliusBrussee/caveman · error

cachebench: invalid observation read limits

Error message

cachebench: invalid observation read limits

What it means

ReadObservationJSONLWithLimits requires MaxLineBytes in (0, 64MiB] and MaxRecords in (0, 1,000,000]. Zero, negative, or over-cap values are rejected because the scanner buffer and memory bounds depend on them being sane; the defaults are 8MiB lines and 100k records.

Source

Thrown at cacheengine/cachebench/observed.go:97

type ObservationReadLimits struct {
	MaxLineBytes int
	MaxRecords   int
}

// DefaultObservationReadLimits returns conservative retained-evidence limits.
func DefaultObservationReadLimits() ObservationReadLimits {
	return ObservationReadLimits{MaxLineBytes: 8 << 20, MaxRecords: 100_000}
}

// ReadObservationJSONL reads observation records under default limits.
func ReadObservationJSONL(reader io.Reader) ([]ObservationRecord, error) {
	return ReadObservationJSONLWithLimits(reader, DefaultObservationReadLimits())
}

// ReadObservationJSONLWithLimits reads strict observation JSONL under explicit limits.
func ReadObservationJSONLWithLimits(reader io.Reader, limits ObservationReadLimits) ([]ObservationRecord, error) {
	if limits.MaxLineBytes <= 0 || limits.MaxLineBytes > 64<<20 || limits.MaxRecords <= 0 || limits.MaxRecords > 1_000_000 {
		return nil, errors.New("cachebench: invalid observation read limits")
	}
	scanner := bufio.NewScanner(reader)
	initial := 64 * 1024
	if limits.MaxLineBytes < initial {
		initial = limits.MaxLineBytes
	}
	scanner.Buffer(make([]byte, initial), limits.MaxLineBytes)
	seen := map[string]bool{}
	var records []ObservationRecord
	for line := 1; scanner.Scan(); line++ {
		raw := bytes.TrimSpace(scanner.Bytes())
		if len(raw) == 0 {
			continue
		}
		if len(records) >= limits.MaxRecords {
			return nil, fmt.Errorf("cachebench: observations exceed record limit %d", limits.MaxRecords)
		}
		if !validUniqueJSONObject(raw) {

View on GitHub (pinned to 27d5a3981a)

Solutions

1. Set both fields explicitly within the allowed ranges (or call ReadObservationJSONL to use defaults)
2. Derive custom limits from DefaultObservationReadLimits() and adjust one field
3. Treat 0-values in your config as 'use defaults' and substitute before calling

Example fix

// before
limits := ObservationReadLimits{MaxLineBytes: 1 << 20} // MaxRecords == 0
recs, err := ReadObservationJSONLWithLimits(r, limits)

// after
limits := DefaultObservationReadLimits()
limits.MaxLineBytes = 1 << 20
recs, err := ReadObservationJSONLWithLimits(r, limits)

func validReadLimits(l ObservationReadLimits) bool {
    return l.MaxLineBytes > 0 && l.MaxLineBytes <= 64<<20 && l.MaxRecords > 0 && l.MaxRecords <= 1_000_000
}

limits := DefaultObservationReadLimits()
if !validReadLimits(limits) { return errors.New("bad limits") }

func withinObservationLimits(l ObservationReadLimits) bool {
    return l.MaxLineBytes > 0 && l.MaxLineBytes <= 64<<20 && l.MaxRecords > 0 && l.MaxRecords <= 1_000_000
}

if _, err := cachebench.ReadObservationJSONLWithLimits(r, limits); err != nil {
    if err.Error() == "cachebench: invalid observation read limits" {
        limits = cachebench.DefaultObservationReadLimits() // reset and retry once
    }
}

When it happens

Trigger: Passing a custom ObservationReadLimits with MaxLineBytes <= 0 or > 64<<20, or MaxRecords <= 0 or > 1_000_000 — e.g. copying a config where the fields defaulted to zero because only one was set.

Common situations: Partial struct literal ObservationReadLimits{MaxLineBytes: 1<<20} leaving MaxRecords zero; config-driven limits where a missing key decodes as 0; attempting to disable limits by setting 0.

Related errors

• cachebench: provider population exceeds 1024
• cachebench: provider population exceeds 1024
• cacheengine: stable prefix exceeds configured byte limit
• cacheengine: request exceeds configured byte limit
• row limit %d exceeded

AI-assisted analysis of JuliusBrussee/caveman@27d5a3981a (2026-08-15). Data as JSON: /api/errors/9f0ddd8a65292254. Report an issue: GitHub.