What causes "cachebench: no observation records" in JuliusBrussee/caveman?

Verify the observation file is non-empty and starts with a JSON object line before parsing

cachebench: no observation records

Error message

cachebench: no observation records

What it means

ReadObservationJSONLWithLimits parsed zero records from the input stream and treats that as an error: an observations file with no usable lines (only whitespace or empty) cannot support any evaluation. The check runs after the scan loop and scanner error check.

Source

Thrown at cacheengine/cachebench/observed.go:142

		var trailing any
		if err := decoder.Decode(&trailing); err != io.EOF {
			return nil, fmt.Errorf("cachebench: observation line %d: trailing JSON", line)
		}
		if record.Schema != ObservationSchema {
			return nil, fmt.Errorf("cachebench: observation line %d: schema %q", line, record.Schema)
		}
		if strings.TrimSpace(record.RequestID) == "" || seen[record.RequestID] {
			return nil, fmt.Errorf("cachebench: observation line %d: empty or duplicate request_id", line)
		}
		seen[record.RequestID] = true
		record.Usage = append(json.RawMessage(nil), record.Usage...)
		records = append(records, record)
	}
	if err := scanner.Err(); err != nil {
		return nil, fmt.Errorf("cachebench: read observations: %w", err)
	}
	if len(records) == 0 {
		return nil, errors.New("cachebench: no observation records")
	}
	if err := validateObservationRecords(records); err != nil {
		return nil, err
	}
	return records, nil
}

// EvaluateObserved evaluates supplied observations without completeness claim.
func EvaluateObserved(records []ObservationRecord, target Target) (Report, error) {
	if err := validateTarget(target); err != nil {
		return Report{}, err
	}
	if len(records) == 0 {
		return Report{}, errors.New("cachebench: no observation records")
	}
	if err := validateObservationRecords(records); err != nil {
		return Report{}, err
	}

View on GitHub (pinned to 27d5a3981a)

Solutions

Verify the observation file is non-empty and starts with a JSON object line before parsing
Check file size / first bytes after export and treat 0 bytes as an export failure
If empty input is legitimate in your flow, catch this specific error and handle it as 'nothing to evaluate'

Example fix

// before
recs, err := ReadObservationJSONL(emptyFile) // error: no observation records

// after
if fi, _ := f.Stat(); fi.Size() == 0 {
    return fmt.Errorf("observation export %s is empty; rerun exporter", fi.Name())
}
recs, err := ReadObservationJSONL(f)

Defensive patterns

Strategy: validation

Validate before calling

fi, err := os.Stat(path)
if err != nil { return err }
if fi.Size() == 0 {
    return fmt.Errorf("observation file %s is empty", path)
}
records, err := cachebench.ReadObservationJSONL(f)

Try / catch

records, err := cachebench.ReadObservationJSONL(f)
if err != nil {
    if err.Error() == "cachebench: no observation records" {
        // treat as export failure: rerun exporter or flag pipeline
    }
    return err
}

Prevention

Check observation file size right after export; zero bytes means the exporter failed
Don't share one io.Reader across parsers — a consumed reader yields zero records
Validate the first line parses as a JSON object before committing to full read

When it happens

Trigger: Feeding an empty file, a file of blank lines, or a reader already consumed to EOF to ReadObservationJSONL. All lines were skipped as blank, so records stays empty and the function errors instead of returning an empty slice.

Common situations: Observation export produced an empty file (job crashed before writing); reading a file whose content was already consumed by an earlier scanner; wrong path resolving to an empty placeholder file.

Related errors

AI-assisted analysis of JuliusBrussee/caveman@27d5a3981a (2026-08-15). Data as JSON: /api/errors/e92f6408813b2397. Report an issue: GitHub.