JuliusBrussee/caveman · error

cachebench: observation line %d: %w

Error message

cachebench: observation line %d: %w

What it means

The line passed JSON validation but could not decode into ObservationRecord, and the decoder runs with DisallowUnknownFields — so unknown fields are errors, not silent drops. The '%w' clause carries the encoding/json error naming the offending field or type problem, prefixed with the line number.

Source

Thrown at cacheengine/cachebench/observed.go:122

	scanner.Buffer(make([]byte, initial), limits.MaxLineBytes)
	seen := map[string]bool{}
	var records []ObservationRecord
	for line := 1; scanner.Scan(); line++ {
		raw := bytes.TrimSpace(scanner.Bytes())
		if len(raw) == 0 {
			continue
		}
		if len(records) >= limits.MaxRecords {
			return nil, fmt.Errorf("cachebench: observations exceed record limit %d", limits.MaxRecords)
		}
		if !validUniqueJSONObject(raw) {
			return nil, fmt.Errorf("cachebench: observation line %d: duplicate or invalid JSON", line)
		}
		decoder := json.NewDecoder(bytes.NewReader(raw))
		decoder.DisallowUnknownFields()
		var record ObservationRecord
		if err := decoder.Decode(&record); err != nil {
			return nil, fmt.Errorf("cachebench: observation line %d: %w", line, err)
		}
		var trailing any
		if err := decoder.Decode(&trailing); err != io.EOF {
			return nil, fmt.Errorf("cachebench: observation line %d: trailing JSON", line)
		}
		if record.Schema != ObservationSchema {
			return nil, fmt.Errorf("cachebench: observation line %d: schema %q", line, record.Schema)
		}
		if strings.TrimSpace(record.RequestID) == "" || seen[record.RequestID] {
			return nil, fmt.Errorf("cachebench: observation line %d: empty or duplicate request_id", line)
		}
		seen[record.RequestID] = true
		record.Usage = append(json.RawMessage(nil), record.Usage...)
		records = append(records, record)
	}
	if err := scanner.Err(); err != nil {
		return nil, fmt.Errorf("cachebench: read observations: %w", err)
	}

View on GitHub (pinned to 27d5a3981a)

Solutions

1. Read the wrapped json error — it names the unknown field or type mismatch.
2. Strip fields not in the current ObservationRecord schema from the file.
3. Regenerate observations with the same tool version that will evaluate them.

Example fix

// before
{"schema":"...","request_id":"r1","extra_field":true}
// after
{"schema":"...","request_id":"r1"}

func decodeObservation(line []byte) (ObservationRecord, error) {
	dec := json.NewDecoder(bytes.NewReader(line))
	dec.DisallowUnknownFields()
	var rec ObservationRecord
	if err := dec.Decode(&rec); err != nil {
		return ObservationRecord{}, err
	}
	return rec, nil
}
// run over sample lines to catch schema drift before a full load

if err := loadObservations(path, limits); err != nil {
	var se *json.UnmarshalTypeError
	if errors.As(err, &se) {
		// field type drift: regenerate observations with current schema
	}
	if strings.Contains(err.Error(), "unknown field") {
		// strip the named field or regenerate the file
	}
}

When it happens

Trigger: Decoding an observation line that includes a field absent from ObservationRecord (schema drift), or a field with the wrong JSON type (e.g. request_id as a number).

Common situations: Observations written by a newer/older tool version with extra fields; hand-added metadata keys in observation JSONL; type changes across schema versions.

Related errors

• corpus provider conversion supports string or null content
• cachebench: invalid observation read limits
• cachebench: no observation records
• cachebench: provider population exceeds 1024
• cachebench: verifier returned duplicate or invalid JSON

AI-assisted analysis of JuliusBrussee/caveman@27d5a3981a (2026-08-15). Data as JSON: /api/errors/c4351aaabb20ed65. Report an issue: GitHub.