JuliusBrussee/caveman · error

cachebench: replay request %q has empty provider

Error message

cachebench: replay request %q has empty provider

What it means

ValidateReplayTarget rejected a trace record whose Provider field, after strings.TrimSpace and ToLower, is empty. The replay harness groups requests by provider to enforce per-provider minimum eligible request counts, so a providerless record cannot be attributed to any population. The error is thrown before any request is sent.

Source

Thrown at cacheengine/cachebench/replay.go:451

		InputBudgetClaimedProviderCounted: len(tokenBases) == 1 && tokenBases[0] == TokenProviderCounted,
		MaxConcurrency:                    limits.MaxConcurrency,
	}, nil
}

// ValidateReplayTarget rejects a live population that cannot meet its minimum
// sample gate even if every captured request proves cache eligible.
func ValidateReplayTarget(records []TraceRecord, target Target) error {
	if err := validateTarget(target); err != nil {
		return err
	}
	if len(records) == 0 {
		return errors.New("cachebench: replay trace is empty")
	}
	counts := map[string]int{}
	for _, record := range records {
		provider := strings.ToLower(strings.TrimSpace(record.Provider))
		if provider == "" {
			return fmt.Errorf("cachebench: replay request %q has empty provider", record.RequestID)
		}
		counts[provider]++
	}
	for provider, count := range counts {
		if count < target.MinEligibleRequest {
			return fmt.Errorf("cachebench: provider %q population %d cannot meet minimum eligible requests %d", provider, count, target.MinEligibleRequest)
		}
	}
	return nil
}

// Run validates full population, prepares equivalent bodies, then dispatches replay.
func (runner ReplayRunner) Run(ctx context.Context, records []TraceRecord, emit func(ReplayResult) error) error {
	if runner.Engine == nil || runner.Transport == nil || runner.Verifier == nil || emit == nil {
		return errors.New("cachebench: replay engine, transport, verifier, and emitter required")
	}
	if _, err := ValidateReplay(records, runner.Limits, runner.TimeScale); err != nil {
		return err

View on GitHub (pinned to 27d5a3981a)

Solutions

1. Inspect the RequestID named in the message and fix that record's Provider field (e.g. "openai", "anthropic") in the trace source.
2. If generating traces programmatically, assert record.Provider != "" at capture time so bad rows fail at capture, not replay.
3. Sanitize the trace before validation: filter or reject records with strings.TrimSpace(record.Provider) == "".
4. If the provider genuinely cannot be determined, drop the record and re-check that remaining populations still meet MinEligibleRequest (error 881 will otherwise fire).

Example fix

// before
records = append(records, TraceRecord{RequestID: "req-42", At: at, Body: body}) // Provider never set

// after
records = append(records, TraceRecord{RequestID: "req-42", Provider: "openai", At: at, Body: body})

func hasProvider(r cachebench.TraceRecord) bool {
    return strings.ToLower(strings.TrimSpace(r.Provider)) != ""
}

for _, r := range records {
    if !hasProvider(r) {
        return fmt.Errorf("record %s lacks provider", r.RequestID)
    }
}

When it happens

Trigger: Calling ValidateReplayTarget (or Run, which validates the population) with a TraceRecord whose Provider is "", " ", or only whitespace. Records produced by a trace capture step that forgot to copy the provider field, or hand-written JSON traces missing the provider key, trigger it on the first offending record.

Common situations: Importing traces from another tool that names the field differently (vendor vs provider); a JSON trace edited by hand with a typo'd key; a capture bug where the provider column is dropped for one row of many.

Related errors

• cachebench: provider %q population %d cannot meet minimum el
• cachebench: no replay evidence
• message %d: %w
• cachebench: session %q request %d: %w
• cachebench: trace request %d has invalid schema or request_i

AI-assisted analysis of JuliusBrussee/caveman@27d5a3981a (2026-08-15). Data as JSON: /api/errors/d0bdbca9246e3038. Report an issue: GitHub.