JuliusBrussee/caveman · error

cachebench: session %q request %d: %w

Error message

cachebench: session %q request %d: %w

What it means

While replaying a corpus as a trace, building the provider-specific request body for a row failed; the error records the session ID and the 1-based request number within that session, wrapping the body-construction error from corpusProviderBody.

Source

Thrown at cacheengine/cachebench/corpus.go:644

	}
	return prefix, nil
}

func buildCorpusTrace(provider ProviderConfig, corpus AgentCorpus, counter *corpusTokenCounter) (Trace, error) {
	started := time.Date(2026, 8, 10, 0, 0, 0, 0, time.UTC)
	requests := make([]TraceRequest, 0, len(corpus.Rows))
	requestNumber := map[string]int{}
	elapsed := map[string]time.Duration{}
	epochVersion := map[string]int{}
	stableIdentity := map[string]string{}
	for _, row := range corpus.Rows {
		requestNumber[row.SessionID]++
		if requestNumber[row.SessionID] > 1 {
			elapsed[row.SessionID] += time.Duration(row.PreGap * float64(time.Second))
		}
		body, err := corpusProviderBody(provider, row.Input)
		if err != nil {
			return Trace{}, fmt.Errorf("cachebench: session %q request %d: %w", row.SessionID, requestNumber[row.SessionID], err)
		}
		prefix, err := corpusPrefix(row.Input, counter)
		if err != nil {
			return Trace{}, err
		}
		stableCount := 0
		for _, message := range row.Input {
			if message.Role != "system" && message.Role != "developer" {
				break
			}
			stableCount++
		}
		identity, identityErr := corpusStableIdentity(row.Input, stableCount)
		if identityErr != nil {
			return Trace{}, identityErr
		}
		if epochVersion[row.SessionID] == 0 {
			epochVersion[row.SessionID] = 1

View on GitHub (pinned to 27d5a3981a)

Solutions

1. Check the wrapped error: if it is 'unsupported corpus provider', fix ProviderConfig.Provider.
2. Ensure message content is a plain JSON string (or null) for providers whose body builders call contentText.
3. Re-emit the corpus with string content before replaying to anthropic/bedrock/gemini.

switch provider.Provider {
case "openai", "anthropic", "bedrock", "gemini":
default:
	return fmt.Errorf("unsupported corpus provider %q", provider.Provider)
}
// plus verify all messages have string-or-null content for non-openai providers:
for _, row := range corpus.Rows {
	for _, m := range row.Input {
		if _, err := contentText(m.Content); err != nil {
			return fmt.Errorf("row content not convertible: %w", err)
		}
	}
}

trace, err := BuildTrace(corpus, provider)
if err != nil {
	if strings.Contains(err.Error(), "unsupported corpus provider") {
		// fix provider config and re-run
	}
	return err
}

When it happens

Trigger: Calling the trace-building path over a corpus when corpusProviderBody(provider, row.Input) returns an error — most commonly the unsupported-provider error, or content conversion failures for the selected provider (e.g. non-string content that provider's body builder cannot encode).

Common situations: Provider config with a misspelled Provider value; an anthropic/bedrock/gemini replay of messages whose content is array-form, since contentText only accepts string or null.

Related errors

• unsupported corpus provider %q
• cachebench: observation request %q provider/epoch mismatch
• cachebench: observation request %q body digest mismatch
• cachebench: verifier returned duplicate or invalid JSON
• cachebench: verifier returned invalid JSON

AI-assisted analysis of JuliusBrussee/caveman@27d5a3981a (2026-08-15). Data as JSON: /api/errors/901a7c3f4c3675f0. Report an issue: GitHub.