JuliusBrussee/caveman · error

cachebench: invalid scenario token or turn count

Error message

cachebench: invalid scenario token or turn count

What it means

Error "cachebench: invalid scenario token or turn count" thrown in JuliusBrussee/caveman.

Source

Thrown at cacheengine/cachebench/trace.go:91

				ID: fmt.Sprintf("turn-%03d-tool-result", turn+1), Kind: "tool_result", Turn: turn + 1,
				Text:   fixtureText(fmt.Sprintf("workspace result turn %d", turn+1), scenario.ToolResultTokens),
				Tokens: scenario.ToolResultTokens,
			},
		)
	}
	return Trace{
		Provider: provider, Scenario: scenario, Requests: requests,
		TokenBasis:  "deterministic declared fixture tokens; provider counts required for observed evidence",
		TimingBasis: TimingSynthetic,
	}, nil
}

func validateScenario(provider ProviderConfig, scenario Scenario) error {
	if strings.TrimSpace(provider.Provider) == "" || strings.TrimSpace(provider.Model) == "" || strings.TrimSpace(provider.Endpoint) == "" {
		return errors.New("cachebench: provider, model, and endpoint required")
	}
	if scenario.Turns < 2 || scenario.StaticTokens <= 0 || scenario.UserTokens <= 0 || scenario.AssistantTokens < 0 || scenario.ToolResultTokens < 0 || scenario.SummaryTokens < 0 {
		return errors.New("cachebench: invalid scenario token or turn count")
	}
	if scenario.CompactionEvery < 0 || scenario.Step <= 0 || scenario.AssumedTTL <= 0 {
		return errors.New("cachebench: invalid scenario timing")
	}
	return nil
}

func nativeRequest(provider ProviderConfig, scenario Scenario, epoch string, body []byte) cacheengine.NativeRequest {
	expectedCalls := scenario.Turns
	if scenario.CompactionEvery > 0 && expectedCalls > scenario.CompactionEvery {
		expectedCalls = scenario.CompactionEvery
	}
	return cacheengine.NativeRequest{
		Scope: "cachebench/real-agent", Epoch: epoch, PartitionKey: "agent-session-1",
		ExpectedRequestsPerMinute: 20, ExpectedCalls: expectedCalls,
		Provider: provider.Provider, Model: provider.Model, Region: provider.Region, Endpoint: provider.Endpoint,
		Body: body, RuntimeMode: "optimize", AuthMode: "payg", PrefixTokens: scenario.StaticTokens,
	}

View on GitHub (pinned to 27d5a3981a)

Solutions

1. Fix the scenario token and turn counts to valid values.

When it happens

Trigger: Thrown at cacheengine/cachebench/trace.go:91 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of JuliusBrussee/caveman@27d5a3981a (2026-08-15). Data as JSON: /api/errors/4261c0da502eeeb1. Report an issue: GitHub.