What causes "cachebench: no providers" in JuliusBrussee/caveman?

Supply at least one ProviderConfig (name + model) to benchmark against

cachebench: no providers

Error message

cachebench: no providers

What it means

RunCorpus evaluates a corpus across provider compilers, so at least one ProviderConfig is required. An empty providers slice has no meaningful benchmark result and the function rejects it up front.

Source

Thrown at cacheengine/cachebench/corpus.go:448

	for _, row := range rows {
		_ = encoder.Encode(row)
	}
	return hex.EncodeToString(hash.Sum(nil))
}

// RunCorpus evaluates one validated public corpus across provider compilers.
func RunCorpus(ctx context.Context, engine *cacheengine.Engine, corpus AgentCorpus, providers []ProviderConfig, target Target) (Report, error) {
	if engine == nil {
		return Report{}, errors.New("cachebench: nil cache engine")
	}
	if err := validateTarget(target); err != nil {
		return Report{}, err
	}
	if len(corpus.Rows) == 0 || corpus.SHA256 == "" || strings.TrimSpace(corpus.Metadata.Name) == "" {
		return Report{}, errors.New("cachebench: invalid corpus")
	}
	if len(providers) == 0 {
		return Report{}, errors.New("cachebench: no providers")
	}
	if len(providers) > 1024 {
		return Report{}, errors.New("cachebench: provider population exceeds 1024")
	}
	counter := tokens.Default()
	cachedCounter := &corpusTokenCounter{Counter: counter, counts: map[[sha256.Size]byte]int{}}
	summary, err := analyzeCorpus(corpus, cachedCounter)
	if err != nil {
		return Report{}, err
	}
	scenario := Scenario{Name: "public-agent-corpus", Turns: len(corpus.Rows), Step: time.Second, AssumedTTL: 5 * time.Minute}
	report := baseReport(BasisCorpusSimulated, scenario, target, QualityEquivalence)
	report.Corpus = &summary
	report.Scenario.TokenBasis = counter.Name() + " estimate over normalized OpenAI messages"
	report.Scenario.Step = "corpus pre_gap"
	report.Scenario.AssumedTTL = "provider profile TTL"
	for _, provider := range providers {
		trace, buildErr := buildCorpusTrace(provider, corpus, cachedCounter)

View on GitHub (pinned to 27d5a3981a)

Solutions

Supply at least one ProviderConfig (name + model) to benchmark against
Check earlier filtering/validation code that may have removed every provider and log why each was dropped
Fail fast at config load when the providers list is empty, before entering the benchmark path

Example fix

// before
report, err := cachebench.RunCorpus(ctx, engine, corpus, nil, target)

// after
if len(providers) == 0 {
    return fmt.Errorf("no providers configured for corpus benchmark")
}
report, err := cachebench.RunCorpus(ctx, engine, corpus, providers, target)

Defensive patterns

Strategy: validation

Validate before calling

if len(providers) == 0 {
    return fmt.Errorf("benchmark config error: no providers; add at least one entry to the providers list")
}
report, err := cachebench.RunCorpus(ctx, engine, corpus, providers, target)

Type guard

func hasProviders(ps []ProviderConfig) bool { return len(ps) > 0 }

Try / catch

if _, err := cachebench.RunCorpus(ctx, engine, corpus, providers, target); err != nil {
    if err.Error() == "cachebench: no providers" {
        // load/fix provider config and retry once
    }
}

Prevention

Validate the providers section at config load and fail with the config path in the message
Log which providers were dropped by filtering so an empty list is explainable
Default to at least one well-known provider in test configs

When it happens

Trigger: Calling RunCorpus with a zero-length providers slice — often because provider configs were filtered out (unsupported names, empty API config) before the call, or the config list came from an empty env/section.

Common situations: Config file's providers section missing or commented out; providers filtered by a validation pass that rejected all entries; CI running the benchmark with a stripped-down config.

Related errors

AI-assisted analysis of JuliusBrussee/caveman@27d5a3981a (2026-08-15). Data as JSON: /api/errors/edaee5467b48a9be. Report an issue: GitHub.