{"record":{"id":"8276c5d878a387f2","repo":"JuliusBrussee/caveman","slug":"cachebench-provider-q-population-d-cannot-meet","errorCode":null,"errorMessage":"cachebench: provider %q population %d cannot meet minimum eligible requests %d","messagePattern":"cachebench: provider %q population (.+?) cannot meet minimum eligible requests (.+?)","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"cacheengine/cachebench/replay.go","lineNumber":457,"sourceCode":"// sample gate even if every captured request proves cache eligible.\nfunc ValidateReplayTarget(records []TraceRecord, target Target) error {\n\tif err := validateTarget(target); err != nil {\n\t\treturn err\n\t}\n\tif len(records) == 0 {\n\t\treturn errors.New(\"cachebench: replay trace is empty\")\n\t}\n\tcounts := map[string]int{}\n\tfor _, record := range records {\n\t\tprovider := strings.ToLower(strings.TrimSpace(record.Provider))\n\t\tif provider == \"\" {\n\t\t\treturn fmt.Errorf(\"cachebench: replay request %q has empty provider\", record.RequestID)\n\t\t}\n\t\tcounts[provider]++\n\t}\n\tfor provider, count := range counts {\n\t\tif count < target.MinEligibleRequest {\n\t\t\treturn fmt.Errorf(\"cachebench: provider %q population %d cannot meet minimum eligible requests %d\", provider, count, target.MinEligibleRequest)\n\t\t}\n\t}\n\treturn nil\n}\n\n// Run validates full population, prepares equivalent bodies, then dispatches replay.\nfunc (runner ReplayRunner) Run(ctx context.Context, records []TraceRecord, emit func(ReplayResult) error) error {\n\tif runner.Engine == nil || runner.Transport == nil || runner.Verifier == nil || emit == nil {\n\t\treturn errors.New(\"cachebench: replay engine, transport, verifier, and emitter required\")\n\t}\n\tif _, err := ValidateReplay(records, runner.Limits, runner.TimeScale); err != nil {\n\t\treturn err\n\t}\n\tprepared, err := runner.prepare(ctx, records)\n\tif err != nil {\n\t\treturn err\n\t}\n\tif runner.Target != nil {","sourceCodeStart":439,"sourceCodeEnd":475,"githubUrl":"https://github.com/JuliusBrussee/caveman/blob/27d5a3981a347890211bb1bf2439e5c821a63bc9/cacheengine/cachebench/replay.go#L439-L475","documentation":"During target validation, at least one provider's record count is below target.MinEligibleRequest. The benchmark requires each provider to have enough requests for a statistically meaningful cache-eligibility sample, so it refuses to run an underpopulated replay.","triggerScenarios":"ValidateReplayTarget with, e.g., 3 records for \"anthropic\" while Target.MinEligibleRequest is 5. Triggered by small trace files, traces split per provider, or raising MinEligibleRequest above an existing population.","commonSituations":"Testing the harness with a tiny smoke trace; filtering a trace down to one model/provider and forgetting the minimum applies per provider; bumping MinEligibleRequest for a production run and replaying an old small trace.","solutions":["Capture or include more trace records for the provider named in the message until count >= MinEligibleRequest.","Lower Target.MinEligibleRequest to a value every provider population can meet (fine for smoke tests).","Remove providers you do not intend to benchmark from the trace so their small populations no longer count.","Compute per-provider counts up front (group records by lowercased trimmed Provider) and reconcile against the target before calling Run."],"exampleFix":"// before\ntarget := Target{MinEligibleRequest: 10} // trace has 4 anthropic records\n\n// after\ntarget := Target{MinEligibleRequest: 4} // or add records until anthropic >= 10","handlingStrategy":"validation","validationCode":"counts := map[string]int{}\nfor _, r := range records {\n    counts[strings.ToLower(strings.TrimSpace(r.Provider))]++\n}\nfor p, n := range counts {\n    if n < target.MinEligibleRequest {\n        return fmt.Errorf(\"provider %s underpopulated: %d < %d\", p, n, target.MinEligibleRequest)\n    }\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Compute per-provider counts and reconcile with MinEligibleRequest before calling Run.","Keep smoke-test traces paired with a low-minimum Target in test fixtures.","When splitting traces per provider, remember the minimum applies to each provider independently."],"tags":["cachebench","validation","sample-size","replay"],"backgroundTag":null,"analyzedSha":"27d5a3981a347890211bb1bf2439e5c821a63bc9","analyzedAt":"2026-08-15T09:26:11.751Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}