JuliusBrussee/caveman · error

baseline model call: %w

Error message

baseline model call: %w

What it means

Error "baseline model call: %w" thrown in JuliusBrussee/caveman.

Source

Thrown at engine/evals/quality.go:162

		return report, nil
	}
	expected := expectedAnswer(graders)

	baselinePrompt := qualityPrompt(f.QualityTask.Question, input)
	compressedPrompt := qualityPrompt(f.QualityTask.Question, compressed)
	baseline, err := opts.Runner.Complete(ctx, ModelRequest{
		FixtureName:    f.Name,
		Variant:        "baseline",
		Model:          opts.Model,
		ContentType:    contentType,
		Question:       f.QualityTask.Question,
		ExpectedAnswer: expected,
		Seed:           opts.Seed,
		Temperature:    opts.Temperature,
		Prompt:         baselinePrompt,
	})
	if err != nil {
		return QualityReport{}, fmt.Errorf("baseline model call: %w", err)
	}
	compressedResp, err := opts.Runner.Complete(ctx, ModelRequest{
		FixtureName:    f.Name,
		Variant:        "compressed",
		Model:          opts.Model,
		ContentType:    contentType,
		Question:       f.QualityTask.Question,
		ExpectedAnswer: expected,
		Seed:           opts.Seed,
		Temperature:    opts.Temperature,
		Prompt:         compressedPrompt,
	})
	if err != nil {
		return QualityReport{}, fmt.Errorf("compressed model call: %w", err)
	}

	counter := tokens.Default()
	report.BaselineUsage, err = fillUsage(counter, baselinePrompt, baseline, opts.RequireUsage)

View on GitHub (pinned to 27d5a3981a)

Solutions

1. Fix the baseline model call (credentials/endpoint) and retry.

When it happens

Trigger: Thrown at engine/evals/quality.go:162 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of JuliusBrussee/caveman@27d5a3981a (2026-08-15). Data as JSON: /api/errors/f2ea97a2c03048fe. Report an issue: GitHub.