JuliusBrussee/caveman · error

compressed model call: %w

Error message

compressed model call: %w

What it means

Error "compressed model call: %w" thrown in JuliusBrussee/caveman.

Source

Thrown at engine/evals/quality.go:176

		Temperature:    opts.Temperature,
		Prompt:         baselinePrompt,
	})
	if err != nil {
		return QualityReport{}, fmt.Errorf("baseline model call: %w", err)
	}
	compressedResp, err := opts.Runner.Complete(ctx, ModelRequest{
		FixtureName:    f.Name,
		Variant:        "compressed",
		Model:          opts.Model,
		ContentType:    contentType,
		Question:       f.QualityTask.Question,
		ExpectedAnswer: expected,
		Seed:           opts.Seed,
		Temperature:    opts.Temperature,
		Prompt:         compressedPrompt,
	})
	if err != nil {
		return QualityReport{}, fmt.Errorf("compressed model call: %w", err)
	}

	counter := tokens.Default()
	report.BaselineUsage, err = fillUsage(counter, baselinePrompt, baseline, opts.RequireUsage)
	if err != nil {
		return QualityReport{}, fmt.Errorf("baseline usage: %w", err)
	}
	report.CompressedUsage, err = fillUsage(counter, compressedPrompt, compressedResp, opts.RequireUsage)
	if err != nil {
		return QualityReport{}, fmt.Errorf("compressed usage: %w", err)
	}

	baseSubject := Subject{Input: input, Output: baseline.Output, ContentType: contentType, PassedThrough: true}
	compSubject := Subject{Input: input, Output: compressedResp.Output, Ratio: compressionRatio, ContentType: contentType, PassedThrough: passedThrough}
	grader := opts.Grader
	if grader == nil {
		grader = LocalGrader{}
	}

View on GitHub (pinned to 27d5a3981a)

Solutions

1. Fix the compressed model call (credentials/endpoint) and retry.

When it happens

Trigger: Thrown at engine/evals/quality.go:176 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of JuliusBrussee/caveman@27d5a3981a (2026-08-15). Data as JSON: /api/errors/0ab0d871c996d62a. Report an issue: GitHub.