JuliusBrussee/caveman · error

compressed usage: %w

Error message

compressed usage: %w

What it means

Error "compressed usage: %w" thrown in JuliusBrussee/caveman.

Source

Thrown at engine/evals/quality.go:186

		ContentType:    contentType,
		Question:       f.QualityTask.Question,
		ExpectedAnswer: expected,
		Seed:           opts.Seed,
		Temperature:    opts.Temperature,
		Prompt:         compressedPrompt,
	})
	if err != nil {
		return QualityReport{}, fmt.Errorf("compressed model call: %w", err)
	}

	counter := tokens.Default()
	report.BaselineUsage, err = fillUsage(counter, baselinePrompt, baseline, opts.RequireUsage)
	if err != nil {
		return QualityReport{}, fmt.Errorf("baseline usage: %w", err)
	}
	report.CompressedUsage, err = fillUsage(counter, compressedPrompt, compressedResp, opts.RequireUsage)
	if err != nil {
		return QualityReport{}, fmt.Errorf("compressed usage: %w", err)
	}

	baseSubject := Subject{Input: input, Output: baseline.Output, ContentType: contentType, PassedThrough: true}
	compSubject := Subject{Input: input, Output: compressedResp.Output, Ratio: compressionRatio, ContentType: contentType, PassedThrough: passedThrough}
	grader := opts.Grader
	if grader == nil {
		grader = LocalGrader{}
	}
	report.BaselineFailures, err = gradeFailures(ctx, grader, graders, baseSubject)
	if err != nil {
		return QualityReport{}, fmt.Errorf("baseline grading: %w", err)
	}
	report.CompressedFailures, err = gradeFailures(ctx, grader, graders, compSubject)
	if err != nil {
		return QualityReport{}, fmt.Errorf("compressed grading: %w", err)
	}
	report.BaselinePassed = len(report.BaselineFailures) == 0
	report.CompressedPassed = len(report.CompressedFailures) == 0

View on GitHub (pinned to 27d5a3981a)

Solutions

1. Fix compressed usage extraction from the model response.

When it happens

Trigger: Thrown at engine/evals/quality.go:186 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of JuliusBrussee/caveman@27d5a3981a (2026-08-15). Data as JSON: /api/errors/e739ea15b8d764d0. Report an issue: GitHub.