{"record":{"id":"4ad03b3939790d23","repo":"JuliusBrussee/caveman","slug":"quality-fixture-q-w","errorCode":null,"errorMessage":"quality fixture %q: %w","messagePattern":"quality fixture %q: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"engine/evals/harness.go","lineNumber":276,"sourceCode":"\t\tmergeProbeSummary(&report.Probes, probeSummary)\n\t\t// A fixture with no graders is a configuration error, not a free pass.\n\t\tif len(f.Graders) == 0 {\n\t\t\tfr.Passed = false\n\t\t\tfr.Failures = append(fr.Failures, \"no graders configured\")\n\t\t}\n\t\tfor _, g := range f.Graders {\n\t\t\tif v := Grade(g, subject); !v.Passed {\n\t\t\t\tfr.Passed = false\n\t\t\t\tfr.Failures = append(fr.Failures, v.Reason)\n\t\t\t}\n\t\t}\n\t\tif !fr.Passed {\n\t\t\treport.Passed = false\n\t\t}\n\t\tif quality != nil {\n\t\t\tqr, err := runQuality(ctx, *quality, f, input, res.Output, res.ContentType, res.Ratio, res.PassedThrough)\n\t\t\tif err != nil {\n\t\t\t\treturn Report{}, fmt.Errorf(\"quality fixture %q: %w\", f.Name, err)\n\t\t\t}\n\t\t\tfr.Quality = &qr\n\t\t}\n\t\treport.TokensBefore += fr.TokensBefore\n\t\treport.TokensAfter += fr.TokensAfter\n\t\treport.WordsBefore += fr.WordsBefore\n\t\treport.WordsAfter += fr.WordsAfter\n\t\treport.Fixtures = append(report.Fixtures, fr)\n\t}\n\treport.TokenRatio = ratio(report.TokensBefore, report.TokensAfter)\n\treport.WordRatio = ratio(report.WordsBefore, report.WordsAfter)\n\tif quality != nil {\n\t\treport.Quality = summarizeQuality(report.Fixtures, *quality)\n\t\tif !report.Quality.Passed {\n\t\t\treport.Passed = false\n\t\t}\n\t}\n\treturn report, nil","sourceCodeStart":258,"sourceCodeEnd":294,"githubUrl":"https://github.com/JuliusBrussee/caveman/blob/27d5a3981a347890211bb1bf2439e5c821a63bc9/engine/evals/harness.go#L258-L294","documentation":"Error \"quality fixture %q: %w\" thrown in JuliusBrussee/caveman.","triggerScenarios":"Thrown at engine/evals/harness.go:276 when the library encounters an invalid state.","commonSituations":"See trigger scenarios.","solutions":["Fix the quality fixture run error and retry."],"exampleFix":null,"handlingStrategy":null,"validationCode":null,"typeGuard":null,"tryCatchPattern":null,"preventionTips":[],"tags":[],"backgroundTag":null,"analyzedSha":"27d5a3981a347890211bb1bf2439e5c821a63bc9","analyzedAt":"2026-08-15T09:26:11.751Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}