{"record":{"id":"1f5c349a55ecd0dd","repo":"micro/go-micro","slug":"flow-llmgrader-returned-an-empty-grade","errorCode":null,"errorMessage":"flow: LLMGrader returned an empty grade","messagePattern":"flow: LLMGrader returned an empty grade","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"flow/verify.go","lineNumber":139,"sourceCode":"\t\t\treturn false, \"\", fmt.Errorf(\"flow: LLMGrader requires a flow model (set Provider/APIKey)\")\n\t\t}\n\t\tprompt := fmt.Sprintf(\"Grade the latest result against this rubric:\\n%s\\n\\nLatest result:\\n%s\\n\\nAnswer with PASS or FAIL on the first line, followed by one short feedback sentence.\", rubric, out.String())\n\t\tresp, err := d.model.Generate(ctx, &ai.Request{Prompt: prompt})\n\t\tif err != nil {\n\t\t\treturn false, \"\", err\n\t\t}\n\t\treply := resp.Answer\n\t\tif reply == \"\" {\n\t\t\treply = resp.Reply\n\t\t}\n\t\treturn parseGrade(reply)\n\t}\n}\n\nfunc parseGrade(reply string) (bool, string, error) {\n\ttext := strings.TrimSpace(reply)\n\tif text == \"\" {\n\t\treturn false, \"\", fmt.Errorf(\"flow: LLMGrader returned an empty grade\")\n\t}\n\tlines := strings.SplitN(text, \"\\n\", 2)\n\tfirst := strings.ToLower(strings.TrimSpace(lines[0]))\n\tfeedback := \"\"\n\tif len(lines) > 1 {\n\t\tfeedback = strings.TrimSpace(lines[1])\n\t}\n\tpass := strings.HasPrefix(first, \"pass\") || isAffirmative(first)\n\tif !pass && feedback == \"\" {\n\t\tfeedback = text\n\t}\n\treturn pass, feedback, nil\n}\n\nfunc stateWithField(s State, field, value string) (State, error) {\n\tvar obj map[string]any\n\tif len(s.Data) > 0 && json.Unmarshal(s.Data, &obj) == nil && obj != nil {\n\t\tobj[field] = value","sourceCodeStart":121,"sourceCodeEnd":157,"githubUrl":"https://github.com/micro/go-micro/blob/24529f140421a11a33b6999ab7944f2021cfd69c/flow/verify.go#L121-L157","documentation":"flow.parseGrade parses the LLM's reply inside LLMGrader and requires a non-empty first line containing PASS or FAIL. This error is returned when the model's reply, after trimming whitespace, is an empty string — meaning the grader received no usable grade text. It surfaces (wrapped) as a 'verify grade attempt N' error to the caller.","triggerScenarios":"The LLM API returns a 200 with an empty response body / empty content (e.g. content filter, stop-reason quirks, empty completion), and LLMGrader passes that reply to parseGrade.","commonSituations":"Provider returning empty content due to safety filtering; max_tokens set to 0 or extremely small; misconfigured model endpoint returning empty payloads; heavily truncated responses.","solutions":["Retry the grading call — an empty reply is often transient; add a retry wrapper around the grader or configure Verify's attempt loop to tolerate it.","Check the model's max_tokens/stop settings and any content-filter configuration that could yield empty output.","Inspect the raw API response (logging) to confirm whether the provider returned an empty choices/content array.","Use a custom grader that validates the reply and retries once on empty before failing."],"exampleFix":"// before\nresp, err := d.model.Generate(ctx, &ai.Request{Prompt: prompt})\n// after\nresp, err := d.model.Generate(ctx, &ai.Request{Prompt: prompt, MaxTokens: 256})\nif err == nil && strings.TrimSpace(resp.Text()) == \"\" {\n    resp, err = d.model.Generate(ctx, &ai.Request{Prompt: prompt, MaxTokens: 256}) // retry once\n}","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"out, err := step(ctx, in)\nif err != nil {\n    if strings.Contains(err.Error(), \"returned an empty grade\") {\n        time.Sleep(2 * time.Second)\n        out, err = step(ctx, in) // retry; empty replies are often transient\n    }\n    return out, err\n}","preventionTips":["Set a reasonable max_tokens so completions are never truncated to empty.","Log raw model responses to detect empty-choice conditions early.","Wrap the grader with one automatic retry on empty output before surfacing the error.","Check provider content-filter/stop-reason settings."],"tags":["go","flow","llm","grader","empty-response"],"backgroundTag":"llm-empty-response","analyzedSha":"24529f140421a11a33b6999ab7944f2021cfd69c","analyzedAt":"2026-09-01T02:52:24.923Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T05:18:18.240Z"}