run-llama/llama_index · warning · ValueError

Invalid answer line: {answer_line}. Answer line must be of t

Error message

Invalid answer line: {answer_line}. Answer line must be of the form: answer_num: <int>, answer_relevance: <float>

What it means

default_parse_choice_select_answer (llama_index.core.indices.utils) raises ValueError('Invalid answer line: ...') when raise_error=True and a line of the LLM's answer does not contain exactly one comma — i.e. it is not of the form 'answer_num: <int>, answer_relevance: <float>'. With the default raise_error=False the line is silently skipped. The function parses the choice-select output used by rerankers/RouterQueryEngine, where each selected chunk must be listed with its relevance score.

Source

Thrown at llama-index-core/llama_index/core/indices/utils.py:133

        )

    return content_messages


def default_parse_choice_select_answer_fn(
    answer: str, num_choices: int, raise_error: bool = False
) -> Tuple[List[int], List[float]]:
    """Default parse choice select answer function."""
    answer_lines = answer.split("\n")
    answer_nums = []
    answer_relevances = []
    for answer_line in answer_lines:
        line_tokens = answer_line.split(",")
        if len(line_tokens) != 2:
            if not raise_error:
                continue
            else:
                raise ValueError(
                    f"Invalid answer line: {answer_line}. "
                    "Answer line must be of the form: "
                    "answer_num: <int>, answer_relevance: <float>"
                )
        try:
            answer_num = int(line_tokens[0].split(":")[1].strip())
        except (IndexError, ValueError) as e:
            if not raise_error:
                continue
            else:
                raise ValueError(
                    f"Invalid answer line: {answer_line}. "
                    "Answer line must be of the form: "
                    "answer_num: <int>, answer_relevance: <float>"
                )
        if answer_num > num_choices:
            continue
        answer_nums.append(answer_num)

View on GitHub (pinned to afd0fef371)

Solutions

1. Keep raise_error=False (the default) so malformed lines are skipped instead of raising.
2. Improve format adherence: keep the stock choice-select prompt, lower temperature, or use a stronger/structured-output LLM.
3. If you control the calling code, strip preamble/trailing lines and blank lines before parsing.

Example fix

# before
nums, rels = default_parse_choice_select_answer(raw_llm_output, n, raise_error=True)  # raises

# after
nums, rels = default_parse_choice_select_answer(raw_llm_output, n, raise_error=False)
if not nums:
    # LLM output unusable -> fall back to unranked order
    nums, rels = list(range(n)), [1.0] * n

import re

CHOICE_LINE_RE = re.compile(r"^\s*answer_num:\s*\d+\s*,\s*answer_relevance:\s*[\d.]+\s*$")

def is_well_formed_choice_answer(answer: str) -> bool:
    return all(CHOICE_LINE_RE.match(l) for l in answer.splitlines() if l.strip())

try:
    nums, rels = default_parse_choice_select_answer(answer, n, raise_error=True)
except ValueError:
    nums, rels = default_parse_choice_select_answer(answer, n, raise_error=False)
if not nums:  # fully unparseable -> keep original ordering
    nums, rels = list(range(n)), [1.0] * n

When it happens

Trigger: Calling parse_choice_select_answer(llm_output, n, raise_error=True) where the LLM emitted a preamble line ('Here are the relevant chunks:'), a single-field line ('answer_num: 3'), or extra commas; wrap around ChoiceSelectPrompt output from a weak model.

Common situations: LLMs that add chatty framing despite the output format instruction; few-shot prompt customization that changed the format; small local models ignoring the template; switching raise_error=True on to 'catch format bugs' and hitting real ones.

Related errors

• Failed to validate query spec. Error: {e}. Got JSON dict: {j
• First argument to Readability constructor should be a docume
• Aborting parsing document; {numTags} elements found
• Command failed: {command} {result.stderr}
• Git command failed: {result.stderr}

AI-assisted analysis of run-llama/llama_index@afd0fef371 (2026-08-15). Data as JSON: /api/errors/8ad5486c7a6b2f50. Report an issue: GitHub.