run-llama/llama_index · error · ValueError

Structured prediction failed for nodes {idx} - {idx + self.c

Error message

Structured prediction failed for nodes {idx} - {idx + self.choice_batch_size}: {result}

What it means

Error "Structured prediction failed for nodes {idx} - {idx + self.choice_batch_size}: {result}" thrown in run-llama/llama_index.

Source

Thrown at llama-index-core/llama_index/core/postprocessor/structured_llm_rerank.py:189

        ) as event:
            for idx in range(0, len(nodes), self.choice_batch_size):
                nodes_batch = [
                    node.node for node in nodes[idx : idx + self.choice_batch_size]
                ]

                query_str = query_bundle.query_str
                fmt_batch_str = self._format_node_batch_fn(nodes_batch)
                # call each batch independently
                result: Union[BaseModel, str] = self.llm.structured_predict(
                    output_cls=self._document_relevance_list_cls,
                    prompt=self.choice_select_prompt,
                    context_str=fmt_batch_str,
                    query_str=query_str,
                )
                # in case structured prediction fails, a str of the raised exception is returned
                if isinstance(result, str):
                    if self._raise_on_structured_prediction_failure:
                        raise ValueError(
                            f"Structured prediction failed for nodes {idx} - {idx + self.choice_batch_size}: {result}"
                        )
                    logger.warning(
                        f"Structured prediction failed for nodes {idx} - {idx + self.choice_batch_size}: {result}"
                    )
                    # add all nodes with score 0
                    initial_results.extend(
                        [NodeWithScore(node=node, score=0.0) for node in nodes_batch]
                    )
                    continue

                raw_choices, relevances = self._parse_choice_select_answer_fn(
                    result, len(nodes_batch)
                )
                choice_idxs = [int(choice) - 1 for choice in raw_choices]
                choice_nodes = [nodes_batch[idx] for idx in choice_idxs]
                relevances = relevances or [1.0 for _ in choice_nodes]
                initial_results.extend(

View on GitHub (pinned to afd0fef371)

Solutions

1. Check the LLM structured output for the failing batch; retry with a model that supports structured output.
2. Reduce choice_batch_size so each structured prediction is simpler.

When it happens

Trigger: Thrown at llama-index-core/llama_index/core/postprocessor/structured_llm_rerank.py:189 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of run-llama/llama_index@afd0fef371 (2026-08-15). Data as JSON: /api/errors/10168daf3ea24b37. Report an issue: GitHub.