{"record":{"id":"35010ef441a355b5","repo":"microsoft/semantic-kernel","slug":"failed-to-select-an-agent-since-the-model-did-not","errorCode":null,"errorMessage":"Failed to select an agent since the model did not return a valid index","messagePattern":"Failed to select an agent since the model did not return a valid index","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/samples/demos/document_generator/custom_selection_strategy.py","lineNumber":74,"sourceCode":"            chat_history.add_user_message(\"Now follow the rules and select the next agent by typing the agent's index.\")\n\n            for _ in range(self.NUM_OF_RETRIES):\n                completion = await self.chat_completion_service.get_chat_message_content(\n                    chat_history,\n                    AzureChatPromptExecutionSettings(),\n                )\n\n                if completion is None:\n                    continue\n\n                try:\n                    return agents[int(completion.content)]\n                except ValueError as ex:\n                    chat_history.add_message(completion)\n                    chat_history.add_user_message(str(ex))\n                    chat_history.add_user_message(f\"You must only say a number between 0 and {len(agents) - 1}.\")\n\n            raise ValueError(\"Failed to select an agent since the model did not return a valid index\")\n\n    def get_system_message(self, agents: list[\"Agent\"]) -> str:\n        return f\"\"\"\nYou are in a multi-agent chat to create a document.\nEach message in the chat history contains the agent's name and the message content.\n\nInitially, the chat history may be empty.\n\nHere are the agents with their indices, names, and descriptions:\n{NEWLINE.join(f\"[{index}] {agent.name}:{NEWLINE}{agent.description}\" for index, agent in enumerate(agents))}\n\nYour task is to select the next agent based on the conversation history.\n\nThe conversation must follow these steps:\n1. The content creation agent writes a draft.\n2. The code validation agent checks the code in the draft.\n3. The content creation agent updates the draft based on the feedback.\n4. The code validation agent checks the updated code.","sourceCodeStart":56,"sourceCodeEnd":92,"githubUrl":"https://github.com/microsoft/semantic-kernel/blob/c028a0c7dc4f0814cdcbaba9d998f187a41197bf/python/samples/demos/document_generator/custom_selection_strategy.py#L56-L92","documentation":"Raised by custom_selection_strategy.next() after the model repeatedly returned content that cannot be parsed as an integer index (int(completion.content) raises ValueError each retry). The strategy asks the LLM to output a number 0..len(agents)-1; if after all retries the model still fails, it gives up and raises.","triggerScenarios":"The LLM returns prose, multiple numbers, out-of-range numbers, or non-numeric tokens when asked for the agent index; a weak model or low temperature setup that ignores the 'only a number' instruction; completion.content is None or empty.","commonSituations":"Using a smaller/cheaper model that struggles with strict formatting; the prompt being drowned by long history; the agent index exceeding single digits where parsing gets ambiguous.","solutions":["Switch to a stronger instruction-following model for selection (e.g. gpt-4o-class).","Pre-strip and validate completion.content: extract the first integer via regex before int(), reducing parse failures.","Increase the number of retries the loop performs before giving up.","Lower temperature / tighten the system prompt so the model emits only a number."],"exampleFix":"// before\nreturn agents[int(completion.content)]\n\n// after\nimport re\nm = re.search(r'\\d+', completion.content or '')\nif not m:\n    chat_history.add_message(completion)\n    chat_history.add_user_message('Respond with ONLY a single integer index.')\n    continue\nidx = int(m.group())\nif 0 <= idx < len(agents):\n    return agents[idx]","handlingStrategy":"try-catch","validationCode":"import re\ndef parse_index(content: str, count: int) -> int | None:\n    m = re.search(r'\\d+', content or '')\n    if not m:\n        return None\n    idx = int(m.group())\n    return idx if 0 <= idx < count else None","typeGuard":null,"tryCatchPattern":"try:\n    return agents[int(completion.content)]\nexcept (ValueError, IndexError, TypeError) as ex:\n    chat_history.add_message(completion)\n    chat_history.add_user_message(f'Reply with a single integer 0..{len(agents)-1}. Error: {ex}')\n    continue","preventionTips":["Use a stronger model for selection.","Pre-extract digits from the model output before int().","Tune the system prompt to demand a bare integer."],"tags":["llm","multi-agent","selection-strategy","parsing","model-output"],"backgroundTag":null,"analyzedSha":"c028a0c7dc4f0814cdcbaba9d998f187a41197bf","analyzedAt":"2026-08-13T13:48:05.040Z","schemaVersion":2},"datasetVersion":"2026-08-13T14:17:21.547Z"}