{"record":{"id":"58c1b2a9b4042317","repo":"sgl-project/sglang","slug":"select-choices-is-not-supported-for-chat-models-p","errorCode":null,"errorMessage":"select/choices is not supported for chat models. Please try to use a non-chat model such as gpt-3.5-turbo-instruct","messagePattern":"select/choices is not supported for chat models\\. Please try to use a non-chat model such as gpt-3\\.5-turbo-instruct","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/lang/backend/openai.py","lineNumber":321,"sourceCode":"                is_chat=self.is_chat_model,\n                model=self.model_name,\n                prompt=prompt,\n                **kwargs,\n            )\n            return generator\n        else:\n            raise ValueError(f\"Unknown dtype: {sampling_params.dtype}\")\n\n    def select(\n        self,\n        s: StreamExecutor,\n        choices: List[str],\n        temperature: float,\n        choices_method: ChoicesSamplingMethod,\n    ) -> ChoicesDecision:\n        \"\"\"Note: `choices_method` is not used by the OpenAI backend.\"\"\"\n        if self.is_chat_model:\n            raise NotImplementedError(\n                \"select/choices is not supported for chat models. \"\n                \"Please try to use a non-chat model such as gpt-3.5-turbo-instruct\"\n            )\n\n        n_choices = len(choices)\n        token_ids = [self.tokenizer.encode(x) for x in choices]\n        scores = [0] * n_choices\n        valid = [len(x) > 0 for x in token_ids]\n        prompt_tokens = self.tokenizer.encode(s.text_)\n\n        max_len = max([len(x) for x in token_ids])\n        for step in range(max_len):\n            # Build logit bias\n            logit_bias = {}\n            for i in range(n_choices):\n                if valid[i]:\n                    logit_bias[token_ids[i][step]] = 100\n","sourceCodeStart":303,"sourceCodeEnd":339,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/lang/backend/openai.py#L303-L339","documentation":"The OpenAI backend's select() explicitly raises NotImplementedError for chat models: choice scoring relies on logprobs over supplied choice token sequences, which chat/completions cannot provide. The message directs users to a completion model (e.g. gpt-3.5-turbo-instruct) that exposes logprobs.","triggerScenarios":"Calling sgl.select(choices=[...]) in a program running on an OpenAI chat model backend (is_chat_model=True), i.e. model names containing 'chat'/'gpt-3.5-turbo'/'gpt-4' style chat endpoints.","commonSituations":"Porting choice-based workflows (classification, constrained selection) from local sglang or completion APIs to OpenAI chat models.","solutions":["Switch to a non-chat model such as gpt-3.5-turbo-instruct for sgl.select.","Replace sgl.select with sgl.gen plus prompt-based enumeration of choices and parse the text.","Use the RuntimeEndpoint backend (local sglang server), which supports select natively."],"exampleFix":"# before\nbackend = sgl.OpenAI(\"gpt-4o\")\n... sgl.select(x, choices=[\"yes\",\"no\"], temperature=0)\n\n# after\nbackend = sgl.OpenAI(\"gpt-3.5-turbo-instruct\")\n... sgl.select(x, choices=[\"yes\",\"no\"], temperature=0)","handlingStrategy":"type-guard","validationCode":"if getattr(backend, \"is_chat_model\", False):\n    # replace select with gen-based choice prompt\n    pass","typeGuard":"def supports_select(backend) -> bool:\n    return not getattr(backend, \"is_chat_model\", False) and \\\n           type(backend).select is not BaseBackend.select","tryCatchPattern":"try:\n    decision = backend.select(s, choices, temperature, method)\nexcept NotImplementedError:\n    # chat model: enumerate choices in prompt and parse answer\n    s += sgl.gen(\"pick\", ...)","preventionTips":["Check backend.is_chat_model before select-based workflows.","Default to gen+parse for portability across backends."],"tags":["frontend","openai","choices","chat-model","not-supported","sglang"],"backgroundTag":"feature-unsupported-on-chat-model","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}