{"record":{"id":"17d5c2cfa055f050","repo":"hiyouga/LlamaFactory","slug":"sglang-server-error-response-status-code-resp","errorCode":null,"errorMessage":"SGLang server error: {response.status_code}, {response.text}","messagePattern":"SGLang server error: (.+?), (.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/chat/sglang_engine.py","lineNumber":222,"sourceCode":"            or 1.0,  # repetition_penalty must > 0\n            \"skip_special_tokens\": skip_special_tokens\n            if skip_special_tokens is not None\n            else self.generating_args[\"skip_special_tokens\"],\n        }\n        if seed is not None:\n            sampling_params[\"seed\"] = seed\n\n        def stream_request():\n            json_data = {\n                \"input_ids\": prompt_ids,\n                \"sampling_params\": sampling_params,\n                \"stream\": True,\n            }\n            if self.lora_request:\n                json_data[\"lora_request\"] = [\"lora0\"]\n            response = requests.post(f\"{self.base_url}/generate\", json=json_data, stream=True)\n            if response.status_code != 200:\n                raise RuntimeError(f\"SGLang server error: {response.status_code}, {response.text}\")\n\n            for chunk in response.iter_lines(decode_unicode=False):\n                chunk = str(chunk.decode(\"utf-8\"))\n                if chunk == \"data: [DONE]\":\n                    break\n\n                if chunk and chunk.startswith(\"data:\"):\n                    yield json.loads(chunk[5:].strip(\"\\n\"))\n\n        return await asyncio.to_thread(stream_request)\n\n    @override\n    async def chat(\n        self,\n        messages: Sequence[dict[str, str]],\n        system: Optional[str] = None,\n        tools: Optional[str] = None,\n        images: Optional[Sequence[\"ImageInput\"]] = None,","sourceCodeStart":204,"sourceCodeEnd":240,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/chat/sglang_engine.py#L204-L240","documentation":"Raised when the POST to the local SGLang server's /generate endpoint returns a non-200 status. The engine posts input_ids plus a sampling_params dict (and optionally lora_request) with stream=True; any HTTP error body returned by the server (invalid sampling param, token mismatch, LoRA not loaded, internal crash) is surfaced verbatim as RuntimeError with the status code and response text.","triggerScenarios":"Calling generate/chat on SGLangEngine when: a sampling_params key is rejected by the installed sglang version; the LoRA adapter named in lora_request was not served; input_ids contain token ids outside the model vocab; or the server hit an internal error (e.g. OOM during decode) after startup.","commonSituations":"Version skew between the client-built sampling params and the running sglang server; using --adapter_name_or_path with an engine where the LoRA failed to load; extremely long prompts exceeding max_model_len causing a 400/500 from the server.","solutions":["Inspect response.text embedded in the message — it is the SGLang server's own error body and names the exact bad parameter or fault.","Cross-check every key placed into sampling_params (temperature, top_p, top_k, repetition_penalty, max_tokens, stop, seed...) against your installed sglang version's supported SamplingParams.","If a LoRA is involved, confirm the adapter was passed at server start and the name matches (\"lora0\").","Reduce prompt length or raise the server's max_model_len if the error is a context-length rejection.","Restart the engine if the server died mid-run (see the init log for the crash)."],"exampleFix":"# before\nresponse = requests.post(f\"{self.base_url}/generate\", json=json_data, stream=True)\n\n# after (caller-side preflight: validate params against the server)\ninfo = requests.get(f\"{self.base_url}/get_model_info\", timeout=5).json()\n# strip unsupported sampling keys before calling engine.chat(...)","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n    async for delta in engine.chat(messages, **kwargs):\n        ...\nexcept RuntimeError as e:\n    if \"SGLang server error\" in str(e):\n        status, _, body = str(e).partition(\",\")\n        logger.error(\"sglang %s body=%s\", status, body)  # body names the bad param\n        # fix sampling kwargs or restart engine; do not blind-retry 4xx-style errors","preventionTips":["Pin sglang and LlamaFactory versions together; re-validate sampling kwarg names after upgrades.","Prefer template-generated prompts that already fit max_model_len.","Watch the server log alongside the client so HTTP errors map to server-side causes."],"tags":["sglang","http","inference","api-error"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}