{"record":{"id":"1b814e733c1c53c6","repo":"sgl-project/sglang","slug":"glm-image-ar-batch-returned-an-unexpected-response","errorCode":null,"errorMessage":"GLM-Image AR batch returned an unexpected response: expected {len(prompts)} outputs, got {len(data) if isinstance(data, list) else type(data).__name__}.","messagePattern":"GLM-Image AR batch returned an unexpected response: expected (.+?) outputs, got (.+?)\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/glm_image.py","lineNumber":563,"sourceCode":"                    image_grid_thw=image_grid_thw,\n                    is_text_to_image=True,\n                )\n            )\n            input_ids.append(inputs[\"input_ids\"][0].tolist())\n            image_data.append([{\"image_grid_thw\": image_grid_thw.tolist()}])\n            sampling_params.append(\n                self._external_ar_sampling_params(max_new_tokens, seed)\n            )\n            generation_shapes.append((large_image_offset, token_h, token_w))\n\n        payload = {\n            \"input_ids\": input_ids,\n            \"image_data\": image_data,\n            \"sampling_params\": sampling_params,\n        }\n        data = self._request_external_ar(payload, server_args)\n        if not isinstance(data, list) or len(data) != len(prompts):\n            raise RuntimeError(\n                \"GLM-Image AR batch returned an unexpected response: \"\n                f\"expected {len(prompts)} outputs, got \"\n                f\"{len(data) if isinstance(data, list) else type(data).__name__}.\"\n            )\n\n        prior_token_ids = []\n        usages = []\n        for item, generation_shape in zip(data, generation_shapes, strict=True):\n            prior_token_ids.append(\n                self._extract_prior_token_ids(\n                    item.get(\"output_ids\"), generation_shape, device\n                )\n            )\n            usages.append(_extract_srt_usage(item.get(\"meta_info\")))\n        return prior_token_ids, usages\n\n    def run_grouped_requests(\n        self,","sourceCodeStart":545,"sourceCodeEnd":581,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/glm_image.py#L545-L581","documentation":"When batching prior-token generation against the external SGLang AR endpoint, the response payload was expected to be a JSON list with exactly one output entry per input prompt. The response was either not a list or had a different length than the batch, indicating a malformed, truncated, or misrouted server response.","triggerScenarios":"Calling generate_prior_tokens_batch with N prompts while server_args.srt_encoder_url points at an AR server; the endpoint returns a dict (e.g. an error object like {\"error\": ...}) or a list whose length differs from the number of prompts sent.","commonSituations":"The external server hit an OOM/limit and returned a partial batch or an error JSON; version mismatch between client payload schema and server response schema; a proxy/load balancer returning an HTML or dict error body; accidental misalignment between prompts and image_data lists built by the caller.","solutions":["Log/inspect the raw response body from _request_external_ar to see whether it is an error dict or partial list","Check the external AR server logs for the failing batch (OOM, context-length, dropped requests)","Verify the server runs a compatible SGLang version whose batch AR endpoint returns one entry per prompt","Retry with a smaller batch size to rule out server-side truncation","Ensure len(image_data) and len(input_ids) match len(prompts) before sending"],"exampleFix":"# before\npriors = stage.generate_prior_tokens_batch(prompts=8 * [p], ...)\n\n# after\ndata = stage._request_external_ar(payload, server_args)\nassert isinstance(data, list) and len(data) == len(prompts), data  # surface server error early\npriors = stage.generate_prior_tokens_batch(prompts=8 * [p], ...)","handlingStrategy":"retry","validationCode":"assert len(prompts) == len(input_ids) == len(image_data), \"batch inputs misaligned\"","typeGuard":null,"tryCatchPattern":"for attempt in range(3):\n    try:\n        return stage.generate_prior_tokens_batch(prompts, ...)\n    except RuntimeError as e:\n        if \"unexpected response\" in str(e) and attempt < 2:\n            time.sleep(2 ** attempt)\n            continue\n        raise","preventionTips":["Health-check the external AR server before batches","Keep batches small enough to avoid server OOM/truncation","Pin matching client/server SGLang versions"],"tags":["glm-image","external-server","batch-mismatch","response-validation","runtimeerror"],"backgroundTag":"api-response-schema-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}