{"record":{"id":"099c27ffb04efd08","repo":"sgl-project/sglang","slug":"grpc-encode-only-supports-image-modality-got-no","errorCode":null,"errorMessage":"gRPC encode only supports IMAGE modality, got: {non_image}","messagePattern":"gRPC encode only supports IMAGE modality, got: (.+?)","errorType":"validation","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/disaggregation/encoder/receiver.py","lineNumber":2579,"sourceCode":"        embedding_port,\n        endpoint_encode,\n        num_items_assigned=None,\n        encode_urls=None,\n    ):\n        if not mm_data:\n            return\n\n        effective_urls = encode_urls if encode_urls is not None else self.encode_urls\n\n        # gRPC currently only supports image; flatten new dict formats to simple lists\n        if mm_data and isinstance(mm_data[0], dict):\n            non_image = [\n                item.get(\"modality\")\n                for item in mm_data\n                if item.get(\"modality\") != Modality.IMAGE\n            ]\n            if non_image:\n                raise NotImplementedError(\n                    f\"gRPC encode only supports IMAGE modality, got: {non_image}\"\n                )\n            img_data = [item.get(\"url\") for item in mm_data]\n        else:\n            img_data = mm_data\n        if isinstance(num_items_assigned, dict):\n            num_items_assigned = list(num_items_assigned.values())[0]\n\n        encode_requests = []\n        if num_items_assigned is None:\n            encode_idx = list(range(len(effective_urls)))\n            random.shuffle(encode_idx)\n            num_items_assigned = [\n                (idx + len(img_data)) // len(effective_urls) for idx in encode_idx\n            ]\n        num_parts = sum(1 for x in num_items_assigned if x != 0)\n        cum_num_items = 0\n        cum_idx = 0","sourceCodeStart":2561,"sourceCodeEnd":2597,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/disaggregation/encoder/receiver.py#L2561-L2597","documentation":"Raised in the gRPC encode path when any item in mm_data carries a modality other than IMAGE. The gRPC encoder interface currently only transports image payloads, so requests containing video or audio items are rejected with NotImplementedError.","triggerScenarios":"Calling encode on the gRPC receiver with mm_data entries whose \"modality\" is Modality.VIDEO or Modality.AUDIO; e.g. a multimodal request that includes a video_url or audio input while using grpc:// encoder URLs.","commonSituations":"Pointing a vision-language-plus-audio workload (e.g. Qwen-Audio/Video style requests) at a gRPC encoder deployment; the receiver auto-flattening video_data/audio_data into mm_data which then fails the image-only check.","solutions":["Send image-only requests to gRPC encoders, or strip video/audio items before encoding.","Switch to HTTP encoder URLs / receiver mode which supports the broader modality set.","Check upstream request construction: video_data or audio_data being non-empty triggers this."],"exampleFix":"# before\nreq.video_data = [video_item]  # with grpc encoder urls\n# after\nreq.video_data = None  # or use http:// encoder urls for video workloads","handlingStrategy":"validation","validationCode":"def is_image_only(mm_data: list) -> bool:\n    return all(item.get(\"modality\") == Modality.IMAGE for item in mm_data)\n\nif transport_mode == \"grpc\":\n    assert is_image_only(mm_data), \"gRPC encode path accepts IMAGE items only\"","typeGuard":"def is_image_request(req) -> bool:\n    return not (req.video_data or req.audio_data)","tryCatchPattern":"try:\n    result = await receiver.encode(...)\nexcept NotImplementedError as e:\n    if \"IMAGE modality\" in str(e):\n        raise HTTPException(400, \"video/audio not supported over gRPC encoders; use http mode\") from e\n    raise","preventionTips":["Route audio/video workloads to HTTP-mode encoders only.","Clear video_data/audio_data on requests destined for gRPC encoders.","Feature-detect modality support before choosing transport mode."],"tags":["grpc","multimodal","modality","not-implemented","encoder"],"backgroundTag":"unsupported-feature-combination","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}