sgl-project/sglang · error · ValueError

InklingMultimodalProcessor v1 requires pre-rendered input_id

Error message

InklingMultimodalProcessor v1 requires pre-rendered input_ids (request_obj.input_ids); the custom Inkling chat renderer is a separate workstream. No tokenizer available to render text.

What it means

Inkling's v1 processor cannot render raw text prompts: it needs pre-rendered input_ids on the request, and if input_ids is missing it only falls back to self._tokenizer when one is available. When no tokenizer is present, it raises to avoid silently mis-rendering prompts with the custom Inkling chat format.

Source

Thrown at python/sglang/srt/multimodal/processors/inkling.py:302

            audio_token_id=self.AUDIO_TOKEN_ID,
            audio_end_id=self.AUDIO_END_TOKEN_ID,
        )

    # ---- SGLang entrypoint ----------------------------------------------

    async def process_mm_data_async(
        self,
        image_data: Optional[List[Union[str, bytes, Dict]]] = None,
        audio_data: Optional[List[Union[str, bytes, Dict]]] = None,
        input_text: str = "",
        request_obj: Any = None,
        *args,
        **kwargs,
    ) -> Optional[MultimodalProcessorOutput]:
        input_ids = getattr(request_obj, "input_ids", None)
        if input_ids is None:
            if self._tokenizer is None:
                raise ValueError(
                    "InklingMultimodalProcessor v1 requires pre-rendered input_ids "
                    "(request_obj.input_ids); the custom Inkling chat renderer is a "
                    "separate workstream. No tokenizer available to render text."
                )
            input_ids = self._tokenizer(input_text).input_ids
        if isinstance(input_ids, torch.Tensor):
            input_ids = input_ids.flatten().tolist()

        # Resolve request media (data:/http URLs, ImageData objects) to bytes so the
        # Inkling preprocessors can consume them; bytes / paths pass through unchanged.
        if image_data:
            image_data = [_resolve_media_item(it) for it in image_data]
        if audio_data:
            audio_data = [_resolve_media_item(it) for it in audio_data]
        return self.assemble(list(input_ids), image_data, audio_data)

View on GitHub (pinned to 0132848349)

Solutions

1. Pre-render input_ids with the Inkling chat renderer and pass them via request_obj.input_ids
2. Ensure the processor is constructed with the tokenizer so the text fallback works
3. Route text-only requests through the tokenizer manager instead of the raw processor API

Example fix

# before
out = await processor.process_mm_data_async(None, {'text': prompt}, request_obj)
# after
request_obj.input_ids = tokenizer(prompt)['input_ids']
out = await processor.process_mm_data_async(None, {'input_ids': request_obj.input_ids}, request_obj)

if getattr(request_obj, 'input_ids', None) is None and processor has no tokenizer:
    request_obj.input_ids = render_inkling_template(msgs)  # must pre-render

def request_has_prerendered_ids(req) -> bool:
    return getattr(req, 'input_ids', None) is not None

try:
    out = await processor.process_mm_data_async(None, mm_data, req)
except ValueError as e:
    if 'pre-rendered input_ids' in str(e):
        req.input_ids = tokenizer(req.input_text)['input_ids']
        out = await processor.process_mm_data_async(None, mm_data, req)
    else: raise

When it happens

Trigger: Calling process_mm_data_async with a text-only request_obj (input_ids=None) in a deployment path where the processor was constructed without a tokenizer; sending {'text': ...} instead of {'input_ids': [...]}.

Common situations: Embedding/offline pipelines that instantiate the processor standalone without the tokenizer manager; engine configs that skip tokenizer init for pre-tokenized-only workloads but then receive text requests.

Related errors

• InklingMultimodalProcessor: required config field {where}.{n
• InklingMultimodalProcessor: {n_img_ph} image placeholder tok
• InklingMultimodalProcessor: {n_aud_ph} audio placeholder tok
• [internvl] Cannot process raw images/videos with pre-tokeniz
• MXFP8 fused prologue requires head_dim-aligned Q/K/V.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/e3bfc1f4f4c1708f. Report an issue: GitHub.