{"record":{"id":"799b3a6a9984f5ba","repo":"sgl-project/sglang","slug":"inklingmultimodalprocessor-n-aud-ph-audio-place","errorCode":null,"errorMessage":"InklingMultimodalProcessor: {n_aud_ph} audio placeholder token(s) in input_ids but {len(audio_data)} audio(s) provided; counts must match.","messagePattern":"InklingMultimodalProcessor: (.+?) audio placeholder token\\(s\\) in input_ids but (.+?) audio\\(s\\) provided; counts must match\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/multimodal/processors/inkling.py","lineNumber":215,"sourceCode":"        # One placeholder per media item (expanded below); a count mismatch (incl. a\n        # None token id absent from config) must fail loudly, not drop media silently.\n        n_img_ph = (\n            sum(1 for t in input_ids if t == self.IMAGE_TOKEN_ID)\n            if self.IMAGE_TOKEN_ID is not None\n            else 0\n        )\n        n_aud_ph = (\n            sum(1 for t in input_ids if t == self.AUDIO_TOKEN_ID)\n            if self.AUDIO_TOKEN_ID is not None\n            else 0\n        )\n        if n_img_ph != len(image_data):\n            raise ValueError(\n                f\"InklingMultimodalProcessor: {n_img_ph} image placeholder token(s) in \"\n                f\"input_ids but {len(image_data)} image(s) provided; counts must match.\"\n            )\n        if n_aud_ph != len(audio_data):\n            raise ValueError(\n                f\"InklingMultimodalProcessor: {n_aud_ph} audio placeholder token(s) in \"\n                f\"input_ids but {len(audio_data)} audio(s) provided; counts must match.\"\n            )\n\n        img_feat = (\n            self.inkling_processor.process_images(image_data) if image_data else None\n        )\n        aud_feat = (\n            self.inkling_processor.process_audios(audio_data) if audio_data else None\n        )\n\n        # Rust processor returns content_hashes; original processor does not.\n        img_hashes = img_feat.get(\"content_hashes\") if img_feat else None\n\n        out_ids: List[int] = []\n        image_items: List[Tuple[int, int, torch.Tensor]] = []  # (start, end, feature)\n        audio_items: List[Tuple[int, int, torch.Tensor]] = []\n        i_img = i_aud = 0","sourceCodeStart":197,"sourceCodeEnd":233,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/multimodal/processors/inkling.py#L197-L233","documentation":"Companion check to the image one: the count of audio placeholder tokens in input_ids must equal len(audio_data) during Inkling assembly. Mismatches mean the prompt was rendered without the right number of audio placeholders, or the audio list diverged from the messages used to build the prompt.","triggerScenarios":"Pre-tokenized input_ids rendered with a text-only template while audio_data is non-empty; audio list filtered (e.g. dropping unreadable files) after the prompt was built; AUDIO_TOKEN_ID is None so n_aud_ph=0 while audio was supplied.","commonSituations":"Chat template that doesn't expand consecutive audio tokens; data loader dropping corrupt audio files after prompt construction; template upgrade changing audio token syntax.","solutions":["Re-render input_ids so every audio sample gets exactly one audio placeholder token","Ensure audio filtering/removal happens before prompt construction so counts stay in sync","Check that AUDIO_TOKEN_ID is configured (not None) when audio is used"],"exampleFix":"# before\nprompt = template.render(msgs)            # text only\ninput_ids = tok(prompt)['input_ids']       # 0 audio tokens, 1 audio file\n# after\nmsgs = insert_audio_placeholders(msgs, n=len(audio_data))\ninput_ids = tok(template.render(msgs))['input_ids']","handlingStrategy":"validation","validationCode":"n_ph = sum(1 for t in input_ids if t == processor.AUDIO_TOKEN_ID) if processor.AUDIO_TOKEN_ID else 0\nassert n_ph == len(audio_data or []), f'{n_ph} audio placeholders vs {len(audio_data)} audio'","typeGuard":"def audio_placeholders_match(input_ids: list[int], token_id: int|None, data: list) -> bool:\n    n = sum(1 for t in input_ids if t == token_id) if token_id is not None else 0\n    return n == len(data or [])","tryCatchPattern":"try:\n    out = await processor.process_mm_data_async(None, None, request_obj)\nexcept ValueError as e:\n    if 'audio placeholder' in str(e):\n        request_obj.input_ids = render_with_audio_placeholders(msgs, n=len(audio_data))\n        out = await processor.process_mm_data_async(None, None, request_obj)\n    else: raise","preventionTips":["Filter unreadable audio before prompt construction","Ensure the chat template emits one audio token per clip"],"tags":["multimodal","inkling","placeholder-mismatch","audio"],"backgroundTag":"placeholder-count-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}