{"record":{"id":"3d9765794cd2e2f5","repo":"sgl-project/sglang","slug":"cosmos3-requires-text-ids-and-text-mask-to-be-pass","errorCode":null,"errorMessage":"Cosmos3 requires text_ids and text_mask to be passed","messagePattern":"Cosmos3 requires text_ids and text_mask to be passed","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/models/dits/cosmos3video.py","lineNumber":1482,"sourceCode":"            max_text_seq_len: Real text length already computed during\n                tokenization. When omitted it is derived from ``text_mask``.\n            action_latents: Optional [B, T_action, D_action] noisy action\n                latents for action generation.\n            action_domain_ids: [B] embodiment domain IDs (0=no-action default).\n            action_noisy_mask: [B, T_action, 1] where 1=noisy, 0=conditioned;\n                controls which action tokens receive the timestep embedding.\n                ``None`` means all tokens are noisy.\n            action_fps: Frame rate for action token temporal mRoPE scaling.\n                Defaults to the video fps when None.\n            action_start_frame_offset: Temporal offset applied to action\n                position IDs relative to the video's media_offset (default 1).\n\n        Returns:\n            [B, C, T, H, W] velocity prediction, or a tuple\n            (video_pred, ...) with extra tensors when action/sound are active.\n        \"\"\"\n        if text_ids is None or text_mask is None:\n            raise ValueError(\"Cosmos3 requires text_ids and text_mask to be passed\")\n\n        batch_size, C, T, H, W = hidden_states.shape\n        Hp, Wp, _, _ = self._pad_to_patch_size(H, W)\n        if max_text_seq_len is None:\n            max_text_seq_len = int(text_mask.sum(dim=1).max().item())\n        if max_text_seq_len < text_ids.shape[1]:\n            text_ids = text_ids[:, :max_text_seq_len]\n            text_mask = text_mask[:, :max_text_seq_len]\n\n        sound_frames = sound_latents.shape[-1] if sound_latents is not None else 0\n\n        action_frames = 0\n        if action_latents is not None:\n            if self.sp_size > 1:\n                raise NotImplementedError(\n                    \"Cosmos3 action generation does not support sequence parallelism yet\"\n                )\n            action_frames = action_latents.shape[1]","sourceCodeStart":1464,"sourceCodeEnd":1500,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/models/dits/cosmos3video.py#L1464-L1500","documentation":"Cosmos3's DiT forward is text-conditioned: it requires both text_ids (text token ids/embeddings) and text_mask (validity mask) to compute cross-attention over the conditioning. Calling forward without either raises ValueError immediately, before any compute.","triggerScenarios":"Invoking the Cosmos3 model/diffusion pipeline with hidden_states only, e.g. model(hidden_states, timestep) with text_ids=None or text_mask=None; or a runner that only passes an empty prompt and drops the mask.","commonSituations":"Building an unconditional/unprompted video generation path and assuming text inputs are optional; wiring a new scheduler/runtime that forgets to forward tokenizer outputs; empty prompt handled by skipping both args instead of passing empty tensors with an all-zero mask.","solutions":["Pass both text_ids and text_mask derived from the tokenizer/conditioner for every forward call","For unconditional generation, pass an empty tensor for text_ids with a matching all-false text_mask rather than None","Audit the calling runtime to ensure the text-conditioning outputs are plumbed into the DiT forward signature"],"exampleFix":"# before\nout = dit(hidden_states, timestep, text_ids=None, text_mask=None)\n# after\nout = dit(hidden_states, timestep, text_ids=text_ids, text_mask=text_mask)\n# unconditional:\nout = dit(hidden_states, timestep,\n         text_ids=torch.zeros(B, 0, dtype=torch.long, device=hidden_states.device),\n         text_mask=torch.zeros(B, 0, dtype=torch.bool, device=hidden_states.device))","handlingStrategy":"validation","validationCode":"assert text_ids is not None and text_mask is not None, \"Cosmos3 requires text conditioning\"\n# or supply empties for unconditional runs:\ntext_ids = text_ids if text_ids is not None else torch.zeros(B, 0, dtype=torch.long, device=dev)\ntext_mask = text_mask if text_mask is not None else torch.zeros(B, 0, dtype=torch.bool, device=dev)","typeGuard":"def has_text_conditioning(kwargs: dict) -> bool:\n    return kwargs.get(\"text_ids\") is not None and kwargs.get(\"text_mask\") is not None","tryCatchPattern":"try:\n    out = dit(...)\nexcept ValueError as e:\n    if \"text_ids and text_mask\" in str(e):\n        raise RuntimeError(\"conditioner did not produce text inputs\") from e\n    raise","preventionTips":["Always run the tokenizer/conditioner before forward and thread its outputs through","Use empty tensors plus mask instead of None for unconditional generation"],"tags":["sglang","cosmos3","text-conditioning","forward-validation","required-argument"],"backgroundTag":"missing-required-argument","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}