{"record":{"id":"713fddedc2729b02","repo":"sgl-project/sglang","slug":"joyimage-conditioning-batch-mismatch-hidden-state","errorCode":null,"errorMessage":"JoyImage conditioning batch mismatch: hidden_states batch={batch_size}, encoder_hidden_states batch={cond_batch}.","messagePattern":"JoyImage conditioning batch mismatch: hidden_states batch=(.+?), encoder_hidden_states batch=(.+?)\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/models/dits/joy_image.py","lineNumber":497,"sourceCode":"        forward_batch = get_forward_context().forward_batch\n        sequence_shard_enabled = (\n            forward_batch is not None\n            and getattr(forward_batch, \"enable_sequence_shard\", False)\n            and self.sp_size > 1\n        )\n\n        batch_size = hidden_states.shape[0]\n\n        if not isinstance(encoder_hidden_states, torch.Tensor):\n            encoder_hidden_states = encoder_hidden_states[0]\n\n        if isinstance(encoder_hidden_states_mask, list):\n            encoder_hidden_states_mask = encoder_hidden_states_mask[0]\n\n        cond_batch = int(encoder_hidden_states.shape[0])\n        if cond_batch != int(batch_size):\n            if cond_batch <= 0 or int(batch_size) % cond_batch != 0:\n                raise ValueError(\n                    \"JoyImage conditioning batch mismatch: \"\n                    f\"hidden_states batch={batch_size}, \"\n                    f\"encoder_hidden_states batch={cond_batch}.\"\n                )\n            repeat_factor = int(batch_size) // cond_batch\n            encoder_hidden_states = encoder_hidden_states.repeat_interleave(\n                repeat_factor, dim=0\n            )\n            if encoder_hidden_states_mask is not None:\n                encoder_hidden_states_mask = (\n                    encoder_hidden_states_mask.repeat_interleave(repeat_factor, dim=0)\n                )\n\n        # Prepare img\n        x = rearrange(hidden_states, \"b n c p1 p2 p3 -> (b n) c p1 p2 p3\")\n        x = self.img_in(x)\n        img = rearrange(x, \"(b n) d 1 1 1 -> b n d\", b=batch_size)\n","sourceCodeStart":479,"sourceCodeEnd":515,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/models/dits/joy_image.py#L479-L515","documentation":"JoyImage allows the conditioning (encoder_hidden_states) batch to be smaller than the latent batch only when the latent batch is an exact integer multiple (e.g. N CFG samples per condition). If the conditioning batch is not a positive divisor of the latent batch, shapes cannot be reconciled and forward aborts.","triggerScenarios":"hidden_states has batch B but encoder_hidden_states has batch C where B % C != 0 or C <= 0, e.g. latents batch=3 (odd CFG/batched mix) with a single conditioning tensor of batch=2.","commonSituations":"Batching requests with different guidance scales so B is not 2*C; per-prompt conditioning counts not aligned with the latent batch; empty conditioning tensor (batch 0).","solutions":["Make the latent batch an exact multiple of the conditioning batch (typically B = 2*C for CFG, or B = C without CFG)","repeat_interleave the conditioning yourself to exactly B rows before calling forward","Check the scheduler/batcher that produced hidden_states for off-by-one guidance duplication"],"exampleFix":"// before\nnoise = torch.randn(3, ...)          # batch 3\ncond = encode(prompts)              # batch 2\ndit(noise, encoder_hidden_states=cond)\n\n// after\nnoise = torch.randn(4, ...)          # 2 * cond batch for CFG\ncond = encode(prompts)               # batch 2\ndit(noise, encoder_hidden_states=cond)","handlingStrategy":"validation","validationCode":"B = hidden_states.shape[0]; C = encoder_hidden_states.shape[0]\\nassert C > 0 and B % C == 0, f'latent batch {B} must be a positive multiple of cond batch {C}'","typeGuard":"def conditioning_batches_align(h: torch.Tensor, c: torch.Tensor) -> bool:\\n    return c.shape[0] > 0 and h.shape[0] % c.shape[0] == 0","tryCatchPattern":null,"preventionTips":["Derive latent batch as 2 * len(prompts) under CFG","Expand conditioning to full batch yourself instead of relying on implicit repeat","Log both batch sizes in your sampler"],"tags":["batch-mismatch","cfg-conditioning","joyimage","multimodal"],"backgroundTag":"batch-dimension-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}