{"record":{"id":"6369e79ccfb6347c","repo":"sgl-project/sglang","slug":"the-hidden-states-sequence-length-hidden-states","errorCode":null,"errorMessage":"The `hidden_states` sequence length {hidden_states.shape[1]} should be divisible by the number of learnable registers {self.num_learnable_registers}","messagePattern":"The `hidden_states` sequence length (.+?) should be divisible by the number of learnable registers (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/models/adapter/ltx_2_connector.py","lineNumber":452,"sourceCode":"            self.inner_dim, eps=eps, elementwise_affine=False\n        )\n\n        self.gradient_checkpointing = False\n\n    def forward(\n        self,\n        hidden_states: torch.Tensor,\n        attention_mask: Optional[torch.Tensor] = None,\n        attn_mask_binarize_threshold: float = -9000.0,\n    ) -> Tuple[torch.Tensor, torch.Tensor]:\n        # hidden_states shape: [batch_size, seq_len, hidden_dim]\n        # attention_mask shape: [batch_size, seq_len] or [batch_size, 1, 1, seq_len]\n        batch_size, seq_len, _ = hidden_states.shape\n\n        # 1. Replace padding with learned registers, if using\n        if self.learnable_registers is not None:\n            if seq_len % self.num_learnable_registers != 0:\n                raise ValueError(\n                    f\"The `hidden_states` sequence length {hidden_states.shape[1]} should be divisible by the number\"\n                    f\" of learnable registers {self.num_learnable_registers}\"\n                )\n\n            num_register_repeats = seq_len // self.num_learnable_registers\n            registers = torch.tile(\n                self.learnable_registers, (num_register_repeats, 1)\n            )  # [seq_len, inner_dim]\n\n            binary_attn_mask = (attention_mask >= attn_mask_binarize_threshold).int()\n            if binary_attn_mask.ndim == 4:\n                binary_attn_mask = binary_attn_mask.squeeze(1).squeeze(\n                    1\n                )  # [B, 1, 1, L] --> [B, L]\n\n            hidden_states_non_padded = [\n                hidden_states[i, binary_attn_mask[i].bool(), :]\n                for i in range(batch_size)","sourceCodeStart":434,"sourceCodeEnd":470,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/models/adapter/ltx_2_connector.py#L434-L470","documentation":"ValueError from LTX-2 connector forward: when learnable_registers is enabled, hidden_states seq_len must be divisible by num_learnable_registers because the registers are tiled as seq_len // num_learnable_registers repeats of the register block to replace padding.","triggerScenarios":"Passing hidden_states whose sequence length (e.g. number of video tokens/patches) is not a multiple of num_learnable_registers while learnable_registers is not None.","commonSituations":"Custom resolutions or frame counts producing token counts not divisible by the register count; changing num_learnable_registers in config without adjusting patchified token counts; packing variable-length sequences.","solutions":["Pad hidden_states so seq_len is a multiple of num_learnable_registers (the module expects padding present by design)","Set num_learnable_registers to a value that divides your token count (commonly 1 or a small divisor)","Recompute expected seq_len from (frames/patches) and align it with the register config"],"exampleFix":"// before\nout = connector(hidden_states)  # seq_len=100, num_learnable_registers=4\n// after\npad = (-hidden_states.shape[1]) % connector.num_learnable_registers\nhidden_states = torch.nn.functional.pad(hidden_states, (0,0,0,pad,0,0))\nout = connector(hidden_states)","handlingStrategy":"validation","validationCode":"n = connector.num_learnable_registers\nif connector.learnable_registers is not None and hidden_states.shape[1] % n != 0:\n    pad = (-hidden_states.shape[1]) % n\n    hidden_states = torch.nn.functional.pad(hidden_states, (0,0,0,pad,0,0))","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Compute token counts from resolution/frames and keep them divisible by num_learnable_registers","Pad before forward instead of relying on the error","Validate config: num_learnable_registers should divide your typical seq_len"],"tags":["learnable-registers","shape-validation","ltx-2"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}