{"record":{"id":"7d606074555f9fc4","repo":"sgl-project/sglang","slug":"unknown-history-scale-mode-history-scale-mode","errorCode":null,"errorMessage":"Unknown history_scale_mode: {history_scale_mode}","messagePattern":"Unknown history_scale_mode: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/models/dits/helios.py","lineNumber":284,"sourceCode":"        self.norm_q = RMSNorm(dim, eps=eps)\n        self.norm_k = RMSNorm(dim, eps=eps)\n        self.tp_rmsnorm = tp_size > 1\n\n        self.attn = USPAttention(\n            num_heads=self.local_num_heads,\n            head_size=self.head_dim,\n            causal=False,\n            is_cross_attention=False,\n        )\n\n        self.is_amplify_history = is_amplify_history\n        if is_amplify_history:\n            if history_scale_mode == \"scalar\":\n                self.history_key_scale = nn.Parameter(torch.ones(1))\n            elif history_scale_mode == \"per_head\":\n                self.history_key_scale = nn.Parameter(torch.ones(num_heads))\n            else:\n                raise ValueError(f\"Unknown history_scale_mode: {history_scale_mode}\")\n            self.history_scale_mode = history_scale_mode\n            self.max_scale = 10.0\n\n    def forward(self, hidden_states, rotary_emb=None, original_context_length=None):\n        q, _ = self.to_q(hidden_states)\n        k, _ = self.to_k(hidden_states)\n        v, _ = self.to_v(hidden_states)\n\n        if self.tp_rmsnorm:\n            q = tensor_parallel_rms_norm(q, self.norm_q)\n            k = tensor_parallel_rms_norm(k, self.norm_k)\n        else:\n            q = self.norm_q(q)\n            k = self.norm_k(k)\n\n        q = q.unflatten(2, (self.local_num_heads, self.head_dim))\n        k = k.unflatten(2, (self.local_num_heads, self.head_dim))\n        v = v.unflatten(2, (self.local_num_heads, self.head_dim))","sourceCodeStart":266,"sourceCodeEnd":302,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/models/dits/helios.py#L266-L302","documentation":"In the Helios DiT attention module, when is_amplify_history is enabled the history key scaling mode must be 'scalar' (single learnable scale) or 'per_head' (one scale per head). Any other history_scale_mode string raises this ValueError in __init__.","triggerScenarios":"Constructing Helios attention with is_amplify_history=True and history_scale_mode set to something other than 'scalar' or 'per_head' (e.g. 'per-channel', 'vector', or an unset placeholder string).","commonSituations":"Enabling the history-amplification feature experimentally with a guessed mode name; configs carried over from a fork that renamed the modes.","solutions":["Use history_scale_mode='scalar' or 'per_head' when is_amplify_history is True","If you don't need history amplification, set is_amplify_history=False so history_scale_mode is ignored","Check the Helios config shipped with the pretrained checkpoint for the trained mode value"],"exampleFix":"# before\nattn = HeliosAttention(..., is_amplify_history=True, history_scale_mode=\"vector\")\n\n# after\nattn = HeliosAttention(..., is_amplify_history=True, history_scale_mode=\"per_head\")","handlingStrategy":"validation","validationCode":"if is_amplify_history:\n    assert history_scale_mode in ('scalar', 'per_head'), history_scale_mode","typeGuard":"def is_valid_history_scale_mode(v: str) -> bool:\n    return v in ('scalar', 'per_head')","tryCatchPattern":null,"preventionTips":["Validate feature-mode strings when enabling experimental features","Copy mode names from the checkpoint config verbatim"],"tags":["config","validation","attention","diffusion"],"backgroundTag":"invalid-config-value","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}