{"record":{"id":"cdf8fcfc2bd05e1c","repo":"xai-org/x-algorithm","slug":"cap-method-must-be-in-tanh-soft-sign-cdf8fc","errorCode":null,"errorMessage":"cap_method must be in [tanh, soft_sign]","messagePattern":"cap_method must be in \\[tanh, soft_sign\\]","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"phoenix/xrex/pallas/ranker_attention_varlen.py","lineNumber":795,"sourceCode":"            segment_ref,\n            (pl.dslice(offset_k, block_k),),\n            mask=k_mask,\n            other=PADDING_SEGMENT_ID,\n        ).astype(jnp.int8)[None, :]\n        qk = pl.dot(q, k, trans_b=True)\n\n        if sm_scale != 1.0:\n            qk *= sm_scale\n\n        if cap > 0.0:\n            if cap_method == \"tanh\":\n                qk_tanh = tanh(qk / cap)\n                qk = cap * qk_tanh\n            elif cap_method == \"soft_sign\":\n                soft_sign = 1.0 / (1.0 + jnp.abs(qk) / cap)\n                qk = qk * soft_sign\n            else:\n                raise ValueError(\"cap_method must be in [tanh, soft_sign]\")\n\n        mask = jnp.logical_or(seg_k > 0, span_q[:, None] == span_k[None, :])\n        mask = jnp.logical_and(seq_q_is_not_padding, mask)\n        mask = jnp.logical_and(mask, k_mask[None, :])\n        if causal:\n            causal_mask = span_q[:, None] >= span_k[None, :]\n            mask = jnp.logical_and(mask, causal_mask)\n\n        p = jnp.exp(qk - m[:, None])\n        p = jnp.where(mask, p, 0.0)\n        dp = pl.dot(do, v, trans_b=True).astype(jnp.float32) - di[:, None]\n        ds = p * dp\n\n        if z_loss_weight > 0:\n            ds += z_loss_weight * p * ((jnp.log(l + 1e-12) + m) / l)[:, None]\n\n        if cap > 0.0:\n            if cap_method == \"tanh\":","sourceCodeStart":777,"sourceCodeEnd":813,"githubUrl":"https://github.com/xai-org/x-algorithm/blob/24c60942c5c5fdad3a6addffb4c6e6d2f228f04f/phoenix/xrex/pallas/ranker_attention_varlen.py#L777-L813","documentation":"Raised in the dq backward inner loop of the varlen ranker attention kernel when cap > 0.0 and cap_method is invalid. The forward capping options are the only ones with implemented derivatives, so the backward path guards on the same set of values.","triggerScenarios":"Running gradients through varlen ranker attention with cap > 0.0 and cap_method other than 'tanh'/'soft_sign'.","commonSituations":"Bad cap_method only exercised during training; config drift between train and eval paths; sweep jobs hitting the gradient path for the first time.","solutions":["Set cap_method to 'tanh' or 'soft_sign'.","Set cap=0.0 to skip capping entirely.","Fail fast by validating cap_method at startup in training scripts."],"exampleFix":"# before\ngrad = jax.grad(loss)(..., cap=50.0, cap_method='softmax_cap')\n# after\ngrad = jax.grad(loss)(..., cap=50.0, cap_method='soft_sign')","handlingStrategy":"validation","validationCode":"assert cap <= 0.0 or cap_method in ('tanh', 'soft_sign')","typeGuard":"def valid_cap_for_grad(cap: float, cap_method: str) -> bool:\n    return cap <= 0.0 or cap_method in ('tanh', 'soft_sign')","tryCatchPattern":null,"preventionTips":["Run a tiny grad smoke test on kernel configs before long training runs.","Keep train and eval kernel configs in one shared, validated dataclass."],"tags":["pallas","jax","autodiff","attention","backward-pass"],"backgroundTag":"invalid-enum-value","analyzedSha":"24c60942c5c5fdad3a6addffb4c6e6d2f228f04f","analyzedAt":"2026-08-28T11:40:14.686Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}