{"record":{"id":"bf7098ee8a39bcbe","repo":"xai-org/x-algorithm","slug":"fa2-has-been-removed-but-self-config-attn-impl","errorCode":null,"errorMessage":"FA2 has been removed but {self.config.attn_impl=} requires it","messagePattern":"FA2 has been removed but (.+?) requires it","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"phoenix/xrex/models/layers.py","lineNumber":294,"sourceCode":"            wd_mask = self.scale_config.ln_weight_decay_mask\n            query_heads = norm_fn(\n                query_heads, \"q_norm\", qk_lr, create_scale=True, weight_decay_mask=wd_mask\n            )\n            key_heads = norm_fn(\n                key_heads, \"k_norm\", qk_lr, create_scale=True, weight_decay_mask=wd_mask\n            )\n\n        if add_rope:\n            query_heads, key_heads = self._apply_rope(\n                query_heads,\n                key_heads,\n                positions=positions,\n            )\n            query_heads = checkpoint_name(query_heads, \"query_heads_rope\")\n            key_heads = checkpoint_name(key_heads, \"key_heads_rope\")\n\n        if self.config.attn_impl == \"recsys_flash_attn\":\n            raise NotImplementedError(\n                f\"FA2 has been removed but {self.config.attn_impl=} requires it\"\n            )\n\n        extra_attn_kwargs: dict[str, jax.Array | None] = {}\n\n        temp = jnp.ones(segment_ids.shape, dtype=jnp.int32)\n\n        attn_class, extra_attn_kwargs = self._get_attn_impl(\n            extra_attn_kwargs,\n            mask=mask,\n            seqpack_layout=seqpack_layout,\n        )\n        tmp = attn_class(\n            self.config,\n            self.scale_config,\n            sharding_context=self.sharding_context,\n        )(\n            query_heads,","sourceCodeStart":276,"sourceCodeEnd":312,"githubUrl":"https://github.com/xai-org/x-algorithm/blob/24c60942c5c5fdad3a6addffb4c6e6d2f228f04f/phoenix/xrex/models/layers.py#L276-L312","documentation":"The attention layer's __call__ explicitly blocks attn_impl='recsys_flash_attn': FlashAttention-2 support has been removed from the tree, and this impl required it, so the config is rejected at forward time with a NotImplementedError naming the offending attn_impl.","triggerScenarios":"Loading or running a model whose config still has attn_impl='recsys_flash_attn'; older checkpoints/configs created before the FA2 removal.","commonSituations":"Resuming from a legacy checkpoint or YAML config after upgrading to a version that dropped FA2; configs inherited from an older experiment.","solutions":["Update the model config to a supported attn_impl such as 'jax_attn', 'pallas_attn', or 'flash_attn' with fa_version=3.","Regenerate or edit the saved config rather than reusing the one embedded in the old checkpoint.","Add a config-migration check that rewrites legacy impl names on load."],"exampleFix":"# before\nattn_impl: recsys_flash_attn\n\n# after\nattn_impl: jax_attn","handlingStrategy":"validation","validationCode":"if config.attn_impl == \"recsys_flash_attn\":\n    raise SystemExit(\"FA2-based impl removed; update config to 'jax_attn' or 'flash_attn' with fa_version=3\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Run a config lint pass over checkpoints/configs after upgrading the framework.","Keep a migration table of removed attn_impl names."],"tags":["attention","fa2-removed","legacy-config","migration"],"backgroundTag":"removed-feature-legacy-config","analyzedSha":"24c60942c5c5fdad3a6addffb4c6e6d2f228f04f","analyzedAt":"2026-08-28T11:40:14.686Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}