xai-org/x-algorithm · error · NotImplementedError

Rope type {self.config.rope_type} is not supported

Error message

Rope type {self.config.rope_type} is not supported

What it means

_apply_rope dispatches on config.rope_type via a match statement; only the implemented rotary-embedding variants (the cases above the `case _`) are supported. An unrecognized rope_type falls through to `case _` and raises NotImplementedError with the config value.

Source

Thrown at phoenix/xrex/models/layers.py:379

        key_heads: jax.Array,
        *,
        positions: jax.Array | None,
    ) -> tuple[jax.Array, jax.Array]:
        t: jax.Array | None = None
        if positions is not None:
            t = positions[..., 0].astype(jnp.float32)

        match self.config.rope_type:
            case "1d":
                rotate = RotaryEmbedding(
                    dim=self.config.key_size,
                    base_exponent=self.config.rope_base_exponent,
                )
                key_heads, query_heads = map(
                    rotate, (key_heads, query_heads), (1, 1), (0, 0), (t, t)
                )
            case _:
                raise NotImplementedError(f"Rope type {self.config.rope_type} is not supported")

        return query_heads, key_heads

    @hk.transparent
    def _get_attn_impl(
        self,
        extra_attn_kwargs: dict[str, jax.Array | None],
        *,
        mask: jax.Array | None,
        seqpack_layout: SequencePackedLayout | None,
    ) -> tuple[Type[Attention], dict[str, jax.Array | None]]:
        match self.config.attn_impl:
            case "jax_attn" | "jax_attn_interleaved":
                extra_attn_kwargs["masks"] = mask
                attn_class = JaxAttention
            case "pallas_attn":
                attn_class = PallasAttention
            case "flash_attn":

View on GitHub (pinned to 24c60942c5)

Solutions

1. Set rope_type to one of the values handled in _apply_rope (inspect the match arms in layers.py around line 379).
2. Fix typos/casing in the config string.
3. If a new variant is genuinely needed, add a match arm implementing it before the fallback raise.

Example fix

# before
config.rope_type = "rope_neox"

# after
config.rope_type = "neox"

import inspect, phoenix.xrex.models.layers as L  # inspect supported rope types
# hard-code the supported set from the match arms:
assert config.rope_type in {"", "default"} or config.rope_type in SUPPORTED_ROPE_TYPES

def is_supported_rope(rt: str, supported: set[str]) -> bool:
    return rt in supported

When it happens

Trigger: Setting rope_type to an unsupported or misspelled value (e.g. 'rope_neox' vs 'neox', 'none', 'linear') in the layer config; loading a config from another repo with extra rope variants.

Common situations: Porting configs between model families whose rope_type vocabulary differs; typos after config refactors.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

• attn_logit_cap_method {method!r} is not supported by JaxAtte
• Invalid attention implementation: {self.config.attn_impl}
• Unknown loss_type: {loss_type}
• Invalid argument {arg!r}, not a key=value replacement and no
• Expected bool [True, true, False, false], got {val!r}

AI-assisted analysis of xai-org/x-algorithm@24c60942c5 (2026-08-28). Data as JSON: /api/errors/0b2467da0ee72370. Report an issue: GitHub.