xai-org/x-algorithm · error · NotImplementedError

Unknown remat policy: {policy}

Error message

Unknown remat policy: {policy}

What it means

Raised by custom_remat_policy when the requested gradient checkpointing (rematerialization) policy string does not match any policy known to phoenix.xrex. The function only supports a fixed set of named policies (e.g. everything-save/specific named activations like query_heads, key_heads, value_heads, scalar_stats); anything else falls through to NotImplementedError.

Source

Thrown at phoenix/xrex/models/remat.py:32

    if policy == RematType.WHOLE:
        return jax.checkpoint_policies.save_only_these_names(
            "scalar_stats",
        )
    elif policy == RematType.SAVE_GB300_RECSYS:
        return jax.checkpoint_policies.save_only_these_names(
            "attn_outputs",
            "dense_outputs",
            "dense_outputs_individual",
            "attn",
            "gate_up_proj",
            "dense_up_proj",
            "query_heads",
            "key_heads",
            "value_heads",
            "scalar_stats",
        )
    else:
        raise NotImplementedError(f"Unknown remat policy: {policy}")

View on GitHub (pinned to 24c60942c5)

Solutions

1. Check the policy whitelist at the top of phoenix/xrex/models/remat.py and correct the config string to exactly one of the supported names
2. If you need a new policy, add a branch in custom_remat_policy that maps the name to a jax.ad_checkpoint.checkpoint policy
3. As a workaround use the default 'everything_save' style policy by omitting the custom policy setting

Example fix

# before
model_config.remat_policy = "everything_sav"
# after
model_config.remat_policy = "everything_save"

from phoenix.xrex.models import remat as remat_mod
SUPPORTED = get_supported_policies()  # e.g. names referenced in custom_remat_policy
assert cfg.remat_policy in SUPPORTED, f"bad remat policy {cfg.remat_policy!r}"

def is_valid_remat_policy(p: str) -> bool:
    return p in ("everything_save", "nothing_save", "query_heads", "key_heads", "value_heads", "scalar_stats")

When it happens

Trigger: Calling a model (the policy is invoked via __call__) with a config value for the remat policy that is misspelled (e.g. 'everthing_save'), uses different casing, or refers to a policy introduced/renamed in another phoenix/xrex version.

Common situations: Copy-pasted YAML/JSON configs from older examples or other repos; upgrading xrex where policy names were renamed; custom layers expecting a new policy not yet implemented locally.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

• Please override this method for specific attention impl.
• {namespace} is already exist in ShardingContext {self.name},
• Unknown named dimension: {named_dim} for namespace: {namespa
• unknown optim={optim!r}
• use_async_emb requires an embedding optimizer implementing A

AI-assisted analysis of xai-org/x-algorithm@24c60942c5 (2026-08-28). Data as JSON: /api/errors/43b6ed4c4b8d2eaa. Report an issue: GitHub.