xai-org/x-algorithm · error · ValueError

metric_mask_keys entries

Error message

metric_mask_keys entries {sorted(unknown)} not in available masks {sorted(masks)}

What it means

build_metric_masks constructs a fixed dictionary of evaluation masks (e.g. android_mask, negative-sample-complement 'android_non_negative', etc.). When metric_mask_keys is provided it is treated as a keep-filter, and any key not present in the built masks is reported with both the unknown keys and the full list of available masks.

Solutions

Use only mask names from the 'available masks' list printed in the error.
Remove stale keys from metric_mask_keys after mask renames/removals.
If a mask is genuinely missing (e.g. an ios counterpart), add its computation in build_metric_masks.

Example fix

# before
metric_mask_keys: ["ios_non_negative", "android_non_negative"]

# after
metric_mask_keys: ["android_non_negative"]

Defensive patterns

Strategy: validation

Validate before calling

from phoenix.xrex.models.recsys_model import build_metric_masks  # dry-run to enumerate masks
available = set(build_metric_masks(batch, metric_mask_keys=[]))
unknown = set(metric_mask_keys) - available
assert not unknown, f"unknown mask keys: {sorted(unknown)}"

Prevention

Dry-run build_metric_masks with an empty keep-list to enumerate valid keys at startup.
Add a config lint for metric_mask_keys after mask renames.

When it happens

Trigger: Passing metric_mask_keys=["ios_non_negative", "android"] when only the built mask names exist; referencing a mask removed in a refactor during init, checkpoint finalization, or _build_metric_masks.

Common situations: Configs referencing masks from an older version; renaming of mask keys; typos in key lists.

Related errors

AI-assisted analysis of xai-org/x-algorithm@24c60942c5 (2026-08-28). Data as JSON: /api/errors/4e97fc46bdea60b4. Report an issue: GitHub.

Appendix: source

Thrown at phoenix/xrex/models/recsys_model.py:1405

    if condition_search_relevance_on_prompt:
        prompt_mask = jnp.any(
            raw_targets[:, :, jnp.array(SEARCH_RELEVANCE_ACTION_INDICES)] == 1, axis=-1
        ).astype(mask.dtype)
        masks["prompted"] = mask * prompt_mask
        masks["non_negative_prompted"] = non_negative_mask * prompt_mask

    if enable_platform_metrics and ios_mask is not None and android_mask is not None:
        masks["ios"] = ios_mask
        masks["android"] = android_mask
        masks["ios_non_negative"] = ios_mask * (1 - negative_sample_mask)
        masks["android_non_negative"] = android_mask * (1 - negative_sample_mask)

    if metric_mask_keys:
        keep = set(metric_mask_keys)
        unknown = keep - masks.keys()
        if unknown:
            raise ValueError(
                f"metric_mask_keys entries {sorted(unknown)} not in available masks {sorted(masks)}"
            )
        masks = {k: v for k, v in masks.items() if k in keep}

    return masks


@dataclass
class RecsysAggregatedModel(hk.Module):
    config: RecsysAggregatedModelConfig
    model: Transformer
    sharding_context: ShardingContext

    @property
    def data_axis(self):
        return ("stage", *self.config.model_config.data_axis)

    def _compute_metrics_after_masks(

View on GitHub (pinned to 24c60942c5)