xai-org/x-algorithm · error · ValueError

{name}: bit {idx} outside vocab {vocab}

Error message

{name}: bit {idx} outside vocab {vocab}

What it means

Each action-delay column encodes an action index parsed from its name via action_index_of(). That index must be a valid bit position within the multi-hot vocabulary size (value_type.list_size). If idx >= vocab, writing bits[:,:,:,idx] would be out of bounds, so ValueError is raised.

Source

Thrown at phoenix/xrex/data/conversion_labels.py:120

    )
    mask_name = next(
        (n for n in ("newEventMask", "newEventMaskSeq") if n in batch.schema.names), None
    )
    if mask_name is None:
        raise ValueError("batch missing newEventMask; conversion-label folding is candidate-only")
    mask_col = batch.column(mask_name)
    is_candidate = (
        mask_col.flatten()
        .to_numpy(zero_copy_only=False)
        .astype(np.bool_)
        .reshape(batch.num_rows, mask_col.type.list_size)
    )
    if is_candidate.shape[1] != seq_len:
        raise ValueError(f"{mask_name}: seq len {is_candidate.shape[1]} != multi-hot {seq_len}")
    for name in action_cols:
        idx = action_index_of(name)
        if idx >= vocab:
            raise ValueError(f"{name}: bit {idx} outside vocab {vocab}")
        delays_col = batch.column(name)
        delays = (
            delays_col.flatten()
            .to_numpy(zero_copy_only=False)
            .astype(np.int64)
            .reshape(batch.num_rows, delays_col.type.list_size)
        )
        if delays.shape[1] != seq_len:
            raise ValueError(f"{name}: seq len {delays.shape[1]} != multi-hot {seq_len}")
        bits[:, :, idx] = np.where(
            is_candidate, delays_to_labels(delays, window_ms), bits[:, :, idx]
        )
    inner = pa.FixedSizeListArray.from_arrays(pa.array(bits.reshape(-1)), vocab)
    outer = pa.FixedSizeListArray.from_arrays(inner, seq_len)
    return batch.set_column(col_idx, batch.schema.field(col_idx), outer)


def delays_to_labels(delays: np.ndarray, window_ms: int) -> np.ndarray:

View on GitHub (pinned to 24c60942c5)

Solutions

1. Regenerate the multi-hot column with a vocabulary at least as large as the largest action index.
2. Drop action-delay columns whose index exceeds the current vocab.
3. Regenerate sidecars consistent with the current action vocabulary.

Example fix

// before: action_delay_40 but vocab 32
fold_action_delays_into_multihot(batch, w)  # ValueError

// after: vocab 64 multihot (or drop col)
action_cols = [c for c in cols if action_index_of(c) < vocab]
fold_action_delays_into_multihot(batch_with_v64, w)

from phoenix.xrex.data import conversion_labels as cl
vocab = mh_type.value_type.list_size
bad = [n for n in action_cols if cl.action_index_of(n) >= vocab]
if bad: raise SystemExit(f'action columns outside vocab: {bad}')

try:
    batch = fold_action_delays_into_multihot(batch, w)
except ValueError as e:
    if 'outside vocab' in str(e):
        drop_offending_columns_and_retry()
    else:
        raise

When it happens

Trigger: Sidecar/dataset generated with a larger action vocabulary (e.g. 64 actions) but the batch's multi-hot value_type only has list_size 32; column named with a stale action index after vocab pruning.

Common situations: Vocabulary size changed between the labeling job and the feature pipeline; reusing old delay columns against a new multi-hot schema.

Related errors

• not a partition batch file path: {batch_file_path}
• not a parquet path: {batch_file_path}
• sidecar {sidecar_path} missing column(s) {missing}; availabl
• {name}: delay rows {mat.shape[0]} != batch rows {batch.num_r
• batch missing {ACTION_MULTIHOT_COLUMN}

AI-assisted analysis of xai-org/x-algorithm@24c60942c5 (2026-08-28). Data as JSON: /api/errors/cad9292dabe72103. Report an issue: GitHub.