xai-org/x-algorithm · error · ValueError

batch missing newEventMask; conversion-label folding is cand

Error message

batch missing newEventMask; conversion-label folding is candidate-only

What it means

When folding conversion labels, the function needs a candidate mask to restrict folding to candidate positions. It looks for 'newEventMask' or 'newEventMaskSeq' in the batch schema; if neither exists it raises ValueError stating folding is candidate-only. Without the mask there is no safe way to decide which sequence positions may receive labels.

Source

Thrown at phoenix/xrex/data/conversion_labels.py:107

    col_idx = batch.schema.get_field_index(ACTION_MULTIHOT_COLUMN)
    if col_idx < 0:
        raise ValueError(f"batch missing {ACTION_MULTIHOT_COLUMN}")
    mh = batch.column(col_idx)
    seq_len = mh.type.list_size
    vocab = mh.type.value_type.list_size
    bits = (
        mh.flatten()
        .flatten()
        .to_numpy(zero_copy_only=False)
        .astype(np.bool_)
        .reshape(batch.num_rows, seq_len, vocab)
        .copy()
    )
    mask_name = next(
        (n for n in ("newEventMask", "newEventMaskSeq") if n in batch.schema.names), None
    )
    if mask_name is None:
        raise ValueError("batch missing newEventMask; conversion-label folding is candidate-only")
    mask_col = batch.column(mask_name)
    is_candidate = (
        mask_col.flatten()
        .to_numpy(zero_copy_only=False)
        .astype(np.bool_)
        .reshape(batch.num_rows, mask_col.type.list_size)
    )
    if is_candidate.shape[1] != seq_len:
        raise ValueError(f"{mask_name}: seq len {is_candidate.shape[1]} != multi-hot {seq_len}")
    for name in action_cols:
        idx = action_index_of(name)
        if idx >= vocab:
            raise ValueError(f"{name}: bit {idx} outside vocab {vocab}")
        delays_col = batch.column(name)
        delays = (
            delays_col.flatten()
            .to_numpy(zero_copy_only=False)
            .astype(np.int64)

View on GitHub (pinned to 24c60942c5)

Solutions

1. Include newEventMask (or newEventMaskSeq) in the batch schema before folding.
2. Regenerate the dataset with a pipeline version that emits the mask.
3. Disable action-delay folding until the mask is available.

Example fix

// before
batch = fold_action_delays_into_multihot(batch_no_mask, window_ms)  # ValueError

// after
batch = fold_action_delays_into_multihot(batch_with_newEventMask, window_ms)

mask_name = next((n for n in ('newEventMask','newEventMaskSeq') if n in batch.schema.names), None)
if mask_name is None:
    disable_action_delay_folding()  # config-driven fallback

def has_candidate_mask(batch) -> bool:
    return any(n in batch.schema.names for n in ('newEventMask', 'newEventMaskSeq'))

try:
    batch = fold_action_delays_into_multihot(batch, w)
except ValueError as e:
    if 'newEventMask' in str(e):
        logger.warning('skipping fold: %s', e)
    else:
        raise

When it happens

Trigger: Producer consuming batches from a pipeline version that predates newEventMask; upstream feature selection excluding both mask columns; delay columns attached to a hand-built RecordBatch without the mask.

Common situations: Schema drift after upgrading one pipeline stage but not another; datasets exported before the mask feature existed.

Related errors

• sidecar {sidecar_path} missing column(s) {missing}; availabl
• batch missing {ACTION_MULTIHOT_COLUMN}
• not a partition batch file path: {batch_file_path}
• not a parquet path: {batch_file_path}
• {name}: delay rows {mat.shape[0]} != batch rows {batch.num_r

AI-assisted analysis of xai-org/x-algorithm@24c60942c5 (2026-08-28). Data as JSON: /api/errors/c98c5e18c17f0871. Report an issue: GitHub.