xai-org/x-algorithm · error · ValueError

batch missing newEventMask; conversion-label folding is…

Error message

batch missing newEventMask; conversion-label folding is candidate-only

What it means

When folding conversion labels, the function needs a candidate mask to restrict folding to candidate positions. It looks for 'newEventMask' or 'newEventMaskSeq' in the batch schema; if neither exists it raises ValueError stating folding is candidate-only. Without the mask there is no safe way to decide which sequence positions may receive labels.

Solutions

1. Include newEventMask (or newEventMaskSeq) in the batch schema before folding.
2. Regenerate the dataset with a pipeline version that emits the mask.
3. Disable action-delay folding until the mask is available.

Example fix

// before
batch = fold_action_delays_into_multihot(batch_no_mask, window_ms)  # ValueError

// after
batch = fold_action_delays_into_multihot(batch_with_newEventMask, window_ms)

mask_name = next((n for n in ('newEventMask','newEventMaskSeq') if n in batch.schema.names), None)
if mask_name is None:
    disable_action_delay_folding()  # config-driven fallback

def has_candidate_mask(batch) -> bool:
    return any(n in batch.schema.names for n in ('newEventMask', 'newEventMaskSeq'))

try:
    batch = fold_action_delays_into_multihot(batch, w)
except ValueError as e:
    if 'newEventMask' in str(e):
        logger.warning('skipping fold: %s', e)
    else:
        raise

When it happens

Trigger: Producer consuming batches from a pipeline version that predates newEventMask; upstream feature selection excluding both mask columns; delay columns attached to a hand-built RecordBatch without the mask.

Common situations: Schema drift after upgrading one pipeline stage but not another; datasets exported before the mask feature existed.

Related errors

• batch missing
• sidecar missing column(s) ; available
• failed to serialize ServedHistory
• : seq len != multi-hot
• : bit outside vocab

AI-assisted analysis of xai-org/x-algorithm@24c60942c5 (2026-08-28). Data as JSON: /api/errors/c98c5e18c17f0871. Report an issue: GitHub.

Appendix: source

Thrown at phoenix/xrex/data/conversion_labels.py:107

    col_idx = batch.schema.get_field_index(ACTION_MULTIHOT_COLUMN)
    if col_idx < 0:
        raise ValueError(f"batch missing {ACTION_MULTIHOT_COLUMN}")
    mh = batch.column(col_idx)
    seq_len = mh.type.list_size
    vocab = mh.type.value_type.list_size
    bits = (
        mh.flatten()
        .flatten()
        .to_numpy(zero_copy_only=False)
        .astype(np.bool_)
        .reshape(batch.num_rows, seq_len, vocab)
        .copy()
    )
    mask_name = next(
        (n for n in ("newEventMask", "newEventMaskSeq") if n in batch.schema.names), None
    )
    if mask_name is None:
        raise ValueError("batch missing newEventMask; conversion-label folding is candidate-only")
    mask_col = batch.column(mask_name)
    is_candidate = (
        mask_col.flatten()
        .to_numpy(zero_copy_only=False)
        .astype(np.bool_)
        .reshape(batch.num_rows, mask_col.type.list_size)
    )
    if is_candidate.shape[1] != seq_len:
        raise ValueError(f"{mask_name}: seq len {is_candidate.shape[1]} != multi-hot {seq_len}")
    for name in action_cols:
        idx = action_index_of(name)
        if idx >= vocab:
            raise ValueError(f"{name}: bit {idx} outside vocab {vocab}")
        delays_col = batch.column(name)
        delays = (
            delays_col.flatten()
            .to_numpy(zero_copy_only=False)
            .astype(np.int64)

View on GitHub (pinned to 24c60942c5)