xai-org/x-algorithm · error · ValueError

{name}: delay rows {mat.shape[0]} != batch rows {batch.num_r

Error message

{name}: delay rows {mat.shape[0]} != batch rows {batch.num_rows}

What it means

attach_delays() appends each delay column from the sidecar to a RecordBatch, first asserting row-count alignment: the sidecar matrix must have exactly as many rows as the batch. A mismatch means the sidecar was generated for a different version of the batch file (different row count), so delays cannot be attached.

Source

Thrown at phoenix/xrex/data/conversion_labels.py:77

        raise ValueError(
            f"sidecar {sidecar_path} missing column(s) {missing}; available: {schema_names}"
        )
    t = pq.read_table(sidecar_path, columns=columns)
    out: dict[str, np.ndarray] = {}
    for name in columns:
        col = t.column(name).combine_chunks()
        if isinstance(col, pa.ChunkedArray):
            col = col.chunk(0)
        seq_len = col.type.list_size
        flat = col.flatten().to_numpy(zero_copy_only=False).astype(np.int64)
        out[name] = flat.reshape(len(col), seq_len)
    return out


def attach_delays(batch: pa.RecordBatch, delays: dict[str, np.ndarray]) -> pa.RecordBatch:
    for name, mat in delays.items():
        if mat.shape[0] != batch.num_rows:
            raise ValueError(f"{name}: delay rows {mat.shape[0]} != batch rows {batch.num_rows}")
        arr = pa.FixedSizeListArray.from_arrays(
            pa.array(mat.reshape(-1), type=pa.int64()), mat.shape[1]
        )
        batch = batch.append_column(name, arr)
    return batch


def fold_action_delays_into_multihot(batch: pa.RecordBatch, window_ms: int) -> pa.RecordBatch:
    action_cols = [n for n in batch.schema.names if n.startswith(ACTION_DELAY_PREFIX)]
    if not action_cols:
        return batch
    col_idx = batch.schema.get_field_index(ACTION_MULTIHOT_COLUMN)
    if col_idx < 0:
        raise ValueError(f"batch missing {ACTION_MULTIHOT_COLUMN}")
    mh = batch.column(col_idx)
    seq_len = mh.type.list_size
    vocab = mh.type.value_type.list_size
    bits = (

View on GitHub (pinned to 24c60942c5)

Solutions

1. Regenerate the conversion-label sidecar from the current batch file so row counts match.
2. Delete stale sidecars and rerun the labeling pipeline for the affected partition.
3. Skip conversion-label columns for the corrupted batch until it is relabeled.

Example fix

// before
batch2 = attach_delays(batch, stale_delays)  # ValueError: rows 1000 != 1200

// after
stale_delays = rerun_labeling(batch_path)  # regenerate sidecar
batch2 = attach_delays(batch, stale_delays)

if any(m.shape[0] != batch.num_rows for m in delays.values()):
    raise SystemExit('stale sidecar: regenerate labels before attach')

try:
    batch = attach_delays(batch, delays)
except ValueError as e:
    logger.warning('row mismatch, skipping delay attach: %s', e)
    # fall back to batch without delay columns

When it happens

Trigger: Sidecar regenerated after the batch parquet was rewritten with a different row count; batch truncated/rewritten by a reprocessing job while sidecar stayed stale; reading a batch and sidecar from different runs.

Common situations: Partially re-processed topics where batch files were replaced but labels were not; race between writer jobs producing batches and label jobs.

Related errors

• sidecar {sidecar} column {name} has {mat.shape[0]} rows, bat
• not a partition batch file path: {batch_file_path}
• not a parquet path: {batch_file_path}
• sidecar {sidecar_path} missing column(s) {missing}; availabl
• batch missing {ACTION_MULTIHOT_COLUMN}

AI-assisted analysis of xai-org/x-algorithm@24c60942c5 (2026-08-28). Data as JSON: /api/errors/f03354d0e959fef8. Report an issue: GitHub.