HKUDS/Vibe-Trading · error · ValueError

CSCV ranks strategies against each other and needs at least

Error message

CSCV ranks strategies against each other and needs at least 2, got {n_strategies}

What it means

CSCV ranks strategies against each other within each split, so probability_of_backtest_overfitting needs at least 2 strategy columns to form a cross-sectional rank. With a single strategy there is no 'best of N' selection and the probability of backtest overfitting is not defined, hence the explicit refusal.

Source

Thrown at agent/src/quantlib/multipletesting.py:487

        A :class:`CSCVResult`.

    Raises:
        ValueError: If ``n_splits`` is odd or below 4, if fewer than 2
            strategies are supplied (a rank needs competitors), if the sample
            cannot give each subset at least 2 rows, or if every strategy has
            zero variance so no Sharpe is defined.
    """
    if n_splits < 4 or n_splits % 2 != 0:
        raise ValueError(f"n_splits must be an even number >= 4, got {n_splits}")

    frame = pd.DataFrame(performance)
    matrix = frame.to_numpy(dtype=float)
    if matrix.ndim != 2:
        raise ValueError(f"performance must be 2-D, got shape {matrix.shape}")

    n_rows, n_strategies = matrix.shape
    if n_strategies < 2:
        raise ValueError(
            f"CSCV ranks strategies against each other and needs at least 2, "
            f"got {n_strategies}"
        )

    subset_size = n_rows // n_splits
    if subset_size < 2:
        raise ValueError(
            f"{n_rows} rows split {n_splits} ways gives {subset_size} row(s) per "
            "subset; each subset needs at least 2 for a Sharpe"
        )

    used_rows = subset_size * n_splits
    dropped = n_rows - used_rows
    trimmed = matrix[:used_rows]
    subsets = [
        trimmed[i * subset_size : (i + 1) * subset_size] for i in range(n_splits)
    ]

View on GitHub (pinned to 80ffdda44c)

Solutions

1. Feed all candidate strategies from the trial grid, not just the winner.
2. Check matrix.shape[1] >= 2 before calling.
3. If you genuinely have one strategy, skip PBO — it measures selection overfitting across trials.

Example fix

# before
pbo = probability_of_backtest_overfitting(perf[:, :1], n_splits=16)

# after
assert perf.shape[1] >= 2, 'CSCV needs competing strategies'
pbo = probability_of_backtest_overfitting(perf, n_splits=16)

assert np.asarray(performance).shape[1] >= 2, 'CSCV needs >= 2 strategies'

def has_enough_strategies(p, minimum: int = 2) -> bool:
    return np.asarray(p).ndim == 2 and np.asarray(p).shape[1] >= minimum

try:
    pbo = probability_of_backtest_overfitting(perf, n_splits)
except ValueError as e:
    if 'at least 2' in str(e):
        skip_pbo('single-strategy run')
    else:
        raise

When it happens

Trigger: Passing a performance matrix with exactly one column, e.g. shape (1000, 1), or a one-column DataFrame / list of one series.

Common situations: Prototyping the PBO pipeline with a single candidate strategy; a column-selection bug upstream that accidentally slices to one column; a grid search that filters candidates before feeding CSCV and leaves a sole survivor.

Related errors

• n_splits must be an even number >= 4, got {n_splits}
• performance must be 2-D, got shape {matrix.shape}
• {n_rows} rows split {n_splits} ways gives {subset_size} row(
• no split produced a usable Sharpe; every strategy may have z
• invalid alpha_id

AI-assisted analysis of HKUDS/Vibe-Trading@80ffdda44c (2026-08-28). Data as JSON: /api/errors/b3cf57646bf5d5e7. Report an issue: GitHub.