HKUDS/Vibe-Trading · error · ValueError

CSCV ranks strategies against each other and needs at least…

Error message

CSCV ranks strategies against each other and needs at least 2, got {n_strategies}

What it means

CSCV ranks strategies against each other within each split, so probability_of_backtest_overfitting needs at least 2 strategy columns to form a cross-sectional rank. With a single strategy there is no 'best of N' selection and the probability of backtest overfitting is not defined, hence the explicit refusal.

Solutions

1. Feed all candidate strategies from the trial grid, not just the winner.
2. Check matrix.shape[1] >= 2 before calling.
3. If you genuinely have one strategy, skip PBO — it measures selection overfitting across trials.

Example fix

# before
pbo = probability_of_backtest_overfitting(perf[:, :1], n_splits=16)

# after
assert perf.shape[1] >= 2, 'CSCV needs competing strategies'
pbo = probability_of_backtest_overfitting(perf, n_splits=16)

assert np.asarray(performance).shape[1] >= 2, 'CSCV needs >= 2 strategies'

def has_enough_strategies(p, minimum: int = 2) -> bool:
    return np.asarray(p).ndim == 2 and np.asarray(p).shape[1] >= minimum

try:
    pbo = probability_of_backtest_overfitting(perf, n_splits)
except ValueError as e:
    if 'at least 2' in str(e):
        skip_pbo('single-strategy run')
    else:
        raise

When it happens

Trigger: Passing a performance matrix with exactly one column, e.g. shape (1000, 1), or a one-column DataFrame / list of one series.

Common situations: Prototyping the PBO pipeline with a single candidate strategy; a column-selection bug upstream that accidentally slices to one column; a grid search that filters candidates before feeding CSCV and leaves a sole survivor.

Related errors

• n_splits must be an even number >= 4, got
• rows split ways gives row(s) per subset; each subset needs…
• no split produced a usable Sharpe; every strategy may have…
• performance must be 2-D, got shape
• alpha_id and zoo are mutually exclusive

AI-assisted analysis of HKUDS/Vibe-Trading@80ffdda44c (2026-08-28). Data as JSON: /api/errors/b3cf57646bf5d5e7. Report an issue: GitHub.

Appendix: source

Thrown at agent/src/quantlib/multipletesting.py:487

        A :class:`CSCVResult`.

    Raises:
        ValueError: If ``n_splits`` is odd or below 4, if fewer than 2
            strategies are supplied (a rank needs competitors), if the sample
            cannot give each subset at least 2 rows, or if every strategy has
            zero variance so no Sharpe is defined.
    """
    if n_splits < 4 or n_splits % 2 != 0:
        raise ValueError(f"n_splits must be an even number >= 4, got {n_splits}")

    frame = pd.DataFrame(performance)
    matrix = frame.to_numpy(dtype=float)
    if matrix.ndim != 2:
        raise ValueError(f"performance must be 2-D, got shape {matrix.shape}")

    n_rows, n_strategies = matrix.shape
    if n_strategies < 2:
        raise ValueError(
            f"CSCV ranks strategies against each other and needs at least 2, "
            f"got {n_strategies}"
        )

    subset_size = n_rows // n_splits
    if subset_size < 2:
        raise ValueError(
            f"{n_rows} rows split {n_splits} ways gives {subset_size} row(s) per "
            "subset; each subset needs at least 2 for a Sharpe"
        )

    used_rows = subset_size * n_splits
    dropped = n_rows - used_rows
    trimmed = matrix[:used_rows]
    subsets = [
        trimmed[i * subset_size : (i + 1) * subset_size] for i in range(n_splits)
    ]

View on GitHub (pinned to 80ffdda44c)