HKUDS/Vibe-Trading · error · ValueError

{n_rows} rows split {n_splits} ways gives {subset_size} row(

Error message

{n_rows} rows split {n_splits} ways gives {subset_size} row(s) per subset; each subset needs at least 2 for a Sharpe

What it means

Each CSCV subset must contain at least 2 rows for a Sharpe ratio (a standard deviation needs n >= 2) to exist. If n_rows // n_splits < 2 the per-subset Sharpe would be undefined, so probability_of_backtest_overfitting raises this error and tells you the actual arithmetic (rows, splits, subset size).

Source

Thrown at agent/src/quantlib/multipletesting.py:494

    """
    if n_splits < 4 or n_splits % 2 != 0:
        raise ValueError(f"n_splits must be an even number >= 4, got {n_splits}")

    frame = pd.DataFrame(performance)
    matrix = frame.to_numpy(dtype=float)
    if matrix.ndim != 2:
        raise ValueError(f"performance must be 2-D, got shape {matrix.shape}")

    n_rows, n_strategies = matrix.shape
    if n_strategies < 2:
        raise ValueError(
            f"CSCV ranks strategies against each other and needs at least 2, "
            f"got {n_strategies}"
        )

    subset_size = n_rows // n_splits
    if subset_size < 2:
        raise ValueError(
            f"{n_rows} rows split {n_splits} ways gives {subset_size} row(s) per "
            "subset; each subset needs at least 2 for a Sharpe"
        )

    used_rows = subset_size * n_splits
    dropped = n_rows - used_rows
    trimmed = matrix[:used_rows]
    subsets = [
        trimmed[i * subset_size : (i + 1) * subset_size] for i in range(n_splits)
    ]

    logits: list[float] = []
    in_sample_sharpes: list[float] = []
    out_sample_sharpes: list[float] = []
    all_indices = set(range(n_splits))

    for chosen in combinations(range(n_splits), n_splits // 2):
        rest = sorted(all_indices - set(chosen))

View on GitHub (pinned to 80ffdda44c)

Solutions

1. Lower n_splits (e.g. 16 -> 8 -> 4) until n_rows // n_splits >= 2, preferably much larger for stable Sharpes.
2. Extend the sample: more history or finer time resolution so each block has many rows.
3. Validate the ratio up front: assert len(performance) // n_splits >= 2.

Example fix

# before
pbo = probability_of_backtest_overfitting(perf_60rows, n_splits=16)  # raises

# after
n = len(perf_60rows)
n_splits = max(4, min(16, n // 20))  # >=20 rows per block if possible
pbo = probability_of_backtest_overfitting(perf_60rows, n_splits=n_splits)

n_rows = np.asarray(performance).shape[0]
assert n_rows // n_splits >= 2, f'need >= {2 * n_splits} rows for {n_splits} splits'

def rows_support_splits(n_rows: int, n_splits: int) -> bool:
    return n_rows // n_splits >= 2

try:
    pbo = probability_of_backtest_overfitting(perf, n_splits)
except ValueError as e:
    if 'per subset' in str(e):
        pbo = probability_of_backtest_overfitting(perf, n_splits=max(4, (len(perf) // 2) - (len(perf) // 2) % 2))
    else:
        raise

When it happens

Trigger: Passing e.g. 100 rows with n_splits=64 (subset_size=1), or any combination where n_rows < 2 * n_splits.

Common situations: Short backtests (a few dozen daily returns) combined with the paper-default 16 or more splits; intraday data where the user thinks in trades but supplies rows of aggregated returns; raising n_splits hoping for finer PBO resolution on a fixed sample.

Related errors

• n_splits must be an even number >= 4, got {n_splits}
• performance must be 2-D, got shape {matrix.shape}
• CSCV ranks strategies against each other and needs at least
• no split produced a usable Sharpe; every strategy may have z

AI-assisted analysis of HKUDS/Vibe-Trading@80ffdda44c (2026-08-28). Data as JSON: /api/errors/05fe827e2f95abe7. Report an issue: GitHub.