HKUDS/Vibe-Trading · error · ValueError

no split produced a usable Sharpe; every strategy may have z

Error message

no split produced a usable Sharpe; every strategy may have zero variance within the subsets

What it means

After looping over all CSCV splits, probability_of_backtest_overfitting needs at least one split where an in-sample Sharpe could be computed. If every subset has zero variance for every strategy (constant returns), all logits are undefined and the list is empty, so the function raises this error rather than returning a fabricated PBO.

Source

Thrown at agent/src/quantlib/multipletesting.py:544

        finite_out = np.isfinite(out_scores)
        if not finite_out[best] or finite_out.sum() < 2:
            continue

        # Relative rank of the selected strategy among all strategies OOS, on
        # (0, 1). Ties are resolved by average rank so a plateau does not push
        # the logit to an endpoint.
        ranked = pd.Series(np.where(finite_out, out_scores, np.nan)).rank(
            method="average"
        )
        omega = float(ranked.iloc[best] / (finite_out.sum() + 1))
        omega = min(max(omega, 1e-12), 1.0 - 1e-12)

        logits.append(math.log(omega / (1.0 - omega)))
        in_sample_sharpes.append(float(in_scores[best]))
        out_sample_sharpes.append(float(out_scores[best]))

    if not logits:
        raise ValueError(
            "no split produced a usable Sharpe; every strategy may have zero "
            "variance within the subsets"
        )

    logit_array = np.array(logits)
    in_array = np.array(in_sample_sharpes)
    out_array = np.array(out_sample_sharpes)

    if in_array.size >= 2 and float(in_array.std()) > 0:
        degradation = float(np.polyfit(in_array, out_array, 1)[0])
    else:
        degradation = float("nan")

    return CSCVResult(
        pbo=float((logit_array <= 0).mean()),
        logits=logit_array,
        n_splits=len(logits),
        n_strategies=n_strategies,

View on GitHub (pinned to 80ffdda44c)

Solutions

1. Inspect performance for zero-variance columns: np.var(perf, axis=0) — every column should be > 0.
2. Fix the upstream return computation (e.g. you passed prices or a constant seed) and re-run.
3. If synthetic data is intended, add noise so Sharpe is defined.

Example fix

# before
perf = np.zeros((100, 5))
pbo = probability_of_backtest_overfitting(perf, 16)  # raises

# after
rng = np.random.default_rng(0)
perf = rng.normal(0, 0.01, size=(100, 5))
pbo = probability_of_backtest_overfitting(perf, 16)

v = np.var(np.asarray(performance, dtype=float), axis=0)
assert (v > 0).all(), f'zero-variance strategy columns: {np.where(v == 0)[0]}'

def all_strategies_vary(p) -> bool:
    return (np.var(np.asarray(p, dtype=float), axis=0) > 0).all()

try:
    pbo = probability_of_backtest_overfitting(perf, n_splits)
except ValueError as e:
    if 'usable Sharpe' in str(e):
        raise DataQualityError('constant returns fed to CSCV') from e
    raise

When it happens

Trigger: Passing a constant matrix (all rows identical) as performance; a pipeline bug that feeds cumulative equity curves' first differences of zero; strategies whose per-block returns are all exactly 0.0 due to a data alignment bug producing duplicate rows.

Common situations: Zero-filled data from a failed fetch or an uninitialised array; returns computed from a stale/cached price series so all diffs are zero; feed-forward of a constants-only synthetic fixture in tests.

Related errors

• n_splits must be an even number >= 4, got {n_splits}
• performance must be 2-D, got shape {matrix.shape}
• CSCV ranks strategies against each other and needs at least
• {n_rows} rows split {n_splits} ways gives {subset_size} row(
• the characteristic has no cross-sectional variation, so a z-

AI-assisted analysis of HKUDS/Vibe-Trading@80ffdda44c (2026-08-28). Data as JSON: /api/errors/56f02926090e6bf8. Report an issue: GitHub.