{"record":{"id":"b3cf57646bf5d5e7","repo":"HKUDS/Vibe-Trading","slug":"cscv-ranks-strategies-against-each-other-and-needs","errorCode":null,"errorMessage":"CSCV ranks strategies against each other and needs at least 2, got {n_strategies}","messagePattern":"CSCV ranks strategies against each other and needs at least 2, got (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"agent/src/quantlib/multipletesting.py","lineNumber":487,"sourceCode":"        A :class:`CSCVResult`.\n\n    Raises:\n        ValueError: If ``n_splits`` is odd or below 4, if fewer than 2\n            strategies are supplied (a rank needs competitors), if the sample\n            cannot give each subset at least 2 rows, or if every strategy has\n            zero variance so no Sharpe is defined.\n    \"\"\"\n    if n_splits < 4 or n_splits % 2 != 0:\n        raise ValueError(f\"n_splits must be an even number >= 4, got {n_splits}\")\n\n    frame = pd.DataFrame(performance)\n    matrix = frame.to_numpy(dtype=float)\n    if matrix.ndim != 2:\n        raise ValueError(f\"performance must be 2-D, got shape {matrix.shape}\")\n\n    n_rows, n_strategies = matrix.shape\n    if n_strategies < 2:\n        raise ValueError(\n            f\"CSCV ranks strategies against each other and needs at least 2, \"\n            f\"got {n_strategies}\"\n        )\n\n    subset_size = n_rows // n_splits\n    if subset_size < 2:\n        raise ValueError(\n            f\"{n_rows} rows split {n_splits} ways gives {subset_size} row(s) per \"\n            \"subset; each subset needs at least 2 for a Sharpe\"\n        )\n\n    used_rows = subset_size * n_splits\n    dropped = n_rows - used_rows\n    trimmed = matrix[:used_rows]\n    subsets = [\n        trimmed[i * subset_size : (i + 1) * subset_size] for i in range(n_splits)\n    ]\n","sourceCodeStart":469,"sourceCodeEnd":505,"githubUrl":"https://github.com/HKUDS/Vibe-Trading/blob/80ffdda44c5c4db0dd84d70e051cca591cea67df/agent/src/quantlib/multipletesting.py#L469-L505","documentation":"CSCV ranks strategies against each other within each split, so probability_of_backtest_overfitting needs at least 2 strategy columns to form a cross-sectional rank. With a single strategy there is no 'best of N' selection and the probability of backtest overfitting is not defined, hence the explicit refusal.","triggerScenarios":"Passing a performance matrix with exactly one column, e.g. shape (1000, 1), or a one-column DataFrame / list of one series.","commonSituations":"Prototyping the PBO pipeline with a single candidate strategy; a column-selection bug upstream that accidentally slices to one column; a grid search that filters candidates before feeding CSCV and leaves a sole survivor.","solutions":["Feed all candidate strategies from the trial grid, not just the winner.","Check matrix.shape[1] >= 2 before calling.","If you genuinely have one strategy, skip PBO — it measures selection overfitting across trials."],"exampleFix":"# before\npbo = probability_of_backtest_overfitting(perf[:, :1], n_splits=16)\n\n# after\nassert perf.shape[1] >= 2, 'CSCV needs competing strategies'\npbo = probability_of_backtest_overfitting(perf, n_splits=16)","handlingStrategy":"validation","validationCode":"assert np.asarray(performance).shape[1] >= 2, 'CSCV needs >= 2 strategies'","typeGuard":"def has_enough_strategies(p, minimum: int = 2) -> bool:\n    return np.asarray(p).ndim == 2 and np.asarray(p).shape[1] >= minimum","tryCatchPattern":"try:\n    pbo = probability_of_backtest_overfitting(perf, n_splits)\nexcept ValueError as e:\n    if 'at least 2' in str(e):\n        skip_pbo('single-strategy run')\n    else:\n        raise","preventionTips":["Feed the full trial grid to CSCV, not just survivors.","Guard column selection with shape checks.","Remember PBO measures selection overfitting; with 1 trial it is undefined."],"tags":["backtesting","cscv","pbo","validation"],"backgroundTag":"insufficient-data-for-operation","analyzedSha":"80ffdda44c5c4db0dd84d70e051cca591cea67df","analyzedAt":"2026-08-28T12:46:38.989Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}