HKUDS/Vibe-Trading · error · ValueError

fit_markov_regime needs at least 50 finite observations, got

Error message

fit_markov_regime needs at least 50 finite observations, got {series.size}

What it means

fit_markov_regime needs at least 50 finite observations after dropna; Markov-switching MLE is badly identified on short samples and statsmodels would emit spurious regimes or convergence failures, so the library enforces a floor.

Source

Thrown at agent/src/quantlib/timeseries.py:550

        ``aic``, ``bic`` (float).

    Raises:
        ImportError: If ``statsmodels`` is not installed.
        ValueError: If ``n_regimes`` is below 2, or fewer than 50 finite
            observations survive -- an EM fit on a shorter series produces
            regimes that are numerically fine and substantively meaningless.
    """
    markov = _require(
        "statsmodels.tsa.regime_switching.markov_regression",
        "statsmodels",
        "fit_markov_regime",
    )
    if n_regimes < 2:
        raise ValueError(f"n_regimes must be at least 2, got {n_regimes}")

    series = pd.Series(returns, dtype=float).dropna()
    if series.size < 50:
        raise ValueError(
            f"fit_markov_regime needs at least 50 finite observations, got {series.size}"
        )

    # Passed as a Series, not an array: statsmodels only names the fitted
    # parameters (``const[k]``, ``sigma2[k]``) when the input is pandas, and
    # positional unpacking of that vector would silently break if the package
    # ever reorders it.
    model = markov.MarkovRegression(
        series * 100,
        k_regimes=n_regimes,
        trend="c",
        switching_variance=switching_variance,
    )
    with contextlib.redirect_stdout(io.StringIO()), contextlib.redirect_stderr(io.StringIO()):
        result = model.fit()

    means = np.array(
        [float(result.params[f"const[{k}]"]) / 100 for k in range(n_regimes)]

View on GitHub (pinned to 80ffdda44c)

Solutions

Provide at least 50 (preferably 200+) observations
Use returns.dropna() before passing so the count is what you expect
For short windows, use a simpler model (rolling std, GARCH) instead

Example fix

# before
res = fit_markov_regime(df['close'].pct_change().dropna().tail(30))
# after
rets = df['close'].pct_change().dropna()
assert rets.size >= 50
res = fit_markov_regime(rets)

Defensive patterns

Strategy: validation

Validate before calling

series = pd.Series(returns, dtype=float).dropna()
assert series.size >= 50, f'Markov fit needs >= 50 obs, got {series.size}'

Type guard

def enough_data_for_markov(returns) -> bool:
    return pd.Series(returns, dtype=float).dropna().size >= 50

Try / catch

try:
    fit_markov_regime(returns)
except ValueError as e:
    if 'at least 50 finite observations' in str(e):
        # fall back to rolling volatility or GARCH
        return rolling_vol(returns)
    raise

Prevention

Dropna returns before passing (pct_change creates a leading NaN)
Use >= 200 observations for stable regime estimates
Prefer rolling-std or GARCH on short windows

When it happens

Trigger: Passing fewer than 50 non-NaN returns, series with many NaNs that drop below 50, or returns from a short backtest window.

Common situations: Testing on a month of daily data (~21 points), passing a series with NaNs from pct_change without dropping them, or slicing a recent window that is too short.

Related errors

AI-assisted analysis of HKUDS/Vibe-Trading@80ffdda44c (2026-08-28). Data as JSON: /api/errors/139e35b4a2568c6d. Report an issue: GitHub.