sgl-project/sglang · error · ValueError

Failed to fit coefficients: insufficient rank

Error message

Failed to fit coefficients: insufficient rank

What it means

np.linalg.lstsq returned fewer than 3 coefficients for the quadratic fit, i.e. the design matrix is rank-deficient — typically because all profiled sequence lengths are identical or nearly so.

Source

Thrown at python/sglang/srt/managers/scheduler_pp_mixin.py:1545

        T = np.array(latencies[1:], dtype=np.float64)

        if len(L) < 8:
            raise ValueError(
                f"Not enough data points for quadratic fitting ({len(L)} < 8). "
                "Need at least 8 samples with different sequence lengths."
            )

        # Build design matrix for f(l) = al^2 + bl + c
        X = np.column_stack([L * L, L, np.ones_like(L)])  # [l^2, l, 1]

        try:
            coeffs, residuals, rank, s = np.linalg.lstsq(X, T, rcond=None)
            if len(coeffs) >= 3:
                fitted_a = float(coeffs[0])  # quadratic coefficient
                fitted_b = float(coeffs[1])  # linear coefficient
                fitted_c = float(coeffs[2])  # constant coefficient
            else:
                raise ValueError("Failed to fit coefficients: insufficient rank")
        except np.linalg.LinAlgError as e:
            raise ValueError(f"Failed to fit f(l) = al^2 + bl + c: {e}")

        # Validate coefficients
        if fitted_a <= 0:
            raise ValueError(
                f"Fitted quadratic coefficient a={fitted_a:.2e} is not positive. "
                "Attention has O(n^2) complexity, so a must be positive. "
                "Check warmup data quality."
            )

        if fitted_b < 0:
            logger.warning(
                f"Fitted linear coefficient b={fitted_b:.2e} is negative. Setting b=0."
            )
            fitted_b = 0.0

        self.quadratic_coeff_a = fitted_a

View on GitHub (pinned to 0132848349)

Solutions

1. Ensure profiling uses distinct, spread-out sequence lengths
2. Check the seq_lens list passed to fit for duplicates/constant values
3. Increase profiling length range (e.g. from min to max chunked prefill length)

assert len(set(seq_lens[1:])) >= 8 and len(set(seq_lens[1:])) >= 3, 'need distinct lengths for full-rank fit'

When it happens

Trigger: profile_and_init_predictor collects latency samples whose sequence lengths do not vary (duplicate lengths), so [l², l, 1] columns are linearly dependent.

Common situations: Profiling config that repeats the same sequence length, or a bug that passes a constant list into fit().

Related errors

• Failed to fit f(l) = al^2 + bl + c: {e}
• Not enough data points for quadratic fitting ({len(L)} < 8).
• Fitted quadratic coefficient a={fitted_a:.2e} is not positiv
• Calculated target_latency={self.target_latency:.2f}ms is not
• Currently DFLASH speculative decoding only supports pp_size

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/759de24c7b31f54e. Report an issue: GitHub.