QuantConnect/Lean · error · RegressionTestException

Benchmark value error - expected: {_previousBenchmarkValue}

Error message

Benchmark value error - expected: {_previousBenchmarkValue} {_previousTime}, actual: {currentValue} {slice.Time}. Benchmark value should only change when there is a change in hours

What it means

This RegressionTestException is thrown in OnData when the current and previous slice share the same hour (_previousTime.Hour == slice.Time.Hour). It asserts that the benchmark value did NOT change within the same hour. LEAN throws it because the benchmark uses internal daily resolution via SetBenchmark("SPY"), so its value should remain constant within each hour and only update on hour boundaries.

Source

Thrown at Algorithm.CSharp/CustomUniverseWithBenchmarkRegressionAlgorithm.cs:120

                {
                    throw new RegressionTestException($"Security Price error. Price should change every new hour");
                }
                if (slice.Time.Minute != 0
                    && _previousSecurityValue != security.Price
                    && security.IsTradable)
                {
                    throw new RegressionTestException($"Security Price error. Price should not change every minute");
                }
            }
            _previousSecurityValue = security.Price;

            // assert benchmark updates only on date change
            var currentValue = Benchmark.Evaluate(slice.Time);
            if (_previousTime.Hour == slice.Time.Hour)
            {
                if (currentValue != _previousBenchmarkValue)
                {
                    throw new RegressionTestException($"Benchmark value error - expected: {_previousBenchmarkValue} {_previousTime}, actual: {currentValue} {slice.Time}. " +
                                        "Benchmark value should only change when there is a change in hours");
                }
            }
            else
            {
                if (slice.Time.Minute == 0)
                {
                    if (currentValue == _previousBenchmarkValue)
                    {
                        _benchmarkPriceDidNotChange++;
                        // there are two consecutive equal data points so we give it some room
                        if (_benchmarkPriceDidNotChange > 1)
                        {
                            throw new RegressionTestException($"Benchmark value error - expected a new value, current {currentValue} {slice.Time}" +
                                                "Benchmark value should change when there is a change in hours");
                        }
                    }
                    else

View on GitHub (pinned to d2c3659f87)

Solutions

1. Log currentValue, _previousBenchmarkValue, _previousTime, and slice.Time at the throw point.
2. Verify the benchmark subscription resolution is still daily (internal) and not overridden.
3. Check Benchmark.Evaluate implementation to confirm it returns a cached value within the same hour, not a fresh computation.
4. If testing engine changes, trace the benchmark data feed to ensure only daily-resolution bars update the benchmark value.

Example fix

// before
if (_previousTime.Hour == slice.Time.Hour)
{
    if (currentValue != _previousBenchmarkValue)
    {
        throw new RegressionTestException($"Benchmark value error - expected: {_previousBenchmarkValue} {_previousTime}, actual: {currentValue} {slice.Time}. Benchmark value should only change when there is a change in hours");
    }
}

// after — same check, better diagnostics
if (_previousTime.Hour == slice.Time.Hour && currentValue != _previousBenchmarkValue)
{
    throw new RegressionTestException($"Intra-hour benchmark change at {slice.Time}: prev={_previousBenchmarkValue} ({_previousTime}), current={currentValue}");
}

// Validate benchmark stability within same hour
var currentValue = Benchmark.Evaluate(slice.Time);
if (_previousTime.Hour == slice.Time.Hour && currentValue != _previousBenchmarkValue)
    Log($"Benchmark changed intra-hour at {slice.Time}: prev={_previousBenchmarkValue}, current={currentValue}");

When it happens

Trigger: Benchmark.Evaluate(slice.Time) returns a different value than _previousBenchmarkValue while both slices are in the same hour. This means the benchmark is updating intra-hour, which contradicts the internal daily-resolution benchmark data feed.

Common situations: A LEAN version change to benchmark evaluation logic caused it to use higher-resolution data, the benchmark subscription resolution was changed from daily to something finer, or Benchmark.Evaluate started sampling live prices instead of the cached daily value.

Related errors

• We expected 'SPY' to be added to the Symbol cache, since the
• Unexpected Bar error
• For the same date expected data updates every 1 minute
• Security Price error. Price should change every new hour
• Benchmark value error - expected a new value, current {curre

AI-assisted analysis of QuantConnect/Lean@d2c3659f87 (2026-08-13). Data as JSON: /api/errors/6a3cb6719c9d8267. Report an issue: GitHub.