dotnet/BenchmarkDotNet · error · ArgumentException

Can't parse threshold '{threshold}'

Error message

Can't parse threshold '{threshold}'

What it means

StatisticalTestColumn.Create(string threshold) parses a threshold string (after normalizing units via UnitHelper.NormalizeUnits) using Threshold.TryParse. If parsing fails it throws ArgumentException. Threshold strings look like relative/absolute thresholds, e.g. "10%", "5%", or an absolute value with a unit. A malformed string, an unknown unit, or a non-numeric prefix fails TryParse.

Source

Thrown at src/BenchmarkDotNet/Columns/StatisticalTestColumn.cs:23

using Perfolizer.Mathematics.Common;
using Perfolizer.Mathematics.SignificanceTesting;
using Perfolizer.Mathematics.SignificanceTesting.MannWhitney;
using Perfolizer.Metrology;

namespace BenchmarkDotNet.Columns
{
    public class StatisticalTestColumn(Threshold threshold, SignificanceLevel? significanceLevel = null) : BaselineCustomColumn
    {
        private static readonly SignificanceLevel DefaultSignificanceLevel = SignificanceLevel.P1E5;

        public static StatisticalTestColumn CreateDefault() => new(new PercentValue(10).ToThreshold());

        public static StatisticalTestColumn Create(Threshold threshold, SignificanceLevel? significanceLevel = null) => new(threshold, significanceLevel);

        public static StatisticalTestColumn Create(string threshold, SignificanceLevel? significanceLevel = null)
        {
            if (!Threshold.TryParse(UnitHelper.NormalizeUnits(threshold), out var parsedThreshold))
                throw new ArgumentException($"Can't parse threshold '{threshold}'");
            return new StatisticalTestColumn(parsedThreshold, significanceLevel);
        }

        public Threshold Threshold { get; } = threshold;
        public SignificanceLevel SignificanceLevel { get; } = significanceLevel ?? DefaultSignificanceLevel;

        public override string Id => $"{nameof(StatisticalTestColumn)}/{Threshold}";
        public override string ColumnName => $"MannWhitney({Threshold})";

        public override string GetValue(Summary summary, BenchmarkCase benchmarkCase, Statistics baseline, IReadOnlyDictionary<string, Metric> baselineMetrics,
            Statistics current, IReadOnlyDictionary<string, Metric> currentMetrics, bool isBaseline)
        {
            if (baseline.Sample.Values.SequenceEqual(current.Sample.Values))
                return "Baseline";
            if (current.Sample.Size == 1 && baseline.Sample.Size == 1)
                return "?";

            // See ZeroMeasurementHelper: moving to Pragmastat.Toolkit.Compare2 would change the

View on GitHub (pinned to b515068b61)

Solutions

1. Pre-validate with Threshold.TryParse(UnitHelper.NormalizeUnits(input), out var t) and only call Create when it returns true.
2. Use the strongly-typed overloads: Create(Threshold) / Create(PercentValue) / the constructor taking a Threshold, constructing it from a known-good PercentValue (new PercentValue(10).ToThreshold()).
3. Whitelist accepted units and formats before parsing external input.
4. Surface a clear validation error to the user instead of letting the ArgumentException propagate.

Example fix

// before
var col = StatisticalTestColumn.Create(userThresholdString); // may throw

// after
if (!Threshold.TryParse(UnitHelper.NormalizeUnits(userThresholdString), out var t))
    throw new FormatException($"Invalid threshold '{userThresholdString}'. Use forms like '10%'.");
var col = StatisticalTestColumn.Create(t);

if (!Threshold.TryParse(UnitHelper.NormalizeUnits(threshold), out var parsed))
    throw new FormatException($"Invalid threshold '{threshold}'. Use forms like '10%' or an absolute value with unit.");
return StatisticalTestColumn.Create(parsed);

static bool IsValidThreshold(string s)
    => Threshold.TryParse(UnitHelper.NormalizeUnits(s), out _);

try { return StatisticalTestColumn.Create(userThreshold); }
catch (ArgumentException ex) when (ex.Message.Contains("Can't parse threshold"))
{
    // fall back to a default threshold, e.g. new PercentValue(10).ToThreshold()
}

When it happens

Trigger: Calling StatisticalTestColumn.Create("abc") or Create("10xyz") with an unrecognized value/unit, or passing a user-supplied threshold string from config/CLI without validation.

Common situations: Building a Mann-Whitney statistical-test column from a config value, a CLI flag, or a JSON file whose threshold syntax is wrong (missing %, unknown unit, stray characters).

Related errors

• The value {value} is not assignable to {characteristic} prop
• String cannot be empty.
• String cannot be empty or whitespace.
• List of measurements contains no elements
• Runtime Moniker not supported

AI-assisted analysis of dotnet/BenchmarkDotNet@b515068b61 (2026-08-13). Data as JSON: /api/errors/b7f76af01d65d50b. Report an issue: GitHub.