dotnet/BenchmarkDotNet · error · ArgumentException

'source' cannot be null or whitespace.

Error message

'source' cannot be null or whitespace.

What it means

ValidateSourceIsAssemblyOrThrow throws ArgumentException when the 'source' passed to the VSTest adapter is null or empty. The adapter must be pointed at a benchmark assembly (.dll/.exe); an empty source gives it nothing to load. Note: the message says 'whitespace' but the check is actually IsNullOrEmpty.

Source

Thrown at src/BenchmarkDotNet.TestAdapter/VSTestAdapter.cs:234

            // create the AppDomain for .NET Framework are not part of .NET Standard, and so a multi-targeting solution
            // such as this is required to get this to work. This same approach is also used by other .NET unit testing
            // libraries as well, further justifying this approach to solving how to get the correct assemblies loaded.
#if NETFRAMEWORK
            var appBase = Path.GetDirectoryName(assemblyPath);
            var setup = new AppDomainSetup { ApplicationBase = appBase };
            var domainName = $"Isolated Domain for {type.Name}";
            var appDomain = AppDomain.CreateDomain(domainName, null, setup);
            return appDomain.CreateInstanceAndUnwrap(
                type.Assembly.FullName, type.FullName, false, BindingFlags.Default, null, null, null, null);
#else
            return Activator.CreateInstance(type);
#endif
        }

        private static void ValidateSourceIsAssemblyOrThrow(string source)
        {
            if (string.IsNullOrEmpty(source))
                throw new ArgumentException($"'{nameof(source)}' cannot be null or whitespace.", nameof(source));

            if (!Path.HasExtension(source))
                throw new NotSupportedException($"Missing extension on source '{source}', must have the extension '.dll' or '.exe'.");

            var extension = Path.GetExtension(source);
            if (!string.Equals(extension, ".dll", StringComparison.OrdinalIgnoreCase) && !string.Equals(extension, ".exe", StringComparison.OrdinalIgnoreCase))
                throw new NotSupportedException($"Unsupported extension on source '{source}', must have the extension '.dll' or '.exe'.");
        }
    }
}

View on GitHub (pinned to b515068b61)

Solutions

  1. Verify the benchmark project builds and produces a .dll/.exe that the test platform can locate.
  2. Check .runsettings/test platform configuration for empty source entries.
  3. If invoking the adapter programmatically, guard against null/empty before calling.

Example fix

// before
adapter.RunBenchmarks(source: path ?? "", handle);

// after
if (string.IsNullOrWhiteSpace(path)) throw new ArgumentException(nameof(path));
adapter.RunBenchmarks(path, handle);
Defensive patterns

Strategy: validation

Validate before calling

static string ValidateSource(string source)
{
    if (string.IsNullOrWhiteSpace(source))
        throw new ArgumentException("source must be a benchmark assembly path", nameof(source));
    return source;
}

Prevention

When it happens

Trigger: RunBenchmarks/GetVsTestCasesFromAssembly invoked with source = null or "". Typically the VSTest host passed an empty source list entry, or the consuming code forwarded a null path.

Common situations: Misconfigured .runsettings or test platform passing an empty source; programmatic callers of the adapter that hand in null; build output that failed to copy the benchmark assembly so the resolved path is empty.

Related errors

AI-assisted analysis of dotnet/BenchmarkDotNet@b515068b61 (2026-08-13). Data as JSON: /api/errors/cf3f5e767cac6b5e. Report an issue: GitHub.