microsoft/TypeScript · error · Error

Variations in test option '@${varyBy}' resulted in an empty

Error message

Variations in test option '@${varyBy}' resulted in an empty set.

What it means

Thrown while resolving `@<varyBy>` test-option variations in harnessIO.ts:1124. After includes, the `*` star expansion, and excludes are applied to the variation set for a single varyBy key, if nothing remains the harness aborts — it refuses to run a variation matrix that collapses to zero cases.

Source

Thrown at src/harness/harnessIO.ts:1124

        // add all entries
        for (const [key, value] of ts.arrayFrom(values.entries())) {
            if (ts.findIndex(variations, v => v.key === key || v.value === value) === -1) {
                variations.push({ key, value });
            }
        }
    }

    // remove all excluded entries
    for (const exclude of excludes) {
        const value = values?.get(exclude);
        let index: number;
        while ((index = ts.findIndex(variations, v => v.key === exclude || value !== undefined && v.value === value)) >= 0) {
            ts.orderedRemoveItemAt(variations, index);
        }
    }

    if (variations.length === 0) {
        throw new Error(`Variations in test option '@${varyBy}' resulted in an empty set.`);
    }

    return ts.map(variations, v => v.key);
}

function computeFileBasedTestConfigurationVariations(configurations: FileBasedTestConfiguration[], variationState: FileBasedTestConfiguration, varyByEntries: [string, string[]][], offset: number) {
    if (offset >= varyByEntries.length) {
        // make a copy of the current variation state
        configurations.push({ ...variationState });
        return;
    }

    const [varyBy, entries] = varyByEntries[offset];
    for (const entry of entries) {
        // set or overwrite the variation, then compute the next variation
        variationState[varyBy] = entry;
        computeFileBasedTestConfigurationVariations(configurations, variationState, varyByEntries, offset + 1);
    }

View on GitHub (pinned to b465fdbfe1)

Solutions

1. Open the test file and inspect the `// @<varyBy>: <includes> exclude=<excludes>` line; ensure the exclude list does not cover every included value.
2. If you intended to disable variation for that key, remove the `// @<varyBy>` line entirely rather than excluding everything.
3. Cross-check the option name and values against ts.optionDeclarations (getVaryByStarSettingValues at harnessIO.ts:1147) — values must exist in the option's domain or `*` resolves to nothing.
4. Add at least one include value that is not in the exclude set.

Example fix

// before — exclude removes the only include
// @target: es5 exclude=es5

// after — keep at least one non-excluded value
// @target: es5,esnext exclude=es5

// Validate include/exclude combinations before relying on getFileBasedTestConfigurations.
function variationSetIsNonEmpty(includes: string[], excludes: string[], starValues?: ReadonlyMap<string, unknown>): boolean {
  const all = new Set<string>(includes);
  if (starValues) for (const k of starValues.keys()) all.add(k);
  for (const ex of excludes) all.delete(ex);
  return all.size > 0;
}

function hasSurvivingVariation(includes: readonly string[], excludes: readonly string[]): boolean {
  const ex = new Set(excludes);
  return includes.some(i => !ex.has(i));
}

When it happens

Trigger: A test metadata line like `// @target: es5 exclude=es5`, or `// @target: exclude=*`, or includes whose keys/values do not match any entry in the option's declared domain (ts.optionDeclarations). The exclude loop at harnessIO.ts:1115-1121 removes every entry that was previously added.

Common situations: Typing a value in `exclude=` that matches every included entry; listing an exclude key that is also the only include; using a value that does not exist in the option's map so `*` expansion yields nothing; copy-pasting an exclude clause from a similar option without adjusting it.

Related errors

• Provided test options exceeded the maximum number of variati
• Could not resolve JS module '${moduleName}' starting at '${i
• ${option.name} is a string value; tsconfig JSON must be pars
• verifyCurrentLineContent\n
• verifyFileContent in file '${fileName}' failed:\n${showTextD

AI-assisted analysis of microsoft/TypeScript@b465fdbfe1 (2026-08-12). Data as JSON: /api/errors/2a117ed44888e846. Report an issue: GitHub.