vxcontrol/pentagi · error

no tests to execute

Error message

no tests to execute

What it means

TestProvider collected zero executable test requests after filtering the loaded registry, so there is nothing to run and it fails fast instead of returning empty results. collectTestRequests applies agent-type, group, and test-type filters on top of the registry; if every filter excludes everything, the slice is empty.

Source

Thrown at backend/pkg/providers/tester/runner.go:53

	config := applyOptions(opts)

	// load test registry
	var registry *testdata.TestRegistry
	var err error

	if config.customRegistry != nil {
		registry = config.customRegistry
	} else {
		registry, err = testdata.LoadBuiltinRegistry()
		if err != nil {
			return ProviderTestResults{}, fmt.Errorf("failed to load test registry: %w", err)
		}
	}

	// collect all test requests
	requests := collectTestRequests(registry, prv, config)
	if len(requests) == 0 {
		return ProviderTestResults{}, fmt.Errorf("no tests to execute")
	}

	// execute tests in parallel
	responses := executeTestsParallel(ctx, requests, config)

	// group results by agent type
	return groupResults(responses), nil
}

// collectTestRequests gathers all test requests based on configuration
func collectTestRequests(registry *testdata.TestRegistry, prv provider.Provider, config *testConfig) []testRequest {
	var requests []testRequest

	// create agent type filter
	agentFilter := make(map[pconfig.ProviderOptionsType]bool)
	for _, agentType := range config.agentTypes {
		agentFilter[agentType] = true
	}

View on GitHub (pinned to ea665308ba)

Solutions

1. Print registry contents / enable verbose logging to see which groups and suites actually loaded.
2. Check the WithGroups/WithAgentTypes option values for typos against the registry's defined group names.
3. Remove or widen the restrictive filter options and re-run without filters to confirm the registry is non-empty.
4. If using WithCustomRegistry, verify it has at least one test suite whose group and agent types match the filters.
5. Fix the misspelled group name rather than relying on the silent warning path.

Example fix

// before
tester.TestProvider(ctx, prv, tester.WithGroups("file_edit")) // typo: group is "fileedit"
// after
tester.TestProvider(ctx, prv, tester.WithGroups("fileedit"))

// pre-check that filters will match something
registry, _ := testdata.LoadBuiltinRegistry()
for _, g := range groups {
    if _, err := registry.GetTestSuite(g); err != nil {
        return fmt.Errorf("group %q does not exist in registry", g)
    }
}

results, err := tester.TestProvider(ctx, prv, opts...)
if err != nil {
    if err.Error() == "no tests to execute" {
        log.Fatalf("filters matched nothing — check WithGroups/WithAgentTypes values and registry contents")
    }
    return err
}

When it happens

Trigger: Calling tester.TestProvider with options that over-filter: WithAgentTypes naming agent types with no registered cases, WithGroups naming a nonexistent group (registry.GetTestSuite fails and is only logged as a warning at runner.go:76-79), or a custom registry that is empty / contains no suites matching the requested groups.

Common situations: Typo in a group or agent-type option value; running the tester against a custom registry built programmatically but never populated; filtering to a test type the builtin registry does not define; expecting warnings for bad group names to be fatal but they are silently skipped (visible only in verbose mode).

Related errors

• failed to load test registry: %w
• token validation disabled with default salt
• failed to switch provider: %w
• knowledge: embedding provider is not configured
• test case has no prompt or messages

AI-assisted analysis of vxcontrol/pentagi@ea665308ba (2026-09-01). Data as JSON: /api/errors/4c76087fe625b726. Report an issue: GitHub.