{"record":{"id":"ed19258c0543629f","repo":"vxcontrol/pentagi","slug":"failed-to-load-test-registry-w","errorCode":null,"errorMessage":"failed to load test registry: %w","messagePattern":"failed to load test registry: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"backend/pkg/providers/tester/runner.go","lineNumber":46,"sourceCode":"\tagentType pconfig.ProviderOptionsType\n\tresult    testdata.TestResult\n\terr       error\n}\n\n// TestProvider executes tests for a provider with given options\nfunc TestProvider(ctx context.Context, prv provider.Provider, opts ...TestOption) (ProviderTestResults, error) {\n\tconfig := applyOptions(opts)\n\n\t// load test registry\n\tvar registry *testdata.TestRegistry\n\tvar err error\n\n\tif config.customRegistry != nil {\n\t\tregistry = config.customRegistry\n\t} else {\n\t\tregistry, err = testdata.LoadBuiltinRegistry()\n\t\tif err != nil {\n\t\t\treturn ProviderTestResults{}, fmt.Errorf(\"failed to load test registry: %w\", err)\n\t\t}\n\t}\n\n\t// collect all test requests\n\trequests := collectTestRequests(registry, prv, config)\n\tif len(requests) == 0 {\n\t\treturn ProviderTestResults{}, fmt.Errorf(\"no tests to execute\")\n\t}\n\n\t// execute tests in parallel\n\tresponses := executeTestsParallel(ctx, requests, config)\n\n\t// group results by agent type\n\treturn groupResults(responses), nil\n}\n\n// collectTestRequests gathers all test requests based on configuration\nfunc collectTestRequests(registry *testdata.TestRegistry, prv provider.Provider, config *testConfig) []testRequest {","sourceCodeStart":28,"sourceCodeEnd":64,"githubUrl":"https://github.com/vxcontrol/pentagi/blob/ea665308baaff015b226f308438a68d929d0f29b/backend/pkg/providers/tester/runner.go#L28-L64","documentation":"TestProvider wraps any error from testdata.LoadBuiltinRegistry() with this message. The builtin registry is the embedded set of test suites (prompts, tool-use scenarios, etc.) that the provider tester runs; if it cannot be loaded/parsed from the embedded testdata, no tests can be run and the whole call aborts.","triggerScenarios":"Calling tester.TestProvider(ctx, prv, ...) without WithCustomRegistry when LoadBuiltinRegistry() fails — typically because the embedded registry YAML/JSON is missing, corrupt, or fails schema validation (registry, err = testdata.LoadBuiltinRegistry(); runner.go:44-47).","commonSituations":"A developer added a new test case to the builtin registry with invalid schema; build tags or embed directives excluding testdata files so the embedded FS is empty; a malformed group/test definition committed to testdata; running a stale binary built before testdata was populated.","solutions":["Inspect the wrapped %w cause — it names the underlying load/parse failure; fix that file first.","Validate all testdata registry files against the expected TestRegistry schema (a recently added/edited case is the usual culprit).","Rebuild the binary so go:embed picks up the current testdata contents.","Pass a known-good custom registry via the WithCustomRegistry option to isolate whether the builtin data or the loader is at fault.","git diff the testdata directory to find the most recent registry change."],"exampleFix":"// before: builtin registry contains a malformed case added in testdata\nregistry, err := testdata.LoadBuiltinRegistry() // err: cannot unmarshal ...\n// after: fix the YAML/JSON in testdata, or fall back to a custom registry\nif err != nil {\n    registry, err = loadCustomRegistry(\"testdata/custom-registry.yaml\")\n    if err != nil {\n        return ProviderTestResults{}, fmt.Errorf(\"failed to load test registry: %w\", err)\n    }\n}","handlingStrategy":"validation","validationCode":"registry, err := testdata.LoadBuiltinRegistry()\nif err != nil {\n    return fmt.Errorf(\"builtin registry unusable, fix testdata before running tester: %w\", err)\n}\nif len(registry.Suites) == 0 {\n    return fmt.Errorf(\"builtin registry loaded but is empty — check go:embed of testdata\")\n}","typeGuard":"func registryUsable(r *testdata.TestRegistry) bool {\n    return r != nil\n}","tryCatchPattern":"results, err := tester.TestProvider(ctx, prv)\nif err != nil {\n    var loadErr error\n    if errors.As(err, &loadErr) && strings.Contains(err.Error(), \"failed to load test registry\") {\n        log.Fatalf(\"registry load failed, underlying cause: %v\", errors.Unwrap(err))\n    }\n    return err\n}","preventionTips":["Validate testdata registry files in CI (schema check / round-trip load) before merge","Rebuild after editing embedded testdata so go:embed content is current","Keep testdata changes small and reviewed — a single malformed case breaks all tester runs","Keep a custom-registry fallback for local debugging","Always log errors.Unwrap(err) — this wrapper hides the real cause"],"tags":["go","testing","configuration","embedded-resources"],"backgroundTag":"builtin-registry-load-failed","analyzedSha":"ea665308baaff015b226f308438a68d929d0f29b","analyzedAt":"2026-09-01T14:16:31.421Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}