{"record":{"id":"346cb573e4f94943","repo":"n8n-io/n8n","slug":"set-kind-must-be-regression-or-capability-gap","errorCode":null,"errorMessage":"--set-kind must be \"regression\" or \"capability_gap\"","messagePattern":"--set-kind must be \"regression\" or \"capability_gap\"","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/@n8n/instance-ai/evaluations/cli/langtracer-push.ts","lineNumber":92,"sourceCode":"\t\t\t\tresult.filter = nextArg(argv, i, arg);\n\t\t\t\ti += 2;\n\t\t\t\tbreak;\n\t\t\tcase '--exclude':\n\t\t\t\tresult.exclude = nextArg(argv, i, arg);\n\t\t\t\ti += 2;\n\t\t\t\tbreak;\n\t\t\tcase '--tier':\n\t\t\t\tresult.tier = nextArg(argv, i, arg);\n\t\t\t\ti += 2;\n\t\t\t\tbreak;\n\t\t\tcase '--changed':\n\t\t\t\tresult.changed = true;\n\t\t\t\ti += 1;\n\t\t\t\tbreak;\n\t\t\tcase '--set-kind': {\n\t\t\t\tconst value = nextArg(argv, i, arg);\n\t\t\t\tif (value !== 'regression' && value !== 'capability_gap') {\n\t\t\t\t\tthrow new Error('--set-kind must be \"regression\" or \"capability_gap\"');\n\t\t\t\t}\n\t\t\t\tresult.setKind = value;\n\t\t\t\ti += 2;\n\t\t\t\tbreak;\n\t\t\t}\n\t\t\tcase '--contains-user-data':\n\t\t\t\tresult.synthetic = false;\n\t\t\t\ti += 1;\n\t\t\t\tbreak;\n\t\t\tcase '--dry-run':\n\t\t\t\tresult.dryRun = true;\n\t\t\t\ti += 1;\n\t\t\t\tbreak;\n\t\t\tcase '-h':\n\t\t\tcase '--help':\n\t\t\t\treturn { helpRequested: true };\n\t\t\tdefault:\n\t\t\t\tif (arg.startsWith('--')) {","sourceCodeStart":74,"sourceCodeEnd":110,"githubUrl":"https://github.com/n8n-io/n8n/blob/5ac6606e81f67bb9534255570cd4e86fd8101eee/packages/@n8n/instance-ai/evaluations/cli/langtracer-push.ts#L74-L110","documentation":"The --set-kind flag for langtracer-push controls the test-set classification assigned to created cases in the lang-tracer suite. It accepts exactly two enum values: 'regression' (default) or 'capability_gap'. Any other value is rejected because the lang-tracer API expects these exact strings — an invalid value would fail server-side or silently miscategorize cases.","triggerScenarios":"Running `pnpm eval:langtracer-push --suite baseline --changed --set-kind regression_test` or any value other than 'regression' or 'capability_gap'. The check at line 91 fires immediately after nextArg returns the value.","commonSituations":"Typo or variation (e.g. 'regression_test', 'Regression', 'capability-gap' with a hyphen). Guessing the enum value. Using a camelCase variant like 'capabilityGap'.","solutions":["Use exactly 'regression' or 'capability_gap' (lowercase, underscore-separated)","Omit the flag to use the default value 'regression'"],"exampleFix":"# before\npnpm eval:langtracer-push --suite baseline --changed --set-kind regression_test\n\n# after\npnpm eval:langtracer-push --suite baseline --changed --set-kind regression","handlingStrategy":"validation","validationCode":"const VALID_SET_KINDS = ['regression', 'capability_gap'] as const;\nfunction parseSetKind(raw: string): typeof VALID_SET_KINDS[number] {\n  if (!VALID_SET_KINDS.includes(raw as never)) {\n    throw new Error(`--set-kind must be one of: ${VALID_SET_KINDS.join(', ')}`);\n  }\n  return raw as typeof VALID_SET_KINDS[number];\n}","typeGuard":"function isSetKind(v: unknown): v is 'regression' | 'capability_gap' {\n  return v === 'regression' || v === 'capability_gap';\n}","tryCatchPattern":null,"preventionTips":["Use exactly 'regression' or 'capability_gap' (lowercase, underscore)","Omit --set-kind to use the default 'regression' if unsure"],"tags":["cli","validation","argument-parsing","langtracer-push"],"backgroundTag":null,"analyzedSha":"5ac6606e81f67bb9534255570cd4e86fd8101eee","analyzedAt":"2026-08-12T05:26:35.080Z","schemaVersion":2},"datasetVersion":"2026-08-12T18:17:37.767Z"}