{"record":{"id":"f31812a996d6be67","repo":"linshenkx/prompt-optimizer","slug":"compare-snapshot-index-1-references-unknown","errorCode":null,"errorMessage":"Compare snapshot #${index + 1} references unknown testCaseId \"${snapshot.testCaseId}\".","messagePattern":"Compare snapshot #(.+?) references unknown testCaseId \"(.+?)\"\\.","errorType":"validation","errorClass":"EvaluationValidationError","httpStatus":null,"severity":"error","filePath":"packages/core/src/services/evaluation/service.ts","lineNumber":511,"sourceCode":"        if (!Array.isArray(request.snapshots) || request.snapshots.length < 2) {\n          throw new EvaluationValidationError('Compare evaluation requires at least two snapshots.');\n        }\n\n        const testCaseIds = new Set<string>();\n        request.testCases.forEach((testCase, index) => {\n          this.validateTestCase(testCase, `Compare test case #${index + 1}`);\n          if (testCaseIds.has(testCase.id)) {\n            throw new EvaluationValidationError(\n              `Compare test case #${index + 1} id must be unique.`\n            );\n          }\n          testCaseIds.add(testCase.id);\n        });\n\n        request.snapshots.forEach((snapshot, index) => {\n          this.validateSnapshot(snapshot, `Compare snapshot #${index + 1}`);\n          if (!testCaseIds.has(snapshot.testCaseId)) {\n            throw new EvaluationValidationError(\n              `Compare snapshot #${index + 1} references unknown testCaseId \"${snapshot.testCaseId}\".`\n            );\n          }\n        });\n        if (this.isImageText2ImageMode(request)) {\n          const snapshotsWithMedia = request.snapshots.filter((snapshot) =>\n            this.hasSnapshotOutputMedia(snapshot)\n          );\n          if (snapshotsWithMedia.length < 2) {\n            throw new EvaluationValidationError(\n              'Image compare evaluation requires at least two snapshots with output image evidence.'\n            );\n          }\n          if (snapshotsWithMedia.length !== request.snapshots.length) {\n            throw new EvaluationValidationError(\n              'Image compare evaluation requires every compared snapshot to include output image evidence.'\n            );\n          }","sourceCodeStart":493,"sourceCodeEnd":529,"githubUrl":"https://github.com/linshenkx/prompt-optimizer/blob/3e677b1d9f7e0493c142c175560531e7ae786dce/packages/core/src/services/evaluation/service.ts#L493-L529","documentation":"Thrown during compare validation when a snapshot's testCaseId does not appear among the ids of the provided testCases array. Every compared snapshot must belong to a known test case so the evaluator can group candidates per case. The message includes the 1-based snapshot index and the offending testCaseId.","triggerScenarios":"evaluate({type:'compare'}) where some snapshot.testCaseId is not in the set of testCases[].id — e.g. snapshot from a deleted test case, typo'd id, or stale fixture data.","commonSituations":"Test cases deleted or re-keyed after snapshots were captured; snapshots loaded from another workspace/run; frontend filtering test cases but not their snapshots.","solutions":["Filter snapshots to those whose testCaseId exists in the testCases list before calling evaluate","Refresh/re-capture snapshots after test case ids change","Include the missing test case in request.testCases if it should participate"],"exampleFix":"// before\nawait svc.evaluate({ type: 'compare', testCases, snapshots: allSnapshots, ... });\n\n// after\nconst ids = new Set(testCases.map(tc => tc.id));\nconst snapshots = allSnapshots.filter(s => ids.has(s.testCaseId));\nawait svc.evaluate({ type: 'compare', testCases, snapshots, ... });","handlingStrategy":"validation","validationCode":"const ids = new Set(req.testCases.map(tc => tc.id));\nconst orphans = req.snapshots.filter(s => !ids.has(s.testCaseId));\nif (orphans.length) throw new Error(`Orphan snapshots: ${orphans.map(s => s.testCaseId).join(', ')}`);\nawait svc.evaluate(req);","typeGuard":"const snapshotsAllReferenced = (snaps: EvaluationSnapshot[], tcs: EvaluationTestCase[]) =>\n  snaps.every(s => new Set(tcs.map(t => t.id)).has(s.testCaseId));","tryCatchPattern":null,"preventionTips":["Load snapshots scoped by the same test-case query","Cascade-delete or refresh snapshots when test cases are removed"],"tags":["evaluation","validation","compare","snapshot","foreign-key"],"backgroundTag":"dangling-reference-id","analyzedSha":"3e677b1d9f7e0493c142c175560531e7ae786dce","analyzedAt":"2026-08-27T21:29:16.709Z","schemaVersion":2},"datasetVersion":"2026-08-28T00:17:15.603Z"}