{"record":{"id":"e278547fe9afffc4","repo":"linshenkx/prompt-optimizer","slug":"image-compare-evaluation-requires-every-compared-s","errorCode":null,"errorMessage":"Image compare evaluation requires every compared snapshot to include output image evidence.","messagePattern":"Image compare evaluation requires every compared snapshot to include output image evidence\\.","errorType":"validation","errorClass":"EvaluationValidationError","httpStatus":null,"severity":"error","filePath":"packages/core/src/services/evaluation/service.ts","lineNumber":526,"sourceCode":"        request.snapshots.forEach((snapshot, index) => {\n          this.validateSnapshot(snapshot, `Compare snapshot #${index + 1}`);\n          if (!testCaseIds.has(snapshot.testCaseId)) {\n            throw new EvaluationValidationError(\n              `Compare snapshot #${index + 1} references unknown testCaseId \"${snapshot.testCaseId}\".`\n            );\n          }\n        });\n        if (this.isImageText2ImageMode(request)) {\n          const snapshotsWithMedia = request.snapshots.filter((snapshot) =>\n            this.hasSnapshotOutputMedia(snapshot)\n          );\n          if (snapshotsWithMedia.length < 2) {\n            throw new EvaluationValidationError(\n              'Image compare evaluation requires at least two snapshots with output image evidence.'\n            );\n          }\n          if (snapshotsWithMedia.length !== request.snapshots.length) {\n            throw new EvaluationValidationError(\n              'Image compare evaluation requires every compared snapshot to include output image evidence.'\n            );\n          }\n        }\n        break;\n\n      case 'prompt-only':\n        if (!request.target?.workspacePrompt?.trim()) {\n          throw new EvaluationValidationError('Workspace prompt must not be empty.');\n        }\n        break;\n\n      case 'prompt-iterate':\n        if (!request.target?.workspacePrompt?.trim()) {\n          throw new EvaluationValidationError('Workspace prompt must not be empty.');\n        }\n        if (!request.iterateRequirement?.trim()) {\n          throw new EvaluationValidationError('Iteration requirement must not be empty.');","sourceCodeStart":508,"sourceCodeEnd":544,"githubUrl":"https://github.com/linshenkx/prompt-optimizer/blob/3e677b1d9f7e0493c142c175560531e7ae786dce/packages/core/src/services/evaluation/service.ts#L508-L544","documentation":"Thrown for compare evaluations in image text-2-image mode when at least two snapshots have output image evidence but not every snapshot does. Image compare is all-or-nothing: the service requires the count of snapshots with media to equal the total snapshot count so the comparison is fair.","triggerScenarios":"evaluate({type:'compare', mode:<image text-2-image>}) where some snapshots have output image media and others do not.","commonSituations":"One variant's image generation failed or is still in progress while others succeeded; heterogeneous compare payload mixing image and text snapshots; evidence field dropped for one snapshot during persistence.","solutions":["Either attach output images to every snapshot or drop the image-less snapshots entirely before comparing","Verify every variant run completed image generation before starting the compare","Check storage/serialization that one snapshot's media evidence was not lost"],"exampleFix":"// before\nconst snapshots = [imgSnapA, imgSnapB, textSnapC];\nawait svc.evaluate({ type: 'compare', mode, snapshots, testCases, ... });\n\n// after\nconst snapshots = [imgSnapA, imgSnapB]; // all have output media\nawait svc.evaluate({ type: 'compare', mode, snapshots, testCases, ... });","handlingStrategy":"validation","validationCode":"const withMedia = snapshots.filter(hasSnapshotOutputMedia);\nif (isImageMode(mode) && withMedia.length !== snapshots.length) {\n  // drop image-less snapshots or block the compare\n  if (withMedia.length >= 2) snapshots = withMedia; else throw new Error('Incomplete image evidence');\n}\nawait svc.evaluate({ ...req, snapshots });","typeGuard":"const everySnapshotHasMedia = (snaps: EvaluationSnapshot[]) => snaps.every(hasSnapshotOutputMedia);","tryCatchPattern":null,"preventionTips":["Treat image compare as all-or-nothing in the UI","Warn on partially-completed variant sets"],"tags":["evaluation","image","compare","validation","evidence"],"backgroundTag":"inconsistent-input-items","analyzedSha":"3e677b1d9f7e0493c142c175560531e7ae786dce","analyzedAt":"2026-08-27T21:29:16.709Z","schemaVersion":2},"datasetVersion":"2026-08-28T00:17:15.603Z"}