{"record":{"id":"39a8ed64714286d0","repo":"abi/screenshot-to-code","slug":"text-eval-set-not-found-set-name","errorCode":null,"errorMessage":"Text eval set not found: {set_name}","messagePattern":"Text eval set not found: (.+?)","errorType":"exception","errorClass":"EvalSetNotFoundError","httpStatus":null,"severity":"error","filePath":"backend/evals/sets.py","lineNumber":101,"sourceCode":"\ndef _briefs_path(set_name: str) -> str:\n    return os.path.join(_get_set_dir(set_name), \"briefs.json\")\n\n\n_BRIEF_ID_PATTERN = re.compile(r\"^[a-z0-9][a-z0-9-]*$\")\n\n\ndef get_set_kind(set_name: str) -> str:\n    \"\"\"\"text\" when the set is a briefs.json collection, else \"image\".\"\"\"\n    if os.path.isfile(_briefs_path(set_name)):\n        return \"text\"\n    return \"image\"\n\n\ndef list_set_briefs(set_name: str) -> list[EvalSetBrief]:\n    path = _briefs_path(set_name)\n    if not os.path.isfile(path):\n        raise EvalSetNotFoundError(f\"Text eval set not found: {set_name}\")\n    try:\n        with open(path, \"r\", encoding=\"utf-8\") as f:\n            loaded = cast(dict[str, Any], json.load(f))\n    except (OSError, json.JSONDecodeError) as exc:\n        raise EvalSetNotFoundError(f\"Unreadable briefs.json for {set_name}: {exc}\")\n    briefs: list[EvalSetBrief] = []\n    raw_briefs = loaded.get(\"briefs\")\n    entries = (\n        cast(list[object], raw_briefs) if isinstance(raw_briefs, list) else []\n    )\n    for entry in entries:\n        if not isinstance(entry, dict):\n            continue\n        record = cast(dict[str, Any], entry)\n        brief_id = str(record.get(\"id\") or \"\")\n        brief = str(record.get(\"brief\") or \"\")\n        if not _BRIEF_ID_PATTERN.match(brief_id) or not brief:\n            raise InvalidSetNameError(","sourceCodeStart":83,"sourceCodeEnd":119,"githubUrl":"https://github.com/abi/screenshot-to-code/blob/d026163f586dfa8c5c10d28c36edd59a9d3b0e88/backend/evals/sets.py#L83-L119","documentation":"EvalSetNotFoundError raised by list_set_briefs when the set's briefs.json does not exist at the expected path. A set is 'text' kind only when briefs.json is present (get_set_kind probes the same file); requesting briefs for an image-kind set or a set that doesn't exist at all both land here.","triggerScenarios":"Calling list_set_briefs on a set directory with only an inputs/ folder (image set), a misspelled/deleted set name, or a set created without writing briefs.json.","commonSituations":"UI letting users pick any set for a text-brief flow, set folder renamed manually on disk, or a race where the set was deleted between listing and reading.","solutions":["Check get_set_kind(set_name) == 'text' before listing briefs.","Verify the set name against list of existing sets before use.","If the set should be text-kind, create a valid briefs.json with a {\"briefs\": [...]} structure."],"exampleFix":"# before\nbriefs = list_set_briefs(set_name)\n\n# after\nif get_set_kind(set_name) != \"text\":\n    raise ValueError(f\"Set {set_name!r} has no briefs (image set)\")\nbriefs = list_set_briefs(set_name)","handlingStrategy":"type-guard","validationCode":"from evals.sets import get_set_kind\nif get_set_kind(set_name) != \"text\":\n    raise ValueError(f\"{set_name!r} is not a text/briefs set\")","typeGuard":"def is_text_set(set_name: str) -> bool:\n    return os.path.isfile(_briefs_path(set_name))","tryCatchPattern":"try:\n    briefs = list_set_briefs(set_name)\nexcept EvalSetNotFoundError:\n    return HTTPException(status_code=404, detail=f\"Eval set {set_name} not found\")","preventionTips":["Gate text-brief flows on get_set_kind before listing briefs.","Filter set pickers by kind so users cannot select image sets for brief runs.","Handle 404 gracefully in the UI with a 'set missing' retry flow."],"tags":["evals","not-found","filesystem","text-set"],"backgroundTag":null,"analyzedSha":"d026163f586dfa8c5c10d28c36edd59a9d3b0e88","analyzedAt":"2026-08-14T22:02:06.951Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}