{"record":{"id":"37e3c42008f52e5d","repo":"abi/screenshot-to-code","slug":"invalid-eval-set-name-set-name-r-37e3c4","errorCode":null,"errorMessage":"Invalid eval set name: {set_name!r}","messagePattern":"Invalid eval set name: (.+?)","errorType":"http","errorClass":"HTTPException","httpStatus":400,"severity":"warning","filePath":"backend/routes/evals.py","lineNumber":126,"sourceCode":"    models: List[str]\n    stack: Stack\n    files: List[str] = []  # Optional list of specific file paths to run evals on\n    diff_mode: bool = False\n    # When set, inputs come from {EVALS_DIR}/sets/{set_name}/inputs and runs\n    # attach to the active eval session (auto-created when none exists).\n    set_name: Optional[str] = None\n\n\ndef _resolve_set_run(\n    request: RunEvalsRequest,\n) -> tuple[Optional[str], Optional[eval_sessions.EvalSession], Dict[str, set[str]]]:\n    \"\"\"Validate the requested set and resolve the session + per-model skips.\"\"\"\n    if not request.set_name:\n        return None, None, {}\n    try:\n        set_info = eval_sets.get_set(request.set_name)\n    except eval_sets.InvalidSetNameError as e:\n        raise HTTPException(status_code=400, detail=str(e))\n    except eval_sets.EvalSetNotFoundError:\n        raise HTTPException(\n            status_code=404, detail=f\"Eval set not found: {request.set_name}\"\n        )\n    if set_info.image_count == 0:\n        raise HTTPException(\n            status_code=400, detail=f\"Eval set {request.set_name!r} has no images\"\n        )\n    try:\n        session = eval_sessions.resolve_session_for_run(request.set_name)\n    except eval_sessions.SessionSetMismatchError as e:\n        raise HTTPException(\n            status_code=400,\n            detail=(\n                f\"Active session {e.active_session.name!r} is pinned to set \"\n                f\"{e.active_session.eval_set!r}. Start a new session for set \"\n                f\"{request.set_name!r} first (POST /eval-sessions).\"\n            ),","sourceCodeStart":108,"sourceCodeEnd":144,"githubUrl":"https://github.com/abi/screenshot-to-code/blob/d026163f586dfa8c5c10d28c36edd59a9d3b0e88/backend/routes/evals.py#L108-L144","documentation":"400 raised inside _resolve_set_run in backend/routes/evals.py:126 when request.set_name fails eval_sets.get_set() with InvalidSetNameError. That error comes from the eval-set store when the name does not match the allowed naming scheme, so the set name is malformed before any on-disk lookup happens.","triggerScenarios":"POST /run_evals_stream (or /run_evals) with set_name containing characters outside the allowed set-name pattern, e.g. '../escape', spaces, or empty-ish strings that still parse as invalid.","commonSituations":"Frontend passes a user-typed set name without sanitizing; scripts constructing set names from file paths; traversal-style names rejected by the store's validation regex.","solutions":["Use a set name created via the eval-set management routes (POST /eval-sets) — those names are known-valid.","Sanitize the name to the allowed pattern (typically lowercase letters, digits, dashes/underscores) before sending.","Read str(e) in the response detail: it states the expected naming rule."],"exampleFix":"// before\nbody: JSON.stringify({ set_name: rawUserInput, ... })\n\n// after\nconst setName = rawUserInput.trim().toLowerCase().replace(/[^a-z0-9-_]/g, '-');\nif (!/^[a-z0-9][a-z0-9-_]*$/.test(setName)) throw new Error('Invalid set name');\nbody: JSON.stringify({ set_name: setName, ... })","handlingStrategy":"validation","validationCode":"const SET_NAME_RE = /^[a-z0-9][a-z0-9-_]*$/;\nif (!SET_NAME_RE.test(setName)) throw new Error(`Invalid set name: ${setName}`);","typeGuard":"function isValidSetName(name: string): boolean {\n  return /^[a-z0-9][a-z0-9-_]*$/.test(name);\n}","tryCatchPattern":"const res = await fetch('/run_evals_stream', ...);\nif (res.status === 400) { /* detail explains the naming rule; show it */ }","preventionTips":["Derive set names only from the eval-sets management endpoints.","Sanitize user input before it reaches a run request."],"tags":["http-400","validation","eval-sets","fastapi"],"backgroundTag":null,"analyzedSha":"d026163f586dfa8c5c10d28c36edd59a9d3b0e88","analyzedAt":"2026-08-14T22:02:06.951Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}