abi/screenshot-to-code · error · EvalSetNotFoundError

Unreadable briefs.json for {set_name}: {exc}

Error message

Unreadable briefs.json for {set_name}: {exc}

What it means

EvalSetNotFoundError raised when briefs.json exists but cannot be read or parsed — OSError (permissions, deleted mid-read, I/O error) or json.JSONDecodeError (malformed JSON). The original exception is interpolated into the message so the root cause is visible. It maps 'corrupted file' onto the same not-found semantics: the briefs are unusable either way.

Source

Thrown at backend/evals/sets.py:106

_BRIEF_ID_PATTERN = re.compile(r"^[a-z0-9][a-z0-9-]*$")


def get_set_kind(set_name: str) -> str:
    """"text" when the set is a briefs.json collection, else "image"."""
    if os.path.isfile(_briefs_path(set_name)):
        return "text"
    return "image"


def list_set_briefs(set_name: str) -> list[EvalSetBrief]:
    path = _briefs_path(set_name)
    if not os.path.isfile(path):
        raise EvalSetNotFoundError(f"Text eval set not found: {set_name}")
    try:
        with open(path, "r", encoding="utf-8") as f:
            loaded = cast(dict[str, Any], json.load(f))
    except (OSError, json.JSONDecodeError) as exc:
        raise EvalSetNotFoundError(f"Unreadable briefs.json for {set_name}: {exc}")
    briefs: list[EvalSetBrief] = []
    raw_briefs = loaded.get("briefs")
    entries = (
        cast(list[object], raw_briefs) if isinstance(raw_briefs, list) else []
    )
    for entry in entries:
        if not isinstance(entry, dict):
            continue
        record = cast(dict[str, Any], entry)
        brief_id = str(record.get("id") or "")
        brief = str(record.get("brief") or "")
        if not _BRIEF_ID_PATTERN.match(brief_id) or not brief:
            raise InvalidSetNameError(
                f"Invalid brief entry in {set_name}: id={brief_id!r}"
            )
        briefs.append(
            EvalSetBrief(
                id=brief_id,

View on GitHub (pinned to d026163f58)

Solutions

1. Validate the file: python -m json.tool briefs.json shows the exact parse location.
2. Fix structure to {"briefs": [{"id": "...", "brief": "..."}, ...]}.
3. Check file permissions if OSError: the backend process needs read access.
4. Write briefs.json atomically (temp file + rename) when generating it programmatically.

Example fix

// before
{ 'briefs': [ { id: 'a1', brief: 'landing page' } ] }

// after
{
  "briefs": [
    { "id": "a1", "brief": "landing page" }
  ]
}

import json, pathlib
def briefs_loadable(path: str) -> bool:
    p = pathlib.Path(path)
    if not p.is_file():
        return False
    try:
        json.loads(p.read_text(encoding="utf-8"))
        return True
    except (OSError, json.JSONDecodeError):
        return False

import json

def is_valid_briefs_payload(data: unknown) -> data is {"briefs": list}:
    return (
        typeof data === "object" and data !== null and
        Array.isArray((data as any).briefs)
    );

try:
    briefs = list_set_briefs(set_name)
except EvalSetNotFoundError as e:
    log.warning("briefs.json problem for %s: %s", set_name, e)
    return HTTPException(status_code=422, detail=f"Corrupt briefs.json: {e}")

When it happens

Trigger: briefs.json with trailing commas, single quotes, truncation from an interrupted write, BOM/encoding issues, or file permissions denying read to the server process.

Common situations: Hand-edited briefs.json with invalid JSON, concurrent writers (set edit + eval read), file synced/uploaded with corruption, or read-only mount.

Related errors

• Text eval set not found: {set_name}
• Invalid brief entry in {set_name}: id={brief_id!r}
• Eval set not found: {set_name}
• Design systems storage is not valid JSON
• Design systems storage must contain a list

AI-assisted analysis of abi/screenshot-to-code@d026163f58 (2026-08-14). Data as JSON: /api/errors/5aa685095ee97e56. Report an issue: GitHub.