abi/screenshot-to-code · error · HTTPException

Error processing evals: {str(e)}

Error message

Error processing evals: {str(e)}

What it means

500 raised by GET /evals in backend/routes/evals.py:104 as a catch-all for any exception while scanning the folder for .html files, reading output HTML, or converting input images to data URLs (image_to_data_url). The detail embeds the underlying exception string, which is the real diagnostic.

Source

Thrown at backend/routes/evals.py:104

                continue

            # Find matching output file
            output_file = None
            for filename, filepath in files.items():
                if filename.startswith(base_name):
                    output_file = filepath
                    break

            if output_file:
                input_data = await image_to_data_url(input_path)
                with open(output_file, "r", encoding="utf-8") as f:
                    output_html = f.read()
                evals.append(Eval(input=input_data, outputs=[output_html]))

        return evals

    except Exception as e:
        raise HTTPException(status_code=500, detail=f"Error processing evals: {str(e)}")


class RunEvalsRequest(BaseModel):
    models: List[str]
    stack: Stack
    files: List[str] = []  # Optional list of specific file paths to run evals on
    diff_mode: bool = False
    # When set, inputs come from {EVALS_DIR}/sets/{set_name}/inputs and runs
    # attach to the active eval session (auto-created when none exists).
    set_name: Optional[str] = None


def _resolve_set_run(
    request: RunEvalsRequest,
) -> tuple[Optional[str], Optional[eval_sessions.EvalSession], Dict[str, set[str]]]:
    """Validate the requested set and resolve the session + per-model skips."""
    if not request.set_name:
        return None, None, {}

View on GitHub (pinned to d026163f58)

Solutions

1. Read the detail field of the 500 response — it contains the original exception message (e.g. FileNotFoundError with the exact path).
2. Fix the named file/directory: restore the missing png/html pair or permissions.
3. Re-run after ensuring the folder contents are stable (no cleanup job running concurrently).

try {
  const evals = await fetch(`/evals?folder=${encodeURIComponent(folder)}`).then(r => {
    if (!r.ok) throw new Error(`HTTP ${r.status}`);
    return r.json();
  });
} catch (e) {
  // detail field carries the root-cause exception; surface it verbatim
  console.error('Eval scan failed:', e.message);
}

When it happens

Trigger: The folder stops existing mid-scan (deleted between the exists() check and listdir), an .html output file is unreadable (permissions), or an input .png referenced by the pairing logic is missing/corrupt so image_to_data_url raises.

Common situations: Concurrent cleanup deleting eval outputs while the endpoint iterates; mismatched input/output basenames causing a missing-file read; unreadable files under restrictive permissions.

Related errors

• Error reading input files: {str(e)}
• Error reading output folders: {str(e)}
• Folder not found: {folder}
• Text eval set not found: {set_name}
• Unreadable briefs.json for {set_name}: {exc}

AI-assisted analysis of abi/screenshot-to-code@d026163f58 (2026-08-14). Data as JSON: /api/errors/e823ab678be71e26. Report an issue: GitHub.