abi/screenshot-to-code · error · SessionSetMismatchError

Active session {active_session.name!r} is pinned to set {act

Error message

Active session {active_session.name!r} is pinned to set {active_session.eval_set!r}, not {requested_set!r}

What it means

SessionSetMismatchError raised by resolve_session_for_run when an active eval session exists but its eval_set differs from the requested set. By design, one active session per set: a mismatched active session is an error so runs never silently attach to the wrong session; the caller must explicitly start a new session. The f-string interpolates the active session's name, its set, and the requested set.

Source

Thrown at backend/evals/sessions.py:162

        except sqlite3.IntegrityError:
            _activate_in_transaction(conn, session_id)
    finally:
        conn.close()
    return get_session(session_id)


def resolve_session_for_run(set_name: str) -> EvalSession:
    """The session eval runs should attach to for this set.

    Active session on the same set wins; a mismatched active session is an
    error (the caller must explicitly start a new session); no session at
    all auto-creates one so a bare "/run_evals with a set" just works.
    """
    active = get_active_session()
    if active is None:
        return create_session(set_name)
    if active.eval_set != set_name:
        raise SessionSetMismatchError(active, set_name)
    return active


@dataclass
class SessionModelMeta:
    position: Optional[int]
    notes: str


def get_model_meta(session_id: str) -> dict[str, SessionModelMeta]:
    conn = open_index_db()
    try:
        rows = conn.execute(
            "SELECT model, position, notes FROM eval_session_models "
            "WHERE session_id = ?",
            (session_id,),
        ).fetchall()
    finally:

View on GitHub (pinned to d026163f58)

Solutions

1. Start a new session for the target set (POST /eval-sessions) or deactivate/finish the current active session, then retry the run.
2. Catch SessionSetMismatchError and surface 'finish or start a new session' to the user instead of a raw traceback.
3. For automation, always create a session per set instead of relying on the ambient active one.

Example fix

# before
resolve_session_for_run("set-b")  # active session is on set-a -> raises

# after
finish_active_session()  # or create a new session via the UI/API
resolve_session_for_run("set-b")

active = get_active_session()
if active is not None and active.eval_set != requested_set:
    raise SessionSetMismatchError(active, requested_set)  # caller decides: finish or new session

try:
    session = resolve_session_for_run(set_name)
except SessionSetMismatchError as e:
    prompt_user_to_start_new_session(set_name)

When it happens

Trigger: Activating session for set A, then running evals for set B via /run_evals with a set argument (or the eval sessions page) without deactivating/starting a new session first.

Common situations: Switching between eval sets mid-session, forgotten active session from a previous run, or automation that assumes a fresh context per invocation.

Related errors

• No stack was provided
• No model was provided
• Invalid eval set name: {set_name!r}
• Invalid brief entry in {set_name}: id={brief_id!r}
• Not a set image: {filename!r}

AI-assisted analysis of abi/screenshot-to-code@d026163f58 (2026-08-14). Data as JSON: /api/errors/3b6f4b015f34b059. Report an issue: GitHub.