abi/screenshot-to-code · warning · HTTPException

Active session {e.active_session.name!r} is pinned to set {e

Error message

Active session {e.active_session.name!r} is pinned to set {e.active_session.eval_set!r}. Start a new session for set {request.set_name!r} first (POST /eval-sessions).

What it means

400 raised inside _resolve_set_run in backend/routes/evals.py:138 when eval_sessions.resolve_session_for_run raises SessionSetMismatchError: there is an active eval session, but it is pinned to a different eval set. Sessions are set-scoped, so running a new set requires a fresh session. The detail names the active session, its pinned set, and the required remedy (POST /eval-sessions).

Source

Thrown at backend/routes/evals.py:138

    """Validate the requested set and resolve the session + per-model skips."""
    if not request.set_name:
        return None, None, {}
    try:
        set_info = eval_sets.get_set(request.set_name)
    except eval_sets.InvalidSetNameError as e:
        raise HTTPException(status_code=400, detail=str(e))
    except eval_sets.EvalSetNotFoundError:
        raise HTTPException(
            status_code=404, detail=f"Eval set not found: {request.set_name}"
        )
    if set_info.image_count == 0:
        raise HTTPException(
            status_code=400, detail=f"Eval set {request.set_name!r} has no images"
        )
    try:
        session = eval_sessions.resolve_session_for_run(request.set_name)
    except eval_sessions.SessionSetMismatchError as e:
        raise HTTPException(
            status_code=400,
            detail=(
                f"Active session {e.active_session.name!r} is pinned to set "
                f"{e.active_session.eval_set!r}. Start a new session for set "
                f"{request.set_name!r} first (POST /eval-sessions)."
            ),
        )
    skip_per_model: Dict[str, set[str]] = {}
    if request.diff_mode:
        for model in request.models:
            skip_per_model[model] = eval_sessions.completed_eval_inputs(
                session.session_id, model, str(request.stack)
            )
    return request.set_name, session, skip_per_model


class OpenAIInputCompareRequest(BaseModel):
    left_json: str

View on GitHub (pinned to d026163f58)

Solutions

POST /eval-sessions with the new set to start a fresh session, then re-issue the run request.
Or switch the run back to the set the active session is pinned to.
In the UI, auto-create a new session whenever the selected set differs from the active session's set.

Example fix

// before
await post('/run_evals_stream', { set_name: newSetName, ... });

// after
if (activeSession?.eval_set !== newSetName) {
  activeSession = await post('/eval-sessions', { eval_set: newSetName });
}
await post('/run_evals_stream', { set_name: newSetName, ... });

Defensive patterns

Strategy: validation

Validate before calling

if (activeSession && activeSession.eval_set !== requestedSet) {
  activeSession = await fetch('/eval-sessions', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ eval_set: requestedSet }),
  }).then(r => r.json());
}

Try / catch

const res = await fetch('/run_evals_stream', ...);
if (res.status === 400 && detail.includes('pinned to set')) {
  await createNewSession(requestedSet);
  await retryRun();
}

Prevention

Track the active session's eval_set in client state and compare before every run.
Switching sets in the UI should always imply creating a new session.

When it happens

Trigger: Starting a run with set_name=B while the active session was created for set_name=A.

Common situations: User finishes work on one set and switches the set dropdown without starting a new session; long-lived frontend keeps an old active session across set changes.

Related errors

AI-assisted analysis of abi/screenshot-to-code@d026163f58 (2026-08-14). Data as JSON: /api/errors/babc8eee7674dc89. Report an issue: GitHub.