abi/screenshot-to-code · error · HTTPException

Not a set image: {filename!r}

Error message

Not a set image: {filename!r}

What it means

Raised by evals.sets.resolve_set_image_path() (backend/evals/sets.py:297) and surfaced as 400 by GET /eval-sets/{set_name}/images/{filename}. After basename() stripping, the filename must end in '.png' (case-insensitive); any other extension or bare name is rejected as 'not a set image'. Set images are PNG-only by contract.

Source

Thrown at backend/routes/eval_sets.py:175

        **_set_info_to_model(info).model_dump(),
        images=[
            EvalSetImageModel(
                filename=image.filename,
                sha256=image.sha256,
                size_bytes=image.size_bytes,
                tags=image.tags,
            )
            for image in images
        ],
    )


@router.get("/eval-sets/{set_name}/images/{filename}")
async def get_eval_set_image(set_name: str, filename: str) -> FileResponse:
    try:
        path = eval_sets.resolve_set_image_path(set_name, filename)
    except eval_sets.InvalidSetNameError as e:
        raise HTTPException(status_code=400, detail=str(e))
    except eval_sets.EvalSetNotFoundError:
        raise HTTPException(status_code=404, detail=f"Eval set not found: {set_name}")
    except FileNotFoundError:
        raise HTTPException(status_code=404, detail="Image not found")
    return FileResponse(path)


@router.get("/eval-sessions", response_model=EvalSessionListResponse)
async def list_eval_sessions() -> EvalSessionListResponse:
    sessions = eval_sessions.list_sessions()
    active = next((s for s in sessions if s.is_active), None)
    return EvalSessionListResponse(
        sessions=[_session_to_model(s) for s in sessions],
        active_session_id=active.session_id if active else None,
    )


@router.get("/eval-sessions/active", response_model=Optional[EvalSessionModel])

View on GitHub (pinned to d026163f58)

Solutions

1. Request only .png files that the set actually contains (list them via GET /eval-sets/{set_name}).
2. Convert non-PNG images to PNG when adding them to a set's inputs directory.
3. Send the bare filename without directories.

Example fix

# before
GET /eval-sets/my-set/images/shot.jpg  # 400 'Not a set image'

# after
GET /eval-sets/my-set/images/shot.png  # PNG files only

import os

def is_png_filename(filename: str) -> bool:
    return os.path.basename(filename).lower().endswith(".png")

if not is_png_filename(filename):
    raise ValueError(f"only .png images are servable: {filename!r}")

def is_set_image_name(filename: object) -> bool:
    return (
        isinstance(filename, str)
        and filename == os.path.basename(filename)
        and filename.lower().endswith(".png")
    )

resp = requests.get(url + f"/eval-sets/{set_name}/images/{filename}")
if resp.status_code == 400 and "Not a set image" in resp.text:
    raise ValueError("images must be bare .png filenames")
resp.raise_for_status()

When it happens

Trigger: GET /eval-sets/{set}/images/{filename} where filename is 'shot.jpg', 'capture.webp', 'manifest.json', or has no extension. Note the check happens before the existence check, so this fires even for non-existent non-PNG names.

Common situations: Frontend builds image URLs from filenames with non-PNG extensions; users drop JPGs into the inputs directory expecting them to be served; probing for arbitrary files (e.g. manifest.json) through the image route.

Related errors

• Invalid eval set name: {set_name!r}
• Invalid eval set name: {set_name!r}
• Eval set {request.set_name!r} has no images
• Invalid run id
• max_age_days must be >= 1

AI-assisted analysis of abi/screenshot-to-code@d026163f58 (2026-08-14). Data as JSON: /api/errors/79b229dcab49d920. Report an issue: GitHub.