affaan-m/ECC · error · RuntimeError

Image not found on screen: {template_path}

Error message

Image not found on screen: {template_path}

What it means

A RuntimeError raised by click_image() when find_image_on_screen() returns None — i.e. cv2.matchTemplate could not find the template image anywhere on screen at or above the configured confidence threshold. The helper uses pyautogui for the screenshot and OpenCV TM_CCOEFF_NORMED matching.

Source

Thrown at skills/windows-desktop-e2e/SKILL.md:780

def find_image_on_screen(template_path, confidence=0.85):
    """Locate a template image on screen. Returns (x, y) center or None."""
    screen   = np.array(pyautogui.screenshot())
    template = np.array(Image.open(template_path))
    result   = cv2.matchTemplate(
        cv2.cvtColor(screen, cv2.COLOR_RGB2BGR),
        cv2.cvtColor(template, cv2.COLOR_RGB2BGR),
        cv2.TM_CCOEFF_NORMED,
    )
    _, max_val, _, max_loc = cv2.minMaxLoc(result)
    if max_val >= confidence:
        h, w = template.shape[:2]
        return max_loc[0] + w // 2, max_loc[1] + h // 2
    return None

def click_image(template_path, confidence=0.85):
    pos = find_image_on_screen(template_path, confidence)
    if pos is None:
        raise RuntimeError(f"Image not found on screen: {template_path}")
    pyautogui.click(*pos)
```

### DPI / Scaling Rules (screenshot mode only)

Screenshot matching is brutally sensitive to Windows display scaling (100% / 125% / 150%). Three hard rules:

1. **Capture templates at the same scale as the target machine.** Don't try to rescue a mismatch with `PIL.Image.resize` — `cv2.matchTemplate` is very fragile against resampling artefacts.
2. **Pin the CI display scaling.** On `windows-latest` add a step like `Set-DisplayResolution 1920 1080 -Force` and disable per-monitor DPI scaling, so screenshot dimensions are reproducible.
3. **Record the scale alongside each artefact.** On capture, write `GetDpiForWindow(hwnd) / 96` to `artifacts/<test>/metadata.json` — postmortems become obvious instead of guess-work.

> Process-level DPI awareness (`SetProcessDpiAwarenessContext`) **can conflict with Qt's own DPI handling** when the app under test is Qt-based. Prefer "same-scale templates + CI pin" over flipping process-wide DPI mode in fixtures.

### Debugging Match Confidence

When tuning the `confidence` threshold, the only sane workflow is to **see** where the match landed. The helper below is diagnosis-only — do not call it from test code.

```python

View on GitHub (pinned to 01e15490f0)

Solutions

1. Run the debug_match() helper to see the best score and where it landed — tune confidence against real data, not by guessing.
2. Re-capture the template on a machine with the same display scaling as the target (pin CI scaling with Set-DisplayResolution).
3. Lower confidence incrementally (e.g. 0.85 -> 0.80) only after confirming the match location is correct via debug_match.
4. Ensure the app window is foregrounded and fully rendered before calling click_image.
5. Prefer UIA-based selectors over screenshot matching where possible — screenshot matching is fragile to scaling and theme changes.

Example fix

# before
pos = find_image_on_screen(template_path, confidence)
if pos is None:
    raise RuntimeError(f"Image not found on screen: {template_path}")

# after: capture a diagnostic screenshot and surface the best score
score, pos = find_image_on_screen(template_path, confidence, return_score=True)
if pos is None:
    debug_match(template_path, out=f"artifacts/{Path(template_path).stem}_miss.png", confidence=confidence)
    raise RuntimeError(f"Image not found (best score {score:.3f} < {confidence}): {template_path}")

def image_likely_on_screen(template_path, confidence=0.85) -> bool:
    return find_image_on_screen(template_path, confidence) is not None

if not image_likely_on_screen(template_path, confidence):
    debug_match(template_path, out="artifacts/miss.png", confidence=confidence)
    raise RuntimeError(f"template not visible; see artifacts/miss.png")

null

try:
    click_image(template_path, confidence=0.85)
except RuntimeError as e:
    debug_match(template_path, out="artifacts/click_miss.png", confidence=0.85)
    raise

When it happens

Trigger: Calling click_image(template_path, confidence) where the template does not visually match the current screen. Causes include: confidence threshold too high, template captured at a different DPI/scale than the runtime screen, the UI element is occluded or off-screen, or the template file references stale artwork.

Common situations: Windows display scaling mismatch (100% vs 125% vs 150%) between capture and CI machines; the app updated its icons; the window was not foregrounded; anti-aliasing/font differences between capture and run environment.

Related errors

• Condition not met within {timeout}s
• Template unreadable: {template_path}
• File path contains unsafe shell characters
• Formatter exited with status ${result.status}
• Claude Code command contains characters that are unsafe for

AI-assisted analysis of affaan-m/ECC@01e15490f0 (2026-08-13). Data as JSON: /api/errors/6850e7eeeac8796c. Report an issue: GitHub.