microsoft/autogen · error · IOError

Failed to capture frame at {timestamp:.2f}s

Error message

Failed to capture frame at {timestamp:.2f}s

What it means

Raised by save_screenshot() when the video opened successfully and the capture position was set to int(timestamp * fps), but cap.read() returned ret=False — no frame came back at that position. Common causes: timestamp beyond the actual (vs. reported) stream length, a variable-frame-rate file where CAP_PROP_POS_FRAMES seeks imprecisely, or a truncated/corrupt file whose header frame count is wrong.

Source

Thrown at python/packages/autogen-ext/src/autogen_ext/agents/video_surfer/tools.py:102

def save_screenshot(video_path: str, timestamp: float, output_path: str) -> None:
    """
    Captures a screenshot at the specified timestamp and saves it to the output path.

    :param video_path: Path to the video file.
    :param timestamp: Timestamp in seconds.
    :param output_path: Path to save the screenshot. The file format is determined by the extension in the path.
    """
    cap = cv2.VideoCapture(video_path)
    if not cap.isOpened():
        raise IOError(f"Cannot open video file {video_path}")
    fps = cap.get(cv2.CAP_PROP_FPS)
    frame_number = int(timestamp * fps)
    cap.set(cv2.CAP_PROP_POS_FRAMES, frame_number)
    ret, frame = cap.read()
    if ret:
        cv2.imwrite(output_path, frame)
    else:
        raise IOError(f"Failed to capture frame at {timestamp:.2f}s")
    cap.release()


async def transcribe_video_screenshot(video_path: str, timestamp: float, model_client: ChatCompletionClient) -> str:
    """
    Transcribes the content of a video screenshot captured at the specified timestamp using OpenAI API.

    :param video_path: Path to the video file.
    :param timestamp: Timestamp in seconds.
    :param model_client: ChatCompletionClient instance.
    :return: Description of the screenshot content.
    """
    screenshots = get_screenshot_at(video_path, [timestamp])
    if not screenshots:
        return "Failed to capture screenshot."

    _, frame = screenshots[0]
    # Convert the frame to bytes and then to base64 encoding

View on GitHub (pinned to 027ecf0a37)

Solutions

1. Use a timestamp strictly less than the duration, e.g. min(timestamp, duration - 1.0/fps)
2. Clamp the frame number: frame_number = min(int(timestamp * fps), int(frame_count) - 1)
3. Re-encode the video to constant frame rate with ffmpeg -vsync cfr to make seeks reliable
4. Verify the file is fully downloaded/uncorrupted (ffprobe reports no errors)

Example fix

// before
save_screenshot(video_path, duration, 'last.png')  # seek past end -> IOError

// after
fps = cv2.VideoCapture(video_path).get(cv2.CAP_PROP_FPS)
frames = cv2.VideoCapture(video_path).get(cv2.CAP_PROP_FRAME_COUNT)
safe_ts = min(timestamp, (frames - 1) / fps)
save_screenshot(video_path, safe_ts, 'last.png')

import cv2
def clamp_timestamp(video: str, ts: float) -> float:
    cap = cv2.VideoCapture(video)
    fps = cap.get(cv2.CAP_PROP_FPS) or 30.0
    frames = cap.get(cv2.CAP_PROP_FRAME_COUNT)
    cap.release()
    return max(0.0, min(ts, (frames - 1) / fps))

try:
    save_screenshot(video, ts, out)
except IOError:
    save_screenshot(video, max(0.0, ts - 0.5), out)  # retry slightly earlier frame

When it happens

Trigger: Calling save_screenshot(video, timestamp) with a timestamp at or near the reported duration (e.g. duration exactly), seeking in a variable-frame-rate video, or reading a partially downloaded file whose metadata over-reports frame count.

Common situations: Agents computing a timestamp from get_video_length() output (frame_count/fps can overestimate), taking a screenshot of the final frame, handling webcam/DVR streams with unreliable CAP_PROP_FRAME_COUNT.

Related errors

• Cannot open video file {video_path}
• Timestamp {timestamp:.2f}s is out of range [0s, {duration:.2
• video_path must be a local file path, not a URL.
• audio_output_path must end with .mp3.
• audio_output_path must be within the current working directo

AI-assisted analysis of microsoft/autogen@027ecf0a37 (2026-08-15). Data as JSON: /api/errors/237a40c6bcdee5bb. Report an issue: GitHub.