langchain-ai/deepagents · error · VideoExtractionError

Video decoding exceeded the {MAX_VIDEO_DECODE_SECONDS:.1f}s

Error message

Video decoding exceeded the {MAX_VIDEO_DECODE_SECONDS:.1f}s safety budget

What it means

Video decoding is best-effort with a wall-clock budget of `MAX_VIDEO_DECODE_SECONDS`. `_check_decode_deadline` raises `VideoExtractionError` when `time.monotonic()` passes the deadline, protecting the agent from pathological inputs (huge files, slow codecs) that would hang extraction.

Source

Thrown at libs/deepagents/deepagents/middleware/_video.py:298

    if width is None or height is None:
        return None
    return int(width), int(height)


def _validate_dimensions(width: int, height: int) -> None:
    """Reject frame dimensions that are too large to safely convert."""
    if width <= 0 or height <= 0:
        return
    if width > MAX_VIDEO_FRAME_SIDE or height > MAX_VIDEO_FRAME_SIDE:
        msg = f"Video frame dimensions {width}x{height} exceed the maximum {MAX_VIDEO_FRAME_SIDE}px side"
        raise VideoExtractionError(msg)


def _check_decode_deadline(deadline_seconds: float | None) -> None:
    """Raise when best-effort video decoding has exceeded its time budget."""
    if deadline_seconds is not None and time.monotonic() > deadline_seconds:
        msg = f"Video decoding exceeded the {MAX_VIDEO_DECODE_SECONDS:.1f}s safety budget"
        raise VideoExtractionError(msg)


def _sample_frames_in_window(
    decoded_frames: Any,  # noqa: ANN401  # PyAV types are unavailable without the [video] extra
    *,
    offset_seconds: float,
    duration_seconds: float,
    sampling_rate: float,
    time_base: float,
    stream_start_seconds: float = 0.0,
    deadline_seconds: float | None = None,
    decode_error_types: tuple[type[BaseException], ...] = (),
) -> list[ContentBlock]:
    """Pick JPEG+timestamp content blocks for frames inside the requested window."""
    frame_interval_seconds = 1.0 / sampling_rate
    end_seconds = offset_seconds + float(duration_seconds)
    next_emit_seconds = offset_seconds
    blocks: list[ContentBlock] = []

View on GitHub (pinned to a1af029e6e)

Solutions

Extract a shorter window (smaller `duration_seconds`) or lower `sampling_rate`
Pre-transcode/compress the video to a smaller resolution before extraction
Catch `VideoExtractionError` and fall back to metadata-only or thumbnail handling
Run extraction in a worker pool so the budget miss doesn't block the agent

Example fix

// before
frames = extract_video_frames(two_hour_recording, offset_seconds=0, duration_seconds=7200, sampling_rate=2)
// after
frames = extract_video_frames(two_hour_recording, offset_seconds=start, duration_seconds=60, sampling_rate=0.5)

Defensive patterns

Strategy: validation

Validate before calling

duration = get_video_duration(content)  # via ffprobe/PyAV
if duration > MAX_REASONABLE_DURATION:
    raise ValueError("video too long for inline extraction")

Try / catch

try:
    result = extract_video_frames(content, offset_seconds=0, duration_seconds=window, sampling_rate=1)
except VideoExtractionError as exc:
    if "safety budget" in str(exc):
        result = extract_video_frames(content, offset_seconds=0, duration_seconds=window / 4, sampling_rate=0.5)

Prevention

Keep duration/sampling_rate modest for long recordings
Extract in chunks instead of one huge window
Run extraction off the main agent loop with a timeout

When it happens

Trigger: Sampling a long or high-fps video whose decode loop exceeds the safety budget; very slow host CPU; deeply coded streams that decode few usable frames per second.

Common situations: Hour-long screen recordings sampled at high rates; containerized environments with throttled CPU; corrupt streams that make the decoder spin.

Related errors

AI-assisted analysis of langchain-ai/deepagents@a1af029e6e (2026-08-29). Data as JSON: /api/errors/93a5225fa470a4da. Report an issue: GitHub.